我使用的HtmlUnit登錄到一個站點,然後從該表的OutOfMemoryError而使用的HtmlUnit拆解
下載數據,當我運行我的代碼是導致java.lang.OutOfMemoryError
,不能再運行。
以下是我的代碼:
WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_6);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setCssEnabled(false);
webClient.getOptions().setRedirectEnabled(true);
webClient.getCookieManager().setCookiesEnabled(true);
webClient.getOptions().setPrintContentOnFailingStatusCode(false);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setTimeout(50000);
webClient.getOptions().setUseInsecureSSL(true);
webClient.getOptions().setPopupBlockerEnabled(true);
HtmlPage htmlPage=webClient.getPage(url);
Thread.sleep(200);
//~~~~~~~Log-In
HtmlTextInput uname=(HtmlTextInput)htmlPage.getFirstByXPath("//*[@id=\"username\"]");
uname.setValueAttribute("xxx");
HtmlPasswordInput upass=(HtmlPasswordInput)htmlPage.getFirstByXPath("//*[@id=\"password\"]");
upass.setValueAttribute("xxx");
HtmlSubmitInput submit=(HtmlSubmitInput)htmlPage.getFirstByXPath("//*[@id=\"login-button\"]/input");
htmlPage=(HtmlPage) submit.click();
Thread.sleep(200);
webClient.waitForBackgroundJavaScript(10000);
for (int i = 0; i < 250; i++) {
if (!htmlPage.asText().contains("Loading...")) {
break;
}
synchronized (htmlPage) {
htmlPage.wait(500);
}
}
System.out.println(htmlPage.asText());
和以下是堆棧跟蹤
java.lang.OutOfMemoryError: Java heap space
at net.sourceforge.htmlunit.corejs.javascript.Node.newString(Node.java:155)
at net.sourceforge.htmlunit.corejs.javascript.Node.newString(Node.java:151)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.createPropertyGet(IRFactory.java:1990)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformPropertyGet(IRFactory.java:968)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:106)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformPropertyGet(IRFactory.java:964)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:106)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformPropertyGet(IRFactory.java:964)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:106)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformFunctionCall(IRFactory.java:595)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:86)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformInfix(IRFactory.java:775)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:161)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformAssignment(IRFactory.java:368)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:152)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformExprStmt(IRFactory.java:488)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:149)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformBlock(IRFactory.java:406)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:82)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformIf(IRFactory.java:762)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:110)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformBlock(IRFactory.java:406)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:82)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformIf(IRFactory.java:762)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:110)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformBlock(IRFactory.java:406)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:82)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformIf(IRFactory.java:768)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:110)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformBlock(IRFactory.java:406)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transform(IRFactory.java:82)
at net.sourceforge.htmlunit.corejs.javascript.IRFactory.transformFunction(IRFactory.java:560)
我已經把follwoing線catlina.sh文件配發的堆內存不過還是我收到了同樣的錯誤(我的RAM大小是2GB)。
if [ -z "$LOGGING_MANAGER" ]; then
JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
else
JAVA_OPTS="$JAVA_OPTS $LOGGING_MANAGER"
fi
# Uncomment the following line to make the umask available when using the
# org.apache.catalina.security.SecurityListener
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.security.SecurityListener.UMASK=`umask`"
JAVA_OPTS="$JAVA_OPTS -Xms512m -Xmx2048m -XX:MaxPermSize=512m"
JAVA_OPTS="-server -XX:+UseConcMarkSweepGC"
看着堆棧跟蹤,你可以看到,它的遞歸了很多。這似乎不是整個堆棧跟蹤 - 你截斷了多少? – 2013-04-08 10:58:14
我只把原因從堆棧跟蹤 – 2013-04-08 11:04:35
在這個堆棧跟蹤沒有你的代碼的跡象,這就是爲什麼我說你已經截斷了它。在某些時候,你的代碼應該有一行(或者一行代表類似... 1234更多行) – 2013-04-08 11:07:24