2016-04-20 16 views
0

我有一個servlet接收一個巨大的字符串(apprx 301695 length)作爲post參數。將一個巨大的String作爲post參數傳遞給一個servlet

每分鐘,一個.net應用程序都會向servlet發送這麼大的字符串。

最初我用得到如下字符串:

Line 1: String str = request.getParameter("data"); 

但是,3-4小時後。我得到以下異常:

java.lang.OutOfMemoryError: Java heap space 

然後我的註釋行:1。儘管,我的servlet代碼不接收字符串,我得到相同的異常如上所述。

請指導我。我應該如何處理這個問題?我已經閱讀了許多關於它的博客和文章,增加了堆大小和其他東西。但是,還沒有找到任何解決方案。

原代碼象下面這樣:

private String scanType = ""; 
private static final String path = "D:\\Mobile_scan_alerts"; 
private static final String stockFileName = "stock.txt"; 
private static final String foFileName = "fo.txt"; 
private static Logger logger = null; 
private String currDate = ""; 
private DateFormat dateFormat; 
private StringBuffer stockData; 
private StringBuffer foData; 
StringBuffer data = new StringBuffer(""); 
// For average time of received data 
private static float sum = 0; 
private static float count = 0; 
private static float s_sum = 0; 
private static float s_count = 0; 
private static float fo_sum = 0; 
private static float fo_count = 0; 

private static final File dir = new File(path); 
private static final File stockFile = new File(path + "\\" + stockFileName); 
private static final File foFile = new File(path + "\\" + foFileName); 

public void init() { 

    logger = MyLogger.getScanAlertLogger(); 

    if(logger == null) { 
     MyLogger.createLog(); 
     logger = MyLogger.getScanAlertLogger(); 
    } 

} 

/** 
* Processes requests for both HTTP <code>GET</code> and <code>POST</code> 
* methods. 
* 
* @param request servlet request 
* @param response servlet response 
* @throws ServletException if a servlet-specific error occurs 
* @throws IOException if an I/O error occurs 
*/ 
protected void processRequest(HttpServletRequest request, HttpServletResponse response) 
     throws ServletException, IOException { 

    PrintWriter out = response.getWriter(); 
    response.setContentType("text/plain"); 
    String strScan = ""; 

    try { 

     String asof = null; 

     scanType = request.getParameter("type"); 
     scanType = scanType == null ? "" : scanType; 


     if(scanType.length() > 0){ 

      if(scanType.equalsIgnoreCase("s")) { 
       stockData = null; 
       stockData = new StringBuffer(request.getParameter("scanData")); 
       stockData = stockData == null ? new StringBuffer("") : stockData; 
      } else { 
       foData = null; 
       foData = new StringBuffer(request.getParameter("scanData")); 
       foData = foData == null ? new StringBuffer("") : foData; 
      } 

     } 

     asof = request.getParameter("asof"); 
     asof = asof == null ? "" : asof.trim(); 

     // Date format without seconds 
     DateFormat formatWithoutSec = new SimpleDateFormat("yyyy/MM/dd HH:mm"); 

     dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss"); 
     Date tmp = new Date(); 

     // format: yyyy/MM/dd HH:mm:ss 
     currDate = dateFormat.format(tmp); 

     //format: yyyy/MM/dd HH:mm 
     Date asofDate = formatWithoutSec.parse(asof); 
     Date cDate = formatWithoutSec.parse(currDate); 
     cDate.setSeconds(0); 


     System.out.println(asofDate.toString()+" || "+cDate.toString()); 

     int isDataExpired = asofDate.toString().compareTo(cDate.toString()); 

     if(isDataExpired > 0 || isDataExpired == 0) { 

      if(scanType != null && scanType.length() > 0) { 
       checkAndCreateDir(); 
       strScan = scanType.equalsIgnoreCase("s") ? "Stock Data Recieved at "+currDate 
         : "FO Data Recieved at "+currDate; 
       //System.out.println(strScan); 
      } else { 
       strScan = "JSON of scan data not received properly at "+currDate; 
       //System.out.println("GSAS: received null or empty"); 
      } 

     } else { 
      strScan = "GSAS: " + scanType + ": Received Expired Data of "+asofDate.toString()+" at "+cDate.toString(); 
      System.out.println(strScan); 
     } 
     scanType = null; 


    } catch (Exception ex) { 
     strScan = "Mobile server issue for receiving scan data"; 
     System.out.println("GSAS: Exception-1: "+ex); 
     logger.error("GetScanAlertServlet: processRequest(): Exception: "+ex.toString()); 
    } finally { 
     logger.info("GetScanAlertServlet: "+strScan); 
     out.println(strScan); 
    } 

} 

private void checkAndCreateDir() { 

    try { 
      boolean isStock = false; 
      Date ddate = new Date(); 
      currDate = dateFormat.format(ddate); 
      sum += ddate.getSeconds(); 
      count++; 
      logger.info("Total Average Time: "+(sum/count)); 

      if(scanType.equalsIgnoreCase("s")){ //For Stock 
       setStockData(stockData); 

       Date date1 = new Date(); 
       currDate = dateFormat.format(date1); 
       s_sum += date1.getSeconds(); 
       s_count++; 
       logger.info("Stock Average Time: "+(s_sum/s_count)); 

       //file = new File(path + "\\" + stockFileName); 
       isStock = true; 

      } else if (scanType.equalsIgnoreCase("fo")) { //For FO 
       setFOData(foData); 

       Date date2 = new Date(); 
       currDate = dateFormat.format(date2); 
       fo_sum += date2.getSeconds(); 
       fo_count++; 
       logger.info("FO Average Time: "+(fo_sum/fo_count)); 

       //file = new File(path + "\\" +foFileName); 
       isStock = false; 
      } 


      if(!dir.exists()) { // Directory not exists 
       if(dir.mkdir()) { 

        if(isStock) 
         checkAndCreateFile(stockFile); 
        else 
         checkAndCreateFile(foFile); 

       } 
      } else { // Directory already exists 

        if(isStock) 
         checkAndCreateFile(stockFile); 
        else 
         checkAndCreateFile(foFile); 
      } 

    } catch (Exception e) { 
     System.out.println("GSAS: Exception-2: "+e); 
     logger.error("GetScanAlertServlet: checkAndCreateDir(): Exception: "+e); 
    } 

} 

private void checkAndCreateFile(File file) { 

    try{ 
     if(!file.exists()){ // File not exists 

      if(file.createNewFile()){ 
       writeToFile(file); 
      } 

     } else { // File already exists 
      writeToFile(file); 
     } 
    } catch (Exception e) { 
     System.out.println("GSAS: Exception-3: "+e); 
     logger.error("GetScanAlertServlet: checkAndCreateFile(): Exception: "+e.toString()); 
    } 
} 


private void writeToFile(File file) { 

    FileOutputStream fop = null; 

    try{ 

     if(scanType.equalsIgnoreCase("s")){ //For Stock 
      data = getStockData(); 
     } else if (scanType.equalsIgnoreCase("fo")) { //For FO 
      data = getFOData(); 
     } 

     if(data != null && data.length() > 0 && file != null){ 

      fop = new FileOutputStream(file); 

      byte[] contentBytes = data.toString().getBytes(); 
      for(byte b : contentBytes){ 
       fop.write(b); 
      } 
      //fop.write(contentBytes); 

      fop.flush(); 


     } else { 

      System.out.println("GSAS: Data is null/empty string"); 
      logger.info("GSAS: Data is null or empty string"); 

     } 
     data = null; 
    } catch (Exception e) { 
     System.out.println("GSAS: Exception-4: "+e); 
     logger.info("GetScanAlertServlet: writeToFile(): Exception: "+e.toString()); 

    } finally { 

     try { 

      if(fop != null) 
       fop.close(); 

     } catch (IOException ex) { 
       java.util.logging.Logger.getLogger(GetScanAlertServlet.class.getName()).log(Level.SEVERE, null, ex); 
     } 
    } 
} 

private String readFromFile(String fileName){ 

    String fileContent = ""; 
    try{ 

     String temp = ""; 
     File file = new File(fileName); 
     if(file.exists()){ 

      FileReader fr = new FileReader(file); 
      BufferedReader br = new BufferedReader(fr); 

      while((temp = br.readLine()) != null) 
      { 
       fileContent += temp; 
      } 
      br.close(); 
     } else { 
      System.out.println("GSAS: File not exists to read"); 
      logger.info("GetScanAlertServlet: File not exists to read"); 
     } 

     temp = null; 
     file = null; 

    } catch (Exception e) { 
     System.out.println("GSAS: Exception-5: "+e); 
     logger.error("GetScanAlertServlet: readFromFile(): Exception: "+e.toString()); 
    } 
    return fileContent; 
} 

public StringBuffer getStockData() { 

    //String temp=""; 
    //StringBuffer temp = (StringBuffer)scanDataSession.getAttribute("stock"); 
    //if(temp != null && temp.length() > 0) { 
    // return temp; 
    //} 
    if(stockData != null && stockData.length() > 0){ 
     return stockData; 
    } else { 
     stockData = null; 
     stockData = new StringBuffer(readFromFile(path + "\\"+ stockFileName)); 
     return stockData; 
    } 
} 


public StringBuffer getFOData(){ 

    //String temp=""; 
    //StringBuffer temp = (StringBuffer)scanDataSession.getAttribute("fo"); 
    //if(temp != null && temp.length() > 0) { 
    // return temp; 
    //} 
    if(foData != null && foData.length() > 0) { 
     return foData; 
    } else { 
     foData = null; 
     foData = new StringBuffer(readFromFile(path + "\\" + foFileName)); 
     return foData; 
    } 
} 

} 
+0

也許如果你複製stacktrace和相關的servlet代碼,有人可以幫助你。如果你評論了這一行,看起來你的問題與閱讀這個巨大的字符串無關, – RubioRic

+0

我使用visual vm來跟蹤堆大小。令人驚訝的是,儘管我已經評論過該行。它在一段時間後仍然顯示相同的錯誤。但是,發佈舊代碼: –

+0

我認爲你的Servlet不是線程安全的。或者它實現SingleThreadModel? – saka1029

回答

0

增加堆大小是不是這個問題的好辦法。您的上游應用程序應該停止向您的Servlet發送大量字符串。

您的上游(.net)應用程序應該考慮將所有數據寫入文件,只需要將文件的位置作爲參數發送到您的Servlet。一旦您的servlet收到來自上游的通知,就會考慮從該位置下載/讀取文件。

然後我的註釋行:1。儘管,我的servlet代碼不 接收字符串(如註釋),我得到相同的異常如 如上所述。

Line:1是讀取數據。如果你評論它,你不會收到字符串。

+0

由於時間緊迫,我們無法承擔下載文件的責任。實際上,我們收到的字符串是JSON字符串,我們必須掃描json對象以從中獲取數據。 –

+0

你目前的Xms和Xmx是什麼? –

+0

分別爲128m和1024m –

0

您可以使用Apache的commons-文件上傳庫流API,這樣一來,你會得到你上傳的文件作爲一個流,並將其寫入文件:

ServletFileUpload upload = new ServletFileUpload(); 

// Parse the request 
FileItemIterator iter = upload.getItemIterator(request); 
while (iter.hasNext()) { 
    FileItemStream item = iter.next(); 
    String name = item.getFieldName(); 
    InputStream stream = item.openStream(); 
    if (item.isFormField()) { 
     System.out.println("Form field " + name + " with value " 
      + Streams.asString(stream) + " detected."); 
    } else { 
     System.out.println("File field " + name + " with file name " 
      + item.getName() + " detected."); 
     // Process the input stream 
     ... 
    } 
} 

現在你有InputStream的,所以你可以把它寫在輸出流中。

但是爲了使用它,您需要.NET應用程序將字節上載到服務器,而不是將整個字符串作爲請求參數發送。

http://commons.apache.org/proper/commons-fileupload/streaming.html

0

請檢查您的VM參數和approriately修改它們,如果你有被傳遞給servlet沒有字符串的控制。例如:

set JAVA_OPTS=-Dfile.encoding=UTF-8 -Xms128m -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=256m 

檢查完整說明here

0

我們使用GZip壓縮/解壓縮來降低字符串的大小。它有效地工作。

因此,.net服務壓縮了巨大的字符串,將它發送到我們的Servlet。然後我們在我們的服務器上解壓縮它。

相關問題