讀取文本文件時，GC開銷限制超出錯誤

Question

我得到java.lang.OutOfMemoryError：從文本文件讀取時，GC開銷限制超出錯誤。我不確定發生了什麼問題。我正在運行我的程序具有足夠的內存。外層循環迭代16000次，外層循環的每次迭代內層循環迭代大約300,000次。當代碼試圖從內層循環讀取一行時拋出錯誤。任何建議都會grately appreciated.The下面是我的代碼片段：

//Read from the test data output file till not equals null 
//Reads a single line at a time from the test data 
while((line=br.readLine())!=null) 
{ 
    //Clears the hashmap 
    leastFive.clear(); 

    //Clears the arraylist 
    fiveTrainURLs.clear(); 
    try 
    { 
     StringTokenizer st=new StringTokenizer(line," "); 
     while(st.hasMoreTokens()) 
     { 
      String currentToken=st.nextToken(); 

      if(currentToken.contains("File")) 
      { 
       testDataFileNo=st.nextToken(); 
       String tok=""; 
       while((tok=st.nextToken())!=null) 
       { 
        if (tok==null) break; 

        int topic_no=Integer.parseInt(tok); 
        topic_no=Integer.parseInt(tok); 
        String prob=st.nextToken(); 

        //Obtains the double value of the probability 
        double double_prob=Double.parseDouble(prob); 
        p1[topic_no]=double_prob; 

       } 
       break; 
      } 
     } 
    } 
    catch(Exception e) 
    { 
    } 

    //Used to read over all the training data file 
    FileReader fr1=new FileReader("/homes/output_train_2000.txt"); 

    BufferedReader br1=new BufferedReader(fr1); 
    String line1=""; 

    //Reads the training data output file,one row at a time 
    //This is the line on which an exception occurs! 
    while((line1=br1.readLine())!=null) 
    { 
     try 
     { 
      StringTokenizer st=new StringTokenizer(line1," "); 

      while(st.hasMoreTokens()) 
      { 
       String currentToken=st.nextToken(); 

       if(currentToken.contains("File")) 
       { 
        trainDataFileNo=st.nextToken(); 
        String tok=""; 
        while((tok=st.nextToken())!=null) 
        { 
         if(tok==null) 
          break; 

         int topic_no=Integer.parseInt(tok); 
         topic_no=Integer.parseInt(tok); 
         String prob=st.nextToken(); 

         double double_prob=Double.parseDouble(prob); 

         //p2 will contain the probability values of each of the topics based on the indices 
         p2[topic_no]=double_prob; 

        } 
        break; 
       } 
      } 
     } 
     catch(Exception e) 
     { 
      double result=klDivergence(p1,p2); 

      leastFive.put(trainDataFileNo,result); 
     } 
    } 
} 

Answer 1

16000 * 30萬= 4.8億元。如果每個令牌只佔用6個字節，本身就超過24GB。當垃圾收集器最終以24GB啓動到gc時，它將運行很長時間。好像你需要把它分解成更小的塊。您可以將您的應用內存限制在1GB等合理的範圍內，這樣GC就可以更快地開始工作，並在完成工作的時候完成某些工作。