2016-01-01 38 views
1

我有大量運行的並行進程。似乎僅僅創建平行scala期貨會造成內存泄漏。如何防止Scala Futures產生內存泄漏

下面的示例代碼。設置VM標誌取決於你的機器,所以它不會在開始時轉儲,因爲你有很多核心,「-Xmx100m -XX:+ HeapDumpOnOutOfMemoryError」。

import scala.concurrent.Future 
import scala.concurrent.ExecutionContext.Implicits.global 
import scala.concurrent.Await 
import scala.concurrent.duration._ 

object Bug extends App { 
    println("Memory leak") 
    while(true){ 
     Future { 
      val data = new Array[Byte](1000000*1) // 1 MB 
      println(".") 
     } 
    } 
} 

幾分鐘後,此代碼將顯着減慢。堆轉儲分析顯示數組scala.concurrent.forkjoin.ForkJoinTask []似乎溢出。

下面的鏈接似乎相關,但並沒有真正提供任何解決問題:https://issues.scala-lang.org/browse/SI-7336

+0

你使用的是什麼版本的scala和java? –

+0

Java版本「1.8.0_66」和Scala代碼亞軍版本2.11.7 –

+0

起初我以爲這可能是'println'擋住你的期貨最終將導致備份和隨後的內存不足的異常。但是如果沒有println,我仍然可以看到你提到的結果。我不相信這證明有泄漏。 **如果**的JVM是在產卵期貨比釋放/重複使用的內存不會有泄漏,但相同的異常會導致更快... –

回答

5

這裏發生的事情是,你要快得多分配內存,然後GC`ing它。

while循環的每個執行你把一個任務,線程池分配1MB,你是非常快的內存分配。

過了一會兒,GC是什麼減慢。

解決方案?分配內存較慢。

順便說一句,它會以相似的方式,即使你不聲明數組。當您運行Future時,您正在向線程池的隊列提交task,因此內存將以非常快的速度增長。

要看到自己,你可以運行jstat(http://www.cubrid.org/blog/dev-platform/how-to-monitor-java-garbage-collection/

jstat -gc $pid 1000

下面是從我的筆記本電腦的樣本,看看GCT柱(累計的總時間爲GC操作)如何增加

➜ ~ jstat -gc 24504 1000 
S0C S1C S0U S1U  EC  EU  OC   OU  MC  MU CCSC CCSU YGC  YGCT FGC FGCT  GCT 
35328.0 35840.0 22962.9 0.0 275968.0 0.0  395264.0 383080.0 11904.0 11387.3 1664.0 1545.4  30 1.333 3  2.125 3.458 
35328.0 35840.0 22962.9 0.0 275968.0 0.0  395264.0 383080.0 11904.0 11387.3 1664.0 1545.4  30 1.333 3  2.125 3.458 
35328.0 35840.0 22962.9 0.0 275968.0 0.0  395264.0 383080.0 11904.0 11387.3 1664.0 1545.4  30 1.333 3  2.125 3.458 
36352.0 36864.0 8148.0 0.0 275456.0 0.0  562688.0 533891.1 11904.0 11407.6 1664.0 1548.2  36 1.772 4  4.751 6.524 
36352.0 36864.0 8148.0 0.0 275456.0 0.0  562688.0 533891.1 11904.0 11407.6 1664.0 1548.2  36 1.772 4  4.751 6.524 
36352.0 36864.0 0.0 0.0 275456.0 143402.5 699392.0 486444.1 11904.0 11407.6 1664.0 1548.2  37 1.772 4  6.867 8.640 
32768.0 33792.0 22498.3 11986.9 278528.0 278251.8 699392.0 632212.1 11904.0 11437.5 1664.0 1549.8  47 2.548 4  6.867 9.415 
29696.0 31232.0 20498.9 0.0 284160.0 0.0  699392.0 678668.1 11904.0 11448.8 1664.0 1551.3  50 2.768 5  6.867 9.635 
29696.0 31232.0 20498.9 0.0 284160.0 0.0  699392.0 678668.1 11904.0 11448.8 1664.0 1551.3  50 2.768 5  6.867 9.635 
29696.0 31232.0 20498.9 0.0 284160.0 0.0  699392.0 678668.1 11904.0 11448.8 1664.0 1551.3  50 2.768 5  6.867 9.635 
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3  50 2.768 6  10.452 13.220 
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3  50 2.768 6  10.452 13.220 
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3  50 2.768 6  10.452 13.220 
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3  50 2.768 6  10.452 13.220 
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3  50 2.768 6  10.452 13.220 
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3  50 2.768 7  15.069 17.837 
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3  50 2.768 7  15.069 17.837 
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3  50 2.768 7  15.069 17.837 
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3  50 2.768 7  15.069 17.837 
29696.0 31232.0 0.0 0.0 284160.0 16821.1 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3  50 2.768 7  19.562 22.330 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3  50 2.768 8  19.562 22.330 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3  50 2.768 8  19.562 22.330 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3  50 2.768 8  19.562 22.330 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3  50 2.768 8  19.562 22.330 
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3  50 2.768 9  24.112 26.880 
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3  50 2.768 9  24.112 26.880 
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3  50 2.768 9  24.112 26.880 
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3  50 2.768 9  24.112 26.880 
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3  50 2.768 9  24.112 26.880 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3  50 2.768 10  28.728 31.496 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3  50 2.768 10  28.728 31.496 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3  50 2.768 10  28.728 31.496 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3  50 2.768 10  28.728 31.496 
29696.0 31232.0 0.0 0.0 284160.0 176242.5 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3  50 2.768 11  33.550 36.318 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3  50 2.768 11  33.550 36.318 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3  50 2.768 11  33.550 36.318 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3  50 2.768 11  33.550 36.318 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3  50 2.768 11  33.550 36.318 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3  50 2.768 11  33.550 36.318 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3  50 2.768 12  38.797 41.565 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3  50 2.768 12  38.797 41.565 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3  50 2.768 12  38.797 41.565 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3  50 2.768 12  38.797 41.565 
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3  50 2.768 12  38.797 41.565 
+0

已驗證,謝謝。也感謝@RamonJRomeroyVigil –