Glassfish 4灰熊線程沉重的CPU使用率

Question

我有一個在Glassfish 4（4.1版本13），JDK 1.7 update 67和AWS Linux AMI上運行的澤西島應用程序，我注意到在運行它幾個小時後，CPU使用率上升，即使客戶停下來也會保持住。

運行「top -H」標識具有較高CPU使用率的2個http-listener-1內核線程（總數爲16）。然後，我花了線程轉儲來檢查這兩個線程：

"http-listener-1-kernel(3) SelectorRunner" daemon prio=10 tid=0x00007fbc68251000 nid=0xaee runnable [0x00007fbcb55ce000] 
    java.lang.Thread.State: RUNNABLE 
     at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) 
     at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) 
     at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) 
     at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) 
     - locked <0x000000060263ad88> (a sun.nio.ch.Util$2) 
     - locked <0x000000060263ad78> (a java.util.Collections$UnmodifiableSet) 
     - locked <0x00000006025fb068> (a sun.nio.ch.EPollSelectorImpl) 
     at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106) 
     at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:114) 
     at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:338) 
     at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:278) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) 
     at java.lang.Thread.run(Thread.java:745) 


"http-listener-1-kernel(8) SelectorRunner" daemon prio=10 tid=0x00007fbc6825b800 nid=0xaf3 runnable [0x00007fbcb50c9000] 
    java.lang.Thread.State: RUNNABLE 
     at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) 
     at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) 
     at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) 
     at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) 
     - locked <0x00000006026648b8> (a sun.nio.ch.Util$2) 
     - locked <0x00000006026648a8> (a java.util.Collections$UnmodifiableSet) 
     - locked <0x0000000602664790> (a sun.nio.ch.EPollSelectorImpl) 
     at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106) 
     at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:114) 
     at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:338) 
     at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:278) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) 
     at java.lang.Thread.run(Thread.java:745) 

有趣的是，不消耗CPU的其他線程使用的sun.nio.ch.SelectorImpl不同的方法（選擇而非selectNow）：

"http-listener-1-kernel(7) SelectorRunner" daemon prio=10 tid=0x00007fbc68259800 nid=0xaf2 runnable [0x00007fbcb51ca000] 
    java.lang.Thread.State: RUNNABLE 
     at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) 
     at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) 
     at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) 
     at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) 
     - locked <0x0000000602665f20> (a sun.nio.ch.Util$2) 
     - locked <0x0000000602665f10> (a java.util.Collections$UnmodifiableSet) 
     - locked <0x0000000602665df8> (a sun.nio.ch.EPollSelectorImpl) 
     at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) 
     at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:112) 
     at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:338) 
     at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:278) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) 
     at java.lang.Thread.run(Thread.java:745) 

然後我再次打開客戶端，讓他們再運行幾個小時，然後再次停止。果然，「top -H」表明CPU使用率再次增加。另一個線程轉儲揭示了一個更加線程切換到「selectNow」，現在（與其他2一起）採取更多的CPU時間：

"http-listener-1-kernel(6) SelectorRunner" daemon prio=10 tid=0x00007fbc68257800 nid=0xaf1 runnable [0x00007fbcb52cb000] 
    java.lang.Thread.State: RUNNABLE 
     at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) 
     at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) 
     at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) 
     at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) 
     - locked <0x00000006026675c0> (a sun.nio.ch.Util$2) 
     - locked <0x00000006026675b0> (a java.util.Collections$UnmodifiableSet) 
     - locked <0x0000000602667498> (a sun.nio.ch.EPollSelectorImpl) 
     at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106) 
     at org.glassfish.grizzly.nio.DefaultSelectorHandler.select(DefaultSelectorHandler.java:114) 
     at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:338) 
     at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:278) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) 
     at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) 
     at java.lang.Thread.run(Thread.java:745) 

看來，東西導致線程調用的「selectNow」方法sun.nio.ch.SelectorImpl，一旦它們進入那裏，CPU使用率將大量增加，並且不會減少，除非服務器重新啓動。

這是一個已知的問題？這可能是由我的代碼造成的？

感謝您的幫助！

Answer 1

可能是這個問題：

https://java.net/jira/browse/GLASSFISH-21211

沒有爲灰熊包含在這一問題的補丁，但我沒有測試它。