2013-04-18 65 views
2

我在獨角獸和Nginx(多個網絡節點)上運行了一個相當高流量的Rails 3.2應用程序,但每隔一段時間我都會看到獨角獸工作人員開始計時並由獨角獸開始工作掌握所有節點。當然,當一名獨角獸工作人員受到獨角獸的掌控時,一名新的工人被派上了位置,但它也只是掛了60秒,然後超時並被殺死。這基本上一直髮生,直到我殺死所有獨角獸的大師和工人。隨機獨角獸超時和撲動

Unicorn log:

E, [2013-04-18T12:57:50.007623 #14002] ERROR -- : worker=8 PID:14968 timeout (62s > 60s), killing 
E, [2013-04-18T12:57:50.108364 #14002] ERROR -- : reaped #<Process::Status: pid 14968 SIGKILL (signal 9)> worker=8 
I, [2013-04-18T12:57:50.489505 #15726] INFO -- : worker=8 ready 
E, [2013-04-18T12:57:52.175842 #14002] ERROR -- : worker=5 PID:15033 timeout (61s > 60s), killing 
E, [2013-04-18T12:57:52.276586 #14002] ERROR -- : reaped #<Process::Status: pid 15033 SIGKILL (signal 9)> worker=5 
I, [2013-04-18T12:57:52.653069 #15782] INFO -- : worker=5 ready 
E, [2013-04-18T12:57:56.340290 #14002] ERROR -- : worker=3 PID:15074 timeout (61s > 60s), killing 
E, [2013-04-18T12:57:56.440993 #14002] ERROR -- : reaped #<Process::Status: pid 15074 SIGKILL (signal 9)> worker=3 
I, [2013-04-18T12:57:56.809730 #15832] INFO -- : worker=3 ready 
E, [2013-04-18T12:57:57.504142 #14002] ERROR -- : worker=7 PID:15087 timeout (61s > 60s), killing 
E, [2013-04-18T12:57:57.604886 #14002] ERROR -- : reaped #<Process::Status: pid 15087 SIGKILL (signal 9)> worker=7 
I, [2013-04-18T12:57:57.983581 #15845] INFO -- : worker=7 ready 
E, [2013-04-18T12:57:59.669664 #14002] ERROR -- : worker=4 PID:15108 timeout (61s > 60s), killing 
E, [2013-04-18T12:57:59.770427 #14002] ERROR -- : reaped #<Process::Status: pid 15108 SIGKILL (signal 9)> worker=4 
I, [2013-04-18T12:58:00.155461 #15879] INFO -- : worker=4 ready 
E, [2013-04-18T12:58:06.839906 #14002] ERROR -- : worker=9 PID:15192 timeout (61s > 60s), killing 
E, [2013-04-18T12:58:06.940829 #14002] ERROR -- : reaped #<Process::Status: pid 15192 SIGKILL (signal 9)> worker=9 
I, [2013-04-18T12:58:07.302766 #15956] INFO -- : worker=9 ready 
E, [2013-04-18T12:58:08.003330 #14002] ERROR -- : worker=6 PID:15213 timeout (61s > 60s), killing 
E, [2013-04-18T12:58:08.104006 #14002] ERROR -- : reaped #<Process::Status: pid 15213 SIGKILL (signal 9)> worker=6 
I, [2013-04-18T12:58:08.466790 #15973] INFO -- : worker=6 ready 

監測系統顯示,外部服務(Postgres數據庫,Memcached的,Redis的)都是正確響應,無延遲問題。

這裏有一些產出可能的值:

在這些停運我注意到有人嘗試連接麒麟座的大量積壓。當站點沒有關閉時,通常以下命令僅返回一行或兩行。

netstat | grep unic

.... 
unix 2  [ ]   STREAM  CONNECTING 0  /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTING 0  /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTING 0  /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTING 0  /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTING 0  /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7768134 /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7767311 /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7766999 /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7767309 /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7766941 /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7767287 /tmp/unicorn.sock 
unix 2  [ ]   STREAM  CONNECTED  7766225 /tmp/unicorn.sock 

任何人都有一個想法可能導致什麼呢?這發生在多個服務器上,全部在同一時間。

回答

0

DDOS?在研究應用程序級別之前,我會確保您知道網絡級別會發生什麼。當它有效時,你有多少個連接?沒有的時候有多少?從哪些IP?等等。