池到芹菜集團

我正在使用芹菜啓動一項任務，這項任務包括一個使用多處理和ThreadPool構建的端口掃描程序。當我啓動它，我收到以下錯誤，尋找替代或教程遷移我現有的代碼：池到芹菜集團

[2015-10-25 18:53:57,090: ERROR/Worker-3] daemonic processes are not allowed to have children 
Traceback (most recent call last): 
    File "/Users/user/Documents/OpenSource/Development/Python/test/utils/port_scanner.py", line 57, in is_port_opened 
    ports = list(scan_ports(server=server, port=int(port))) 
    File "/Users/user/Documents/OpenSource/Development/Python/test/utils/port_scanner.py", line 37, in scan_ports 
    p = Pool(NUM_CORES) 
    File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/__init__.py", line 232, in Pool 
    return Pool(processes, initializer, initargs, maxtasksperchild) 
    File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 159, in __init__ 
    self._repopulate_pool() 
    File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 223, in _repopulate_pool 
    w.start() 
    File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 124, in start 
    'daemonic processes are not allowed to have children' 
AssertionError: daemonic processes are not allowed to have children

這是我的代碼：

import socket 
from functools import partial 
from multiprocessing import Pool 
from multiprocessing.pool import ThreadPool 
from errno import ECONNREFUSED 

NUM_CORES = 4 
def portscan(target, port): 
    try: 
     # Create Socket 
     s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
     s.settimeout(12) 
     s.connect((target, port)) 
     print 'port_scanner.is_port_opened() ' + str(port) + " is opened" 
     return port 
    except socket.error as err: 
     print str(err) 
     if err.errno == ECONNREFUSED: 
      return False 


# Wrapper function that calls portscanner 
def scan_ports(server=None, port=None, portStart=None, portEnd=None, **kwargs): 
    p = Pool(NUM_CORES) 
    ping_host = partial(portscan, server) 

    if portStart and portStart: 
     # Filter returns Booleans 
     return filter(bool, p.map(ping_host, range(portStart, portStart))) 
    else: 
     return filter(bool, p.map(ping_host, range(port, port + 1))) 


# Check if port is opened 
def is_port_opened(server=None, port=None, **kwargs): 
    try: 
     # check if its IP Address: 
     ports = list(scan_ports(server=server, port=int(port))) 
     if len(ports) != 0: 
      print 'port_scanner.is_port_opened() ' + str(len(ports)) + " port(s) available." 
      return True 
     else: 
      print 'port_scanner.is_port_opened() port not opened: (' + str(port) + ')' 
      return False 

    except Exception, e: 
     print str(e) 

if __name__ == '__main__': 
    is_port_opened(server='110.10.0.144', port=22)

來源

2015-10-26 spicyramen

你的問題是相當接近一個經歷在this question。守護進程無法產生子進程。其背後的原因是因爲守護進程被父級強行終止。但是，如果父母也是一個守護進程，那麼它將被強制終止，而沒有時間對其孩子做同樣的事情。

multiprocessing.Pool使用daemonic進程來確保它們在程序不存在時不會泄漏。我假定芹菜也是一樣來運行你的任務。因此，您正在守護進程中產生守護進程。

可能的解決方案是使用multiprocessing庫中的Process和Queue對象自己快速實現一個進程池。您會產生非守護進程並通過隊列提供它們。

來源

2015-10-29 07:49:37 noxdafox

池到芹菜集團

回答

相關問題