無法導入多線程的Python模塊。

Question

這是我試圖導入爲模塊的多線程代碼。當我將它作爲獨立文件運行時，它工作正常。它只是打印出一個數字列表。我改變的是哪個main（）被註釋掉。我認爲問題可能是導入的代碼在reader文件已經將其稱爲模塊之後將其自己調用爲模塊。

threadtest.py

from multiprocessing import Process, Value, Array 
import time 
import sys 

def f(n, a): 
    n.value = 3.1415927 
    for i in range(len(a)): 
     a[i] = -a[i] 

def is_prime(x, top): 
    for j in range(2,int(top**.5 + 1)): 
     if x%j == 0: 
      return False 
    return True 

def f1(top, c): 
    print('test f1') 
    for i in range(2,top): 
     if is_prime(i,top): 
      c.value = c.value + 1 

def f3(c1,c2): 
    for k in range(0,20): 
     time.sleep(1) # 1 second 
     sys.stdout.write(str(c1.value) + '|' + str(c2.value) + '\n') 
     sys.stdout.flush() 

def main(): 

count1 = Value('d', 0) 
count2 = Value('d', 0) 
#arr = Array('i', range(10)) 
p1 = Process(target=f1, args=(1000000, count1)) 
p2 = Process(target=f1, args=(1000000, count2)) 
p3 = Process(target=f3, args=(count1, count2)) 

p1.start() 
p2.start() 
p3.start() 

p1.join() 
print('p1.join()') 
p2.join() 
print('p2.join()') 
p3.join() 
print('p3.join()') 

print(count1.value) 
print(count2.value) 

if __name__ == '__main__': 
    print('grovetest is being run as main') 
    #main() 
else: 
    print('grovetest is being run as module') 
    main() 

這是導入多線程模塊，並嘗試讀取輸出的代碼。

readertest.py

import threadtest 

def main(fname): 

    try: 

     p = subprocess.Popen(fname, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True) 
     print('success') 
     return p.communicate() # this gets you pipe values 

    except Exception as e: 

     return 'error'+str(e) 

    else: 

     return "else" 

if __name__ == '__main__': 
    main(threadtest) 

這裏是產生的錯誤，當我運行readertest.py

grovetest is being run as module 
grovetest is being run as module 
Traceback (most recent call last): 
    File "<string>", line 1, in <module> 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 106, in spawn_main 
    exitcode = _main(fd) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 115, in _main 
    prepare(preparation_data) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 226, in prepare 
    _fixup_main_from_path(data['init_main_from_path']) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 278, in _fixup_main_from_path 
    run_name="__mp_main__") 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 240, in run_path 
    pkg_name=pkg_name, script_name=fname) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 96, in _run_module_code 
    mod_name, mod_spec, pkg_name, script_name) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 85, in _run_code 
    exec(code, run_globals) 
    File "C:\Users\Joseph\documents\python projects\multitest.py", line 1, in <module> 
    import grovetest 
    File "C:\Users\Joseph\documents\python projects\grovetest.py", line 56, in <module> 
    main() 
    File "C:\Users\Joseph\documents\python projects\grovetest.py", line 37, in main 
    p1.start() 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\process.py", line 105, in start 
    self._popen = self._Popen(self) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\context.py", line 212, in _Popen 
    return _default_context.get_context().Process._Popen(process_obj) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\context.py", line 313, in _Popen 
    return Popen(process_obj) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\popen_spawn_win32.py", line 34, in __init__ 
    prep_data = spawn.get_preparation_data(process_obj._name) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 144, in get_preparation_data 
    _check_not_importing_main() 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 137, in _check_not_importing_main 
    is not going to be frozen to produce an executable.''') 
RuntimeError: 
     An attempt has been made to start a new process before the 
     current process has finished its bootstrapping phase. 

     This probably means that you are not using fork to start your 
     child processes and you have forgotten to use the proper idiom 
     in the main module: 

      if __name__ == '__main__': 
       freeze_support() 
       ... 

     The "freeze_support()" line can be omitted if the program 
     is not going to be frozen to produce an executable. 
Traceback (most recent call last): 
    File "<string>", line 1, in <module> 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 106, in spawn_main 
    exitcode = _main(fd) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 115, in _main 
    prepare(preparation_data) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 226, in prepare 
    _fixup_main_from_path(data['init_main_from_path']) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 278, in _fixup_main_from_path 
    run_name="__mp_main__") 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 240, in run_path 
    pkg_name=pkg_name, script_name=fname) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 96, in _run_module_code 
    mod_name, mod_spec, pkg_name, script_name) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 85, in _run_code 
    exec(code, run_globals) 
    File "C:\Users\Joseph\documents\python projects\multitest.py", line 1, in <module> 
    import grovetest 
    File "C:\Users\Joseph\documents\python projects\grovetest.py", line 56, in <module> 
    main() 
    File "C:\Users\Joseph\documents\python projects\grovetest.py", line 37, in main 
    p1.start() 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\process.py", line 105, in start 
    self._popen = self._Popen(self) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\context.py", line 212, in _Popen 
    return _default_context.get_context().Process._Popen(process_obj) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\context.py", line 313, in _Popen 
    return Popen(process_obj) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\popen_spawn_win32.py", line 34, in __init__ 
    prep_data = spawn.get_preparation_data(process_obj._name) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 144, in get_preparation_data 
    _check_not_importing_main() 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 137, in _check_not_importing_main 
    is not going to be frozen to produce an executable.''') 
RuntimeError: 
     An attempt has been made to start a new process before the 
     current process has finished its bootstrapping phase. 

     This probably means that you are not using fork to start your 
     child processes and you have forgotten to use the proper idiom 
     in the main module: 

      if __name__ == '__main__': 
       freeze_support() 
       ... 

     The "freeze_support()" line can be omitted if the program 
     is not going to be frozen to produce an executable. 
grovetest is being run as module 
Traceback (most recent call last): 
    File "<string>", line 1, in <module> 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 106, in spawn_main 
    exitcode = _main(fd) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 115, in _main 
    prepare(preparation_data) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 226, in prepare 
    _fixup_main_from_path(data['init_main_from_path']) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 278, in _fixup_main_from_path 
    run_name="__mp_main__") 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 240, in run_path 
    pkg_name=pkg_name, script_name=fname) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 96, in _run_module_code 
    mod_name, mod_spec, pkg_name, script_name) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 85, in _run_code 
    exec(code, run_globals) 
    File "C:\Users\Joseph\documents\python projects\multitest.py", line 1, in <module> 
    import grovetest 
    File "C:\Users\Joseph\documents\python projects\grovetest.py", line 56, in <module> 
    main() 
    File "C:\Users\Joseph\documents\python projects\grovetest.py", line 37, in main 
    p1.start() 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\process.py", line 105, in start 
    self._popen = self._Popen(self) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\context.py", line 212, in _Popen 
    return _default_context.get_context().Process._Popen(process_obj) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\context.py", line 313, in _Popen 
    return Popen(process_obj) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\popen_spawn_win32.py", line 34, in __init__ 
    prep_data = spawn.get_preparation_data(process_obj._name) 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 144, in get_preparation_data 
    _check_not_importing_main() 
    File "C:\Users\Joseph\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\spawn.py", line 137, in _check_not_importing_main 
    is not going to be frozen to produce an executable.''') 
RuntimeError: 
     An attempt has been made to start a new process before the 
     current process has finished its bootstrapping phase. 

     This probably means that you are not using fork to start your 
     child processes and you have forgotten to use the proper idiom 
     in the main module: 

      if __name__ == '__main__': 
       freeze_support() 
       ... 

     The "freeze_support()" line can be omitted if the program 
     is not going to be frozen to produce an executable. 
grovetest is being run as module 
p1.join() 
p2.join() 
p3.join() 
0.0 
0.0 

我是新來這。感謝您的反饋意見。

Answer 1

您違反叫了here一個顯著Windows的特定約束：

確保主模塊可以通過一個新的Python解釋器可以安全地進口，而不會導致意想不到的副作用（例如一開始一個新的處理）。在我剛引述的一段

一個缺陷是，它說：「主模塊」時，它應該說「每模塊」，但大多數人不嘗試從導入模塊運行復雜的代碼，因此不要碰到這個問題。在你的情況下，會發生什麼readertest導入grovetest，它調用多處理代碼，該代碼使用特殊的Windows多進程引導系統啓動了一個新的Python解釋器，該系統啓動了多處理代碼，該代碼檢測到它正在被觸發特殊情況下，引導和中止：

(start) 
readertest: 
    import grovetest 

=> grovetest 
    import multiprocessing 
    => multiprocessing: define some stuff 

=> grovetest 
    if __name__ == '__main__': # it's not equal 
     ... # so we skip this 
    else: 
     print('grovetest is being run as module') # prints 
     main() # calls main 
    ... 
    p1 = Process(target=f1, args=(1000000, count1)) # calls multiprocessing module 

     => new process 
     import grovetest # multiprocessing does this using its magic bootstrap 
     ... 
     if __name__ == '__main__': # it's not 
      ... 
     else: 
      print(...) 
      ... 
      p1 = Process(...) 

此時內Process代碼檢測約束衝突並引發第一個錯誤。這完全終止了內部過程，並且我們回到了外部過程，它會再次啓動相同的錯誤，如此類推。

爲了解決這個問題，你不得打電話給你main在grovetest當grovetest導入爲模塊。相反，請致電grovetest.main以後，從readertest。這將允許特殊的引導魔法（在multiprocessing模塊中，當您調用multiprocessing.Process時運行）在新過程中導入grovetest。一旦這個新進程完成導入grovetest，它將等待來自其父進程（另一個Python解釋器，其中您調用multiprocessing.Process，從您的readertest代碼調用grovetest.main）的指令。這些指令將使用提供的參數調用功能f1（從target=f1）。