python多處理共享變量保險箱

Question

我遇到了多處理模塊的麻煩。我有使用鎖，但下面的代碼仍然不安全，我不知道爲什麼，有時計數器不會等於100，我該如何修復代碼讓它安全？

import random 
import threading 
from multiprocessing import Pool, Manager 

import time 

lock = threading.Lock() 


def a_complex_operation(counter): 
    with lock: 
     time.sleep(random.random()) 
     counter.value += 1 


def main(): 
    pool = Pool(16) 
    counter = Manager().Value('i', 0) 

    for i in range(100): 
     pool.apply_async(a_complex_operation, args=(counter,)) 

    pool.close() 
    pool.join() 
    if counter.value != 100: 
     print "not equal 100, current value is: "+str(counter.value) 


if __name__ == '__main__': 
    count = 0 
    while True: 
     t1 = time.time() 
     main() 
     count += 1 
     print "the " + str(count) + " loop, cost time: " + str(time.time() - t1) 

的輸出將是：。

the 1 loop, cost time: 4.1369998455 
the 2 loop, cost time: 3.74100017548 
the 3 loop, cost time: 3.92299985886 
the 4 loop, cost time: 4.05500006676 
not equal 100, current value is: 99 
the 5 loop, cost time: 4.01900005341 
the 6 loop, cost time: 4.14299988747 

然後我測試管理器（）列表（）和管理器（）值（ 'I'，0）一起

import random 
from multiprocessing import Pool, Manager 
import time 


def a_complex_operation(list_, counter): 
    for x in range(10): 
     time.sleep(random.random()/10) 
     list_.append(x) 
     counter.value += 1 


def main(): 
    pool = Pool(16) 
    counter0 = 0 
    list_ = Manager().list() 
    counter = Manager().Value('i', 0) 

    for i in range(100): 
     pool.apply_async(a_complex_operation, args=(list_, counter)) 
     counter0 += 1 

    pool.close() 
    pool.join() 
    if len(list_) != 1000: 
     print "length of list is not equal 1000, current is:" + str(len(list_)) 
    if counter.value != 1000: 
     print "value of counter is not equal 1000, current is :" + str(counter.value) 


if __name__ == '__main__': 
    counter = 0 
    while True: 
     counter += 1 
     t1 = time.time() 
     main() 
     t2 = time.time() 
     print "the " + str(counter) + " loop cost time: " + str(t2 - t1) 

的輸出結果如下：

value of counter is not equal 1000, current is :916 
the 1 loop cost time: 3.92299985886 
value of counter is not equal 1000, current is :911 
the 2 loop cost time: 3.98500013351 
value of counter is not equal 1000, current is :925 
the 3 loop cost time: 4.007999897 
value of counter is not equal 1000, current is :913 
the 4 loop cost time: 3.99399995804 
value of counter is not equal 1000, current is :920 
the 5 loop cost time: 4.09500002861 
value of counter is not equal 1000, current is :915 

我覺得Manager（）.list（）是安全的，Man價值（'我'，0）是不安全的，有趣的，任何人都可以告訴我爲什麼經理（）。列表（）看起來像安全嗎？

Answer 1

您的子過程不會繼承鎖對象。或者他們這樣做，但他們是獨立的副本，沒有鏈接在一起，不能用於任何事情。所以有一個競爭條件，它最終失敗。

您可以通過Manager().Lock()解決此問題，因爲您已經在使用管理器。

def a_complex_operation(counter, alock): 
    with alock: 
     time.sleep(random.random()) 
     counter.value += 1 

def main(): 
    pool = Pool(16) 
    ma = Manager() 
    counter = ma.Value('i', 0) 
    lock = ma.Lock() 
    for i in range(100): 
     pool.apply_async(a_complex_operation, args=(counter, lock)) 

這樣做（你的子過程現在會慢很多，期望每次運行大約50秒，平均100秒平均0.5秒）。

但現在你counter.value始終是100

Answer 2

我認爲下面的代碼應該安全，快速，感謝@Hannu和@gzc

import random 
from multiprocessing import Pool, Manager 
import time 


def a_complex_operation(list_, counter, lock): 
    for x in range(10): 
     time.sleep(random.random()/10) 
     with lock: 
      list_.append(x) 
      counter.value += 1 


def main(): 
    pool = Pool(16) 
    list_ = Manager().list() 
    counter = Manager().Value('i', 0) 
    lock = Manager().Lock() 

    for i in range(100): 
     pool.apply_async(a_complex_operation, args=(list_, counter, lock)) 

    pool.close() 
    pool.join() 
    if len(list_) != 1000: 
     print ">>> length of list is not equal 1000, current is:" + str(len(list_)) 
    elif len(list_) == 1000: 
     print ">>> length of list is equal 1000" 
    if counter.value != 1000: 
     print "value of counter is not equal 1000, current is :" + str(counter.value) 
    elif counter.value == 1000: 
     print "value of counter is equal 1000" 


if __name__ == '__main__': 
    counter = 0 
    while True: 
     counter += 1 
     t1 = time.time() 
     main() 
     t2 = time.time() 
     print "the " + str(counter) + " loop cost time: " + str(t2 - t1) 
     print "--------------------------------" 

輸出將是：

>>> length of list is equal 1000 
value of counter is equal 1000 
the 1 loop cost time: 3.78799986839 
-------------------------------- 
>>> length of list is equal 1000 
value of counter is equal 1000 
the 2 loop cost time: 3.79299998283 
-------------------------------- 
>>> length of list is equal 1000 
value of counter is equal 1000 
the 3 loop cost time: 3.78299999237 
-------------------------------- 
>>> length of list is equal 1000 
value of counter is equal 1000 
the 4 loop cost time: 3.77500009537 
--------------------------------