2016-12-14 29 views
1

我正在使用線程進行遠程API調用,不使用聯接,以便程序可以在不等待最後一個完成的情況下進行下一個API調用。線程提示是連續運行的,不是平行的嗎?

像這樣:

def run_single_thread_no_join(function, args): 
thread = Thread(target=function, args=(args,)) 
thread.start() 
return 

的問題是我需要知道,當所有的API調用完成。所以我轉移到代碼是使用提示&加入。

線程似乎現在串行運行。

我似乎無法弄清楚如何讓連接工作,以便線程並行執行。

我在做什麼錯?

def run_que_block(methods_list, num_worker_threads=10): 
''' 
Runs methods on threads. Stores method returns in a list. Then outputs that list 
after all methods in the list have been completed. 

:param methods_list: example ((method name, args), (method_2, args), (method_3, args) 
:param num_worker_threads: The number of threads to use in the block. 
:return: The full list of returns from each method. 
''' 

method_returns = [] 

# log = StandardLogger(logger_name='run_que_block') 

# lock to serialize console output 
lock = threading.Lock() 

def _output(item): 
    # Make sure the whole print completes or threads can mix up output in one line. 
    with lock: 
     if item: 
      print(item) 
     msg = threading.current_thread().name, item 
     # log.log_debug(msg) 

    return 

# The worker thread pulls an item from the queue and processes it 
def _worker(): 

    while True: 
     item = q.get() 
     if item is None: 
      break 

     method_returns.append(item) 
     _output(item) 

     q.task_done() 

# Create the queue and thread pool. 
q = Queue() 

threads = [] 
# starts worker threads. 
for i in range(num_worker_threads): 
    t = threading.Thread(target=_worker) 
    t.daemon = True # thread dies when main thread (only non-daemon thread) exits. 
    t.start() 
    threads.append(t) 

for method in methods_list: 
    q.put(method[0](*method[1])) 

# block until all tasks are done 
q.join() 

# stop workers 
for i in range(num_worker_threads): 
    q.put(None) 
for t in threads: 
    t.join() 

return method_returns 
+0

的可能的複製[Python的線程不會同時運行(http://stackoverflow.com/questions/19614224/python-threads-dont-run-simultaneously) –

+0

一些你'def's的需要他們縮進的身體。 –

回答

1

你做所有的工作在主線程:

for method in methods_list: 
    q.put(method[0](*method[1])) 

假設在methods_list每個條目是一個可調用和它的參數的順序,你做了所有的工作,在主線程,然後將每個函數調用的結果放入隊列中,除了print ing之外(這通常不足以證明線程/隊列開銷),並不允許任何並行化。

想必,你想要的線程做各項功能的工作,所以改變環路:

for method in methods_list: 
    q.put(method) # Don't call it, queue it to be called in worker 

並更改_worker功能,所以它調用確實在線程工作中的作用:

def _worker(): 
    while True: 
     method, args = q.get() # Extract and unpack callable and arguments 
     item = method(*args) # Call callable with provided args and store result 
     if item is None: 
      break 

     method_returns.append(item) 
     _output(item) 

     q.task_done()