我不知道我怎麼會限制這樣的事情也只能使用10個線程在同一時間python如何設置線程限制?
with open("data.txt") as f:
for line in f:
lines = line.rstrip("\n\r")
t1 = Thread(target=Checker, args=("company"))
t1.start()
我不知道我怎麼會限制這樣的事情也只能使用10個線程在同一時間python如何設置線程限制?
with open("data.txt") as f:
for line in f:
lines = line.rstrip("\n\r")
t1 = Thread(target=Checker, args=("company"))
t1.start()
使用Python的ThreadPoolExecutor有max_workers參數設置爲10
事情是這樣的:`
pool = ThreadPoolExecutor(max_workers=10)
with open("data.txt") as f:
for line in f:
lines = line.rstrip("\n\r")
pool.submit(Checker,"company")
pool.shutdown(wait=True)
的pool
將根據需要自動分配線程,限制分配的最大數量爲10。第一個參數在pool.submit()
是函數名稱,參數只是以逗號分隔的值傳遞。
pool.shutdown(wait=True)
等待所有線程完成執行。
使用ThreadPoolExecutor
,並告訴它你要10個線程。
def your_function_processing_one_line(line):
pass # your computations
with concurrent.futures.ThreadPoolExecutor(10) as executor:
result = executor.map(your_function_processing_one_line, [line for line in f])
...,你將不得不在result
所有結果。
如果多個參數? –
這也是可能的。看看[這個很好的答案](https://stackoverflow.com/questions/6785226/pass-multiple-parameters-to-concurrent-futures-executor-map/6976772#6976772)。 – yogabonito
(對於Python的2.6+和Python 3)
使用threadPool
從multiprocessing
模塊:
from multiprocessing.pool import ThreadPool
的唯一的事情是,它沒有很好的記載......
我編寫了這個嵌套循環來將線程限制爲一個變量。 此代碼依賴於預設的命令數組進行處理。 我從其他答案中借用了一些元素來啓動線程。
import os, sys, datetime, logging, thread, threading, time
from random import randint
# set number of threads
threadcount = 20
# alltests is an array of test data
numbertests = len(alltests)
testcounter = numbertests
# run tests
for test in alltests:
# launch worker thread
def worker():
"""thread worker function"""
os.system(command)
return
threads = []
t = threading.Thread(target=worker)
threads.append(t)
t.start()
testcounter -= 1
# cap the threads if over limit
while threading.active_count() >= threadcount:
threads = threading.active_count()
string = "Excessive threads, pausing 5 secs - " + str(threads)
print (string)
logging.info(string)
time.sleep(5)
# monitor for threads winding down
while threading.active_count() != 1:
threads = threading.active_count()
string = "Active threads running - " + str(threads)
print (string)
logging.info(string)
time.sleep(5)
只是監視線程活着並阻塞,直到其中一個線程退出,如果有10個或更多。 – ForceBru
可能的重複[如何限制在Python中的活動線程數量?](http://stackoverflow.com/questions/1787397/how-do-i-limit-the-number-of-active-threads-in -python) – zondo
[Python線程池類似於多處理池?]可能重複(http://stackoverflow.com/questions/3033952/python-thread-pool-similar-to-the-multiprocessing-pool) –