-2
我有一個列表,並且當我不斷追加元素到它時,我想檢查它不是空的,並在同一時間獲取元素。通常我們等待所有的元素追加到列表中,然後我們從列表中獲取元素做一些事情。在這種情況下,我們會失去一些時間來等待所有元素添加到列表中。我需要獲得什麼知識才能實現這一目標(多處理,多處理虛擬,異步),對不起,我仍然是新手,因此我認爲我更好地向您解釋爲什麼我想要實現這種效果,這個問題來自一個網絡爬蟲Python推送和同時彈出列表
import requests
from model import Document
def add_concrete_content(input_list):
"""input_list data structure [{'url': 'xxx', 'title': 'xxx'}...]"""
for e in input_list:
r = requests.get(e['url'])
html = r.content
e['html'] = html
return input_list
def save(input_list):
for e in input_list:
Document.create(**e)
if __name__ == '__main__':
res = add_concrete_content(list)
save(res)
"""this is I normally do, I save data to mysql or whatever database,but
I think the drawback is I have to wait all the html add to dict and then
save to database, what if I have to deal with tons of data? Can I save
the dict with the html first? Can I save some time? A friend of mine
said this is a typical producer consumer problem, probably gonna use
at least two threads and lock, because without lock, data probably
gonna fall into disorder"""
請將您的代碼直接包含在您的文章中,而不是截圖。 – Delgan
另外,您不一定非要使用pop(它也會從列表中刪除元素)。在python中,你可以簡單地使用list [index]語法。 – doratheexplorer0911