當我提到這個問題時,我期望答案真的使用asyncio進行進程間通信。
我已經發現以下資源有用: https://github.com/python/asyncio/blob/master/examples/child_process.py
並且在下面我的簡化的例子(使用3.5+異步/ AWAIT語法),其內容爲線,並輸出它們來分類:
import asyncio
from subprocess import Popen, PIPE
async def connect_write_pipe(file):
"""Return a write-only transport wrapping a writable pipe"""
loop = asyncio.get_event_loop()
transport, _ = await loop.connect_write_pipe(asyncio.Protocol, file)
return transport
async def connect_read_pipe(file):
"""Wrap a readable pipe in a stream"""
loop = asyncio.get_event_loop()
stream_reader = asyncio.StreamReader(loop=loop)
def factory():
return asyncio.StreamReaderProtocol(stream_reader)
transport, _ = await loop.connect_read_pipe(factory, file)
return stream_reader, transport
async def main(loop):
# start subprocess and wrap stdin, stdout, stderr
p = Popen(['/usr/bin/sort'], stdin=PIPE, stdout=PIPE, stderr=PIPE)
stdin = await connect_write_pipe(p.stdin)
stdout, stdout_transport = await connect_read_pipe(p.stdout)
stderr, stderr_transport = await connect_read_pipe(p.stderr)
# interact with subprocess
name = {stdout: 'OUT', stderr: 'ERR'}
registered = {
asyncio.Task(stderr.read()): stderr,
asyncio.Task(stdout.read()): stdout
}
to_sort = b"one\ntwo\nthree\n"
stdin.write(to_sort)
stdin.close() # this way we tell we do not have anything else
# get and print lines from stdout, stderr
timeout = None
while registered:
done, pending = await asyncio.wait(
registered, timeout=timeout,
return_when=asyncio.FIRST_COMPLETED)
if not done:
break
for f in done:
stream = registered.pop(f)
res = f.result()
if res != b'':
print(name[stream], res.decode('ascii').rstrip())
registered[asyncio.Task(stream.read())] = stream
timeout = 0.0
stdout_transport.close()
stderr_transport.close()
if __name__ == '__main__':
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(main(loop))
finally:
loop.close()
NB :沒有采取特殊措施,要寫入管道的數據量是有限的。在我的系統中,在使用管道緩衝區之前,可以寫入超過700000字節。
那裏還有其他的例子,使用create_subprocess_shell
。
我還沒有在真實項目中使用asyncio,所以歡迎評論中的改進建議。
問題是什麼? –
這不會異步運行。它等待這個過程完成,但我希望它在此期間運行其他的東西。所以就像我先調用這個函數,然後調用另一個函數,這個函數大約需要20s,另一個大約需要2s。第二個func在跑之前必須等待20秒。 – init6
你有避免多線程的理由嗎? –