python中concurrent.futures的async和requests-futures異步有什麼區別？

Question

我用requests-futures來異步抓取網頁，而且我的機器有多核，所以我也想同時抓多個網站，然後我試着用concurrent.futures，看來concurrent.futures也提供異步方法，所以什麼是concurrent.futures'async和request-futures'異步之間的區別？如果它們相同，意味着我可以反對請求 - 期貨？

Answer 1

requests-futures只是在concurrent.futures之上的一個非常小的包裝。您可以通過查看source code（爲了簡潔，刪除文檔字符串）看到這一點：

from concurrent.futures import ThreadPoolExecutor 
from requests import Session 
from requests.adapters import DEFAULT_POOLSIZE, HTTPAdapter 

class FuturesSession(Session): 

    def __init__(self, executor=None, max_workers=2, *args, **kwargs): 
     super(FuturesSession, self).__init__(*args, **kwargs) 
     if executor is None: 
      executor = ThreadPoolExecutor(max_workers=max_workers) 
      # set connection pool size equal to max_workers if needed 
      if max_workers > DEFAULT_POOLSIZE: 
       adapter_kwargs = dict(pool_connections=max_workers, 
             pool_maxsize=max_workers) 
       self.mount('https://', HTTPAdapter(**adapter_kwargs)) 
       self.mount('http://', HTTPAdapter(**adapter_kwargs)) 

     self.executor = executor 

    def request(self, *args, **kwargs): 
     func = sup = super(FuturesSession, self).request 

     background_callback = kwargs.pop('background_callback', None) 
     if background_callback: 
      def wrap(*args_, **kwargs_): 
       resp = sup(*args_, **kwargs_) 
       background_callback(self, resp) 
       return resp 

      func = wrap 

     return self.executor.submit(func, *args, **kwargs) # This returns a concurrent.futures.Future 

當您使用requests-futures，你真的使用concurrent.futures.ThreadPoolExecutor，它返回一個concurrent.futures.Future當你submit給它的任務。如果您更方便地使用requests-futures提供的API來處理HTTP請求，可以堅持使用它，甚至可以使用由concurrent.futures模塊提供的其他方法返回的對象。