Python BigQuery真的很奇怪超時

Question

我正在構建一個服務來將數據傳輸到bigquery。下面的代碼完美的作品，如果我刪除需要4-5分鐘來加載（我預先緩存一些映射）

from googleapiclient import discovery 
from oauth2client import file 
from oauth2client import client 
from oauth2client import tools 

from oauth2client.client import SignedJwtAssertionCredentials 

## load email and key 
credentials = SignedJwtAssertionCredentials(email, key, scope='https://www.googleapis.com/auth/bigquery') 

if credentials is None or credentials.invalid: 
     raw_input('invalid key') 
     exit(0) 

http = httplib2.Http() 
http = credentials.authorize(http) 

service = discovery.build('bigquery', 'v2', http=http) 


## this does not hang, because it is before the long operation 
service.tabledata().insertAll(...) 


## some code that takes 5 minutes to execute 
r = load_mappings() 
## aka long operation 

## this hangs 
service.tabledata().insertAll(...) 

如果我離開這個需要5分鐘來執行的部分的一部分，谷歌API停止響應。事後的要求。它只是掛在那裏，甚至不會返回錯誤。我甚至在10-20分鐘內看到會發生什麼，它只是坐在那裏。如果我按Ctrl + C，我得到這個：

^CTraceback (most recent call last): 
    File "./to_bigquery.py", line 116, in <module> 
    main(sys.argv) 
    File "./to_bigquery.py", line 101, in main 
    print service.tabledata().insertAll(projectId=p_n, datasetId="XXX", tableId="%s_XXXX" % str(shop), body=_mybody).execute() 
    File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper 
    return wrapped(*args, **kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/googleapiclient/http.py", line 716, in execute 
    body=self.body, headers=self.headers) 
    File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper 
    return wrapped(*args, **kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 490, in new_request 
    redirections, connection_type) 
    File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1593, in request 
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) 
    File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1335, in _request 
    (response, content) = self._conn_request(conn, request_uri, method, body, headers) 
    File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1291, in _conn_request 
    response = conn.getresponse() 
    File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse 
    response.begin() 
    File "/usr/lib/python2.7/httplib.py", line 407, in begin 
    version, status, reason = self._read_status() 
    File "/usr/lib/python2.7/httplib.py", line 365, in _read_status 
    line = self.fp.readline() 
    File "/usr/lib/python2.7/socket.py", line 430, in readline 
    data = recv(1) 
    File "/usr/lib/python2.7/ssl.py", line 241, in recv 
    return self.read(buflen) 
    File "/usr/lib/python2.7/ssl.py", line 160, in read 
    return self._sslobj.read(len) 

我已成功地通過將大負荷運行前的憑證授權臨時修復它，但它似乎是我的錯誤。我錯過了什麼？

編輯：我設法得到一個錯誤，同時等待：

File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper 
    return wrapped(*args, **kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/googleapiclient/http.py", line 716, in execute 
    body=self.body, headers=self.headers) 
    File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 132, in positional_wrapper 
    return wrapped(*args, **kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 490, in new_request 
    redirections, connection_type) 
    File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1593, in request 
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) 
    File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1335, in _request 
    (response, content) = self._conn_request(conn, request_uri, method, body, headers) 
    File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1291, in _conn_request 
    response = conn.getresponse() 
    File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse 
    response.begin() 
    File "/usr/lib/python2.7/httplib.py", line 407, in begin 
    version, status, reason = self._read_status() 
    File "/usr/lib/python2.7/httplib.py", line 365, in _read_status 
    line = self.fp.readline() 
    File "/usr/lib/python2.7/socket.py", line 430, in readline 
    data = recv(1) 
    File "/usr/lib/python2.7/ssl.py", line 241, in recv 
    return self.read(buflen) 
    File "/usr/lib/python2.7/ssl.py", line 160, in read 
    return self._sslobj.read(len) 
socket.error: [Errno 110] Connection timed out 

它說超時。這似乎與冷表發生..

Answer 1

def refresh_bq(self): 
    credentials = SignedJwtAssertionCredentials(email, key, scope='https://www.googleapis.com/auth/bigquery') 

    if credentials is None or credentials.invalid: 
     raw_input('invalid key') 
     exit(0) 

    http = httplib2.Http() 
    http = credentials.authorize(http) 

    service = discovery.build('bigquery', 'v2', http=http) 
    self.service = service 

我運行self.refresh_bq（）每次我做一些插入不需要預處理，它完美的作品。凌亂的黑客攻擊，但我需要儘快完成工作。有def。一個地方的錯誤。