Mapreduce作業隨機失敗，'BadValueError：名稱必須小於500'

Question

我正在運行一個映射縮減作業，它映射大約100k條記錄。經過幾千年的記錄（不同的地方，每次），在應用程序引擎日誌BadValueError以下錯誤

地圖步驟失敗：名稱必須小於500

Pipeline job id: baada59c857d11e28527e1efd5a519d7

Output Blob file name: content_contributors_report_1362478582.csv

日誌轉儲

Final result for job '1581739058957C0CA88F5' is 'success' E2013-03-05 
12:25:21.637 name must be under 500 bytes. Traceback (most recent call 
last): File 
"/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1535, in __call__ 
    rv = self.handle_exception(request, response, e) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1529, in __call__ 
    rv = self.router.dispatch(request, response) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1278, in default_dispatcher 
    return route.handler_adapter(request, response) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1102, in __call__ 
    return handler.dispatch() File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 572, in dispatch 
    return self.handle_exception(e, self.app.debug) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 570, in dispatch 
    return method(*args, **kwargs) File "/base/data/home/apps/s~myap/a-276.365486148015822746/mapreduce/base_handler.py", 
line 68, in post 
    self.handle() File "/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/handlers.py", 
line 437, in handle 
    spec, state, self.base_path()) File "/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/handlers.py", 
line 517, in _finalize_job 
    mapreduce_spec.mapper.output_writer_class().finalize_job(mapreduce_state) 
File 
"/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/output_writers.py", 
line 454, in finalize_job 
    files.blobstore.get_blob_key(filename))) File "/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/lib/files/blobstore.py", 
line 112, in get_blob_key 
    _BLOB_FILE_INDEX_KIND, ticket, namespace='')])[0] File "/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore_types.py", 
line 504, in from_path 
    ValidateString(id_or_name, 'name') File "/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore_types.py", 
line 176, in ValidateString 
    raise exception('%s must be under %d bytes.' % (name, max_len)) BadValueError: name must be under 500 bytes. 

Answer 1

不同版本的mapreduce可能會導致此問題。 基於Run Map Reduce on non-default versions? 它建議使用https://code.google.com/p/appengine-mapreduce/

The stale bundle link in the docs is a bug. That'll get cleaned up soon.

A few of our SDKs have bits of MapReduce (for historic reasons), but the open source version is the way to go for now.