在應用程序引擎映射中獲取「應用程序錯誤：10」 - 減少

在縮小步驟中收到類似這樣的錯誤。 MapReduce Overview頁面報告所有作業都已成功完成，但是我試圖寫出的一半記錄沒有寫入。是什麼導致這個錯誤，我該如何解決它？：在應用程序引擎映射中獲取「應用程序錯誤：10」 - 減少

ApplicationError: 10 
Traceback (most recent call last): 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__ 
    rv = self.handle_exception(request, response, e) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__ 
    rv = self.router.dispatch(request, response) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher 
    return route.handler_adapter(request, response) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__ 
    return handler.dispatch() 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch 
    return self.handle_exception(e, self.app.debug) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch 
    return method(*args, **kwargs) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/base_handler.py", line 65, in post 
    self.handle() 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 181, in handle 
    entity, input_reader, ctx, tstate) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 298, in process_data 
    output_writer.write(output, ctx) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 693, in write 
    ctx.get_pool("records_pool").append(str(data)) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 336, in append 
    self.flush() 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 371, in flush 
    COUNTER_IO_WRITE_BYTES, len(str_buf))(self._ctx) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 297, in __exit__ 
    self.close() 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 291, in close 
    self._make_rpc_call_with_retry('Close', request, response) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 427, in _make_rpc_call_with_retry 
    _make_call(method, request, response) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 252, in _make_call 
    _raise_app_error(e) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 195, in _raise_app_error 
    raise FileNotOpenedError(e) 
FileNotOpenedError: ApplicationError: 10 

Traceback (most recent call last): 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/main.py", line 105, in main 
    util.run_wsgi_app(APP) 
    File "/python27_runtime/python27_lib/versions/1/google/appengine/ext/webapp/util.py", line 98, in run_wsgi_app 
    run_bare_wsgi_app(add_wsgi_middleware(application)) 
    File "/python27_runtime/python27_lib/versions/1/google/appengine/ext/webapp/util.py", line 116, in run_bare_wsgi_app 
    result = application(env, _start_response) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1519, in __call__ 
    response = self._internal_error(e) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__ 
    rv = self.handle_exception(request, response, e) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__ 
    rv = self.router.dispatch(request, response) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher 
    return route.handler_adapter(request, response) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__ 
    return handler.dispatch() 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch 
    return self.handle_exception(e, self.app.debug) 
    File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch 
    return method(*args, **kwargs) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/base_handler.py", line 65, in post 
    self.handle() 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 181, in handle 
    entity, input_reader, ctx, tstate) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 298, in process_data 
    output_writer.write(output, ctx) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 693, in write 
    ctx.get_pool("records_pool").append(str(data)) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 336, in append 
    self.flush() 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 371, in flush 
    COUNTER_IO_WRITE_BYTES, len(str_buf))(self._ctx) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 297, in __exit__ 
    self.close() 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 291, in close 
    self._make_rpc_call_with_retry('Close', request, response) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 427, in _make_rpc_call_with_retry 
    _make_call(method, request, response) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 252, in _make_call 
    _raise_app_error(e) 
    File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 195, in _raise_app_error 
    raise FileNotOpenedError(e) 
FileNotOpenedError: ApplicationError: 10

來源

2013-03-18 2daaa

感謝您詳細的追蹤，但是你不覺得你應該已經發布了一些實際減少代碼的？ – 2013-03-18 21:32:48

這裏的縮減代碼是否相關？據我瞭解，問題出在output_writer.py模塊中。 reduce代碼只是產生一個鍵值對，然後output_writer存在一個可能與數據大小有關的問題。我猜它可能與這個問題有關：https://coderwall.com/p/h3cecg。 – 2daaa 2013-03-19 18:59:06

我增加了似乎已經解決了問題的碎片的數量。增加碎片數量可以減少每個碎片將文件打開的時間，這可能是導致我遇到的問題的原因。

來源

2013-03-27 00:24:43 2daaa

將文件打開超過30秒也會導致此錯誤。嘗試將寫入分成多個寫入。

refer link for deatils https://groups.google.com/forum/#!topic/google-appengine-java/5XBVaVDKvvs

來源

2013-08-19 06:57:28 kiran

你知道我該怎麼做Python中的等價物？ – 2daaa 2013-08-19 21:33:47

在應用程序引擎映射中獲取「應用程序錯誤：10」 - 減少

回答

相關問題