2017-02-19 75 views
2

我正在玩TensorFlow 1.0。我的輸入數據是大量的jpeg圖像。其中一些因爲不同的原因被打破,我只是想在輸入時跳過它們。該圖的TensorFlow:如何跳過損壞的數據

圖像裝載部分如下:

filename_queue = tf.train.string_input_producer(tf.train.match_filenames_once(filename_list), capacity=1000, num_epochs=1) 
whole_file_reader = tf.WholeFileReader() 
_, image_binary = whole_file_reader.read(filename_queue) 
image_tensor = tf.cast(tf.image.decode_jpeg(image_binary), tf.float32) 

示範運行的部分像往常一樣:

with sv.managed_session() as sess: 
     sess.run(init_local) 
     sess.run(init_all) 

     coord = tf.train.Coordinator() 
     threads = tf.train.start_queue_runners(coord=coord, sess=sess) 

     try: 
       while not coord.should_stop() and not sv.should_stop(): 
         sess.run(accumulator) 
     except tf.errors.OutOfRangeError: 
       print('Done training -- epoch limit reached') 
       # 
     except Exception as e: 
       # Report exceptions to the coordinator. 
       coord.request_stop(e) 
     finally: 
       coord.request_stop() 

     coord.request_stop() 
     coord.join(threads) 

運行此代碼,我看到以下內容,我無法弄清楚如何正確地捕捉這個異常。

Traceback (most recent call last): 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1022, in _do_call 
    return fn(*args) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1004, in _run_fn 
    status, run_metadata) 
    File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__ 
    next(self.gen) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 469, in raise_exception_on_not_ok_status 
    pywrap_tensorflow.TF_GetCode(status)) 
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid JPEG data, size 0 
     [[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](ReaderReadV2:1)]] 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "calculate_mean.py", line 67, in <module> 
    coord.join(threads) 
    File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__ 
    next(self.gen) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 973, in managed_session 
    self.stop(close_summary_writer=close_summary_writer) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 801, in stop 
    stop_grace_period_secs=self._stop_grace_secs) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 386, in join 
    six.reraise(*self._exc_info_to_raise) 
    File "/usr/lib/python3/dist-packages/six.py", line 686, in reraise 
    raise value 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/training/queue_runner_impl.py", line 234, in _run 
    sess.run(enqueue_op) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 767, in run 
    run_metadata_ptr) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 965, in _run 
    feed_dict_string, options, run_metadata) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run 
    target_list, options, run_metadata) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call 
    raise type(e)(node_def, op, message) 
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid JPEG data, size 0 
     [[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](ReaderReadV2:1)]] 

Caused by op 'DecodeJpeg', defined at: 
    File "calculate_mean.py", line 19, in <module> 
    image_tensor = tf.cast(tf.image.decode_jpeg(image_binary), tf.float32) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/ops/gen_image_ops.py", line 345, in decode_jpeg 
    dct_method=dct_method, name=name) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op 
    op_def=op_def) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op 
    original_op=self._default_original_op, op_def=op_def) 
    File "/home/matwey/venv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__ 
    self._traceback = _extract_stack() 

InvalidArgumentError (see above for traceback): Invalid JPEG data, size 0 
     [[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](ReaderReadV2:1)]] 

遺憾的是,答案在 Skipping nonexistent or corrupt files in Tensorflow 給我不起作用。在我看來,coord.join(threads)提出的例外情況已經太晚了。

回答

1

對不起,遲到的迴應。答案可能包含在您的錯誤信息中:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid JPEG data, size 0 [[Node: DecodeJpeg = DecodeJpegacceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"]]

無論出於何種原因,JPEG文件可能會損壞。但是,您已使用需要完美解碼的tf.image_decode_jpeg的默認設置。相反,你可能想通過設置參數try_recover_truncated = Trueacceptable_fraction=0.5(或其他)來允許一些錯誤。有關更多信息,請參閱​​。