爲什麼配料

Question

我使用的tensorflow輸入管線數據集API R1.2

我建立自己的數據集，並與批量批次之後dataset.output_shapes收益產品尺寸（無）= 128

然後將其輸入到RNN。

但dataset.output_shape返回尺寸（無）在第一維的，所以RNN提出了一個錯誤：

Traceback (most recent call last): 
    File "untitled1.py", line 188, in <module> 
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed) 
    File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run 
    _sys.exit(main(_sys.argv[:1] + flags_passthrough)) 
    File "untitled1.py", line 121, in main 
    run_training() 
    File "untitled1.py", line 57, in run_training 
    is_training=True) 
    File "/home/harold/huawei/ConvLSTM/ConvLSTM.py", line 216, in inference 
    initial_state=initial_state) 
    File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 566, in dynamic_rnn 
    dtype=dtype) 
    File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 636, in _dynamic_rnn_loop 
    "Input size (depth of inputs) must be accessible via shape inference," 
ValueError: Input size (depth of inputs) must be accessible via shape inference, but saw value None. 

我認爲這種錯誤是由輸入的形狀造成的，所述第一尺寸應批量大小但不是無。

這裏是代碼：

origin_dataset = Dataset.BetweenS_Dataset(FLAGS.data_path) 
train_dataset = origin_dataset.train_dataset 
test_dataset = origin_dataset.test_dataset 
shuffle_train_dataset = train_dataset.shuffle(buffer_size=10000) 
shuffle_batch_train_dataset = shuffle_train_dataset.batch(128) 
batch_test_dataset = test_dataset.batch(FLAGS.batch_size) 

iterator = tf.contrib.data.Iterator.from_structure(
          shuffle_batch_train_dataset.output_types, 
          shuffle_batch_train_dataset.output_shapes) 
(images, labels) = iterator.get_next() 

training_init_op = iterator.make_initializer(shuffle_batch_train_dataset) 
test_init_op = iterator.make_initializer(batch_test_dataset) 

print(shuffle_batch_train_dataset.output_shapes) 

我打印output_shapes，它給：

(TensorShape([Dimension(None), Dimension(36), Dimension(100)]), TensorShape([Dimension(None)])) 

我想它應該是128，因爲我已經成批的數據集：

(TensorShape([Dimension(128), Dimension(36), Dimension(100)]), TensorShape([Dimension(128)])) 

Answer 1

它們在實現中對批處理大小進行了硬編碼，並且它總是會返回None（tf 1.3）。

def _padded_shape_to_batch_shape(s): 
    return tensor_shape.vector(None).concatenate(
     tensor_util.constant_value_as_shape(s)) 

以這種方式，它們可以批量的所有元素（例如dataset_size=14，batch_size=5，last_batch_size=4）。

您可以使用dataset.filter和dataset.map來解決這個問題

d = contrib.data.Dataset.from_tensor_slices([[5] for x in range(14)]) 
batch_size = 5 
d = d.batch(batch_size) 
d = d.filter(lambda e: tf.equal(tf.shape(e)[0], batch_size)) 
def batch_reshape(e): 
    return tf.reshape(e, [args.batch_size] + [s if s is not None else -1 for s in e.shape[1:].as_list()]) 
d = d.map(batch_reshape) 
r = d.make_one_shot_iterator().get_next() 
print('dataset_output_shape = %s' % r.shape) 
with tf.Session() as sess: 
    while True: 
     print(sess.run(r)) 

Output

dataset_output_shape = (5, 1)

[[5][5][5][5][5]]

[[5][5][5][5][5]]

OutOfRangeError