在Tensorflow中使用Adadelta優化器時未初始化的值錯誤

Question

我正在嘗試使用Adagrad優化器來構建CNN，但出現以下錯誤。

tensorflow.python.framework.errors.FailedPreconditionError：試圖使用未初始化值Variable_7/Adadelta

[[節點：Adadelta/update_Variable_7/ApplyAdadelta = ApplyAdadelta [T = DT_FLOAT，_class = [「LOC ：變量_7，變量_7/Adadelta，變量_7/Adadelta_1，Adadelta/lr，Adadelta/rho，_變量_7「]，use_locking = false，_device =」/ job：localhost/replica：0/task：0/cpu：0「 adadelta/epsilon，gradients/add_3_grad/tuple/control_dependency_1）]] 由op u'Adadelta/update_Variable_7/ApplyAdadelta引起，

優化= tf.train.AdadeltaOptimizer（learning_rate）.minimize（cross_entropy）

我想在這篇文章中提到的adagrad語句後重新初始化會話變量，但沒有幫助過。

我該如何避免這個錯誤？謝謝。

Tensorflow: Using Adam optimizer

import tensorflow as tf 
import numpy 
from tensorflow.examples.tutorials.mnist import input_data 

def conv2d(x, W): 
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME') 

def max_pool_2x2(x): 
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], 
         strides=[1, 2, 2, 1], padding='SAME') 

def weight_variable(shape): 
    initial = tf.truncated_normal(shape, stddev=0.1) 
    return tf.Variable(initial) 

def bias_variable(shape): 
    initial = tf.constant(0.1, shape=shape) 
    return tf.Variable(initial) 


mnist = input_data.read_data_sets("/tmp/data/", one_hot=True) 

# Parameters 
learning_rate = 0.01 
training_epochs = 100 
batch_size = 1000 
display_step = 1 


# Set model weights 
W = tf.Variable(tf.zeros([784, 10]), name="weights") 
b = tf.Variable(tf.zeros([10]), name="bias") 

W_conv1 = weight_variable([5, 5, 1, 32]) 
b_conv1 = bias_variable([32]) 


W_conv2 = weight_variable([5, 5, 32, 64]) 
b_conv2 = bias_variable([64]) 


W_fc1 = weight_variable([7 * 7 * 64, 1024]) 
b_fc1 = bias_variable([1024]) 

W_fc2 = weight_variable([1024, 10]) 
b_fc2 = bias_variable([10]) 

# Initializing the variables 
init = tf.initialize_all_variables() 

with tf.Session() as sess: 
    sess.run(init) 
    for epoch in range(training_epochs): 
     total_batch = int(mnist.train.num_examples/batch_size) 
     for i in range(total_batch): 

      batch_xs, batch_ys = mnist.train.next_batch(batch_size) 

      x_image = tf.reshape(batch_xs, [-1,28,28,1]) 

      h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1) 
      h_pool1 = max_pool_2x2(h_conv1) 

      h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2) 
      h_pool2 = max_pool_2x2(h_conv2) 

      h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64]) 
      h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1) 


      y_conv=tf.nn.softmax(tf.matmul(h_fc1, W_fc2) + b_fc2) 

      cross_entropy = tf.reduce_mean(-tf.reduce_sum(batch_ys * tf.log(y_conv), reduction_indices=[1])) 
      #optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cross_entropy) 

      optimizer = tf.train.AdadeltaOptimizer(learning_rate).minimize(cross_entropy) 
      sess.run(init) 

      correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(batch_ys,1)) 
      accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 
      sess.run([cross_entropy, y_conv,optimizer]) 
      print cross_entropy.eval() 

Answer 1

這裏的問題是，tf.initialize_all_variables()是一個誤導性的名稱。它確實意味着「返回一個操作，該操作初始化所有已創建的變量（在默認圖形中）」。當您致電tf.train.AdadeltaOptimizer(...).minimize()時，TensorFlow會創建其他變量，但您未在前面創建的init運算符中未包含該變量。

移動線：

init = tf.initialize_all_variables() 

... 的tf.train.AdadeltaOptimizer建成後應該讓你的工作方案。

N.B.在每個培訓步驟中，除了變量之外，您的程序還會重建整個網絡。這可能效率很低，並且Adadelta算法不會按預期進行調整，因爲其狀態在每一步都會重新創建。我強烈建議將代碼從batch_xs的定義移到在兩個嵌套for循環外創建optimizer。您應該爲batch_xs和batch_ys輸入定義tf.placeholder() ops，並使用feed_dict參數指向sess.run()以傳入mnist.train.next_batch()返回的值。