恢復模型時使用批量規範？

Question

我有一個小問題，在張量流中恢復模型時使用批量常量。

下面是我的批處理規範從here：

def _batch_normalization(self, input_tensor, is_training, batch_norm_epsilon, decay=0.999): 
    """batch normalization for dense nets. 

    Args: 
     input_tensor: `tensor`, the input tensor which needed normalized. 
     is_training: `bool`, if true than update the mean/variance using moving average, 
          else using the store mean/variance. 
     batch_norm_epsilon: `float`, param for batch normalization. 
     decay: `float`, param for update move average, default is 0.999. 

    Returns: 
     normalized params. 
    """ 
    # actually batch normalization is according to the channels dimension. 
    input_shape_channels = int(input_tensor.get_shape()[-1]) 

    # scala and beta using in the the formula like that: scala * (x - E(x))/sqrt(var(x)) + beta 
    scale = tf.Variable(tf.ones([input_shape_channels])) 
    beta = tf.Variable(tf.zeros([input_shape_channels])) 

    # global mean and var are the mean and var that after moving averaged. 
    global_mean = tf.Variable(tf.zeros([input_shape_channels]), trainable=False) 
    global_var = tf.Variable(tf.ones([input_shape_channels]), trainable=False) 

    # if training, then update the mean and var, else using the trained mean/var directly. 
    if is_training: 
     # batch norm in the channel axis. 
     axis = list(range(len(input_tensor.get_shape()) - 1)) 
     batch_mean, batch_var = tf.nn.moments(input_tensor, axes=axis) 

     # update the mean and var. 
     train_mean = tf.assign(global_mean, global_mean * decay + batch_mean * (1 - decay)) 
     train_var = tf.assign(global_var, global_var * decay + batch_var * (1 - decay)) 
     with tf.control_dependencies([train_mean, train_var]): 
      return tf.nn.batch_normalization(input_tensor, 
              batch_mean, batch_var, beta, scale, batch_norm_epsilon) 
    else: 
     return tf.nn.batch_normalization(input_tensor, 
             global_mean, global_var, beta, scale, batch_norm_epsilon) 

我訓練模型，並使用tf.train.Saver()保存。下面是測試代碼：

def inference(self, images_for_predict): 
    """load the pre-trained model and do the inference. 

    Args: 
     images_for_predict: `tensor`, images for predict using the pre-trained model. 

    Returns: 
     the predict labels. 
    """ 

    tf.reset_default_graph() 
    images, labels, _, _, prediction, accuracy, saver = self._build_graph(1, False) 

    predictions = [] 
    correct = 0 
    with tf.Session() as sess: 
     sess.run(tf.global_variables_initializer()) 
     # saver = tf.train.import_meta_graph('./models/dense_nets_model/dense_nets.ckpt.meta') 
     # saver.restore(sess, tf.train.latest_checkpoint('./models/dense_nets_model/')) 
     saver.restore(sess, './models/dense_nets_model/dense_nets.ckpt') 
     for i in range(100): 
      pred, corr = sess.run([tf.argmax(prediction, 1), accuracy], 
            feed_dict={ 
             images: [images_for_predict.images[i]], 
             labels: [images_for_predict.labels[i]]}) 
      correct += corr 
      predictions.append(pred[0]) 
    print("PREDICTIONS:", predictions) 
    print("ACCURACY:", correct/100) 

但預測結果總是很糟糕，這樣的：

('PREDICTIONS:', [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]) 

('ACCURACY:', 0.080000000000000002) 

一些提示：images_for_predict = mnist.test和self._build_graph方法有兩個參數：batch_size和is_training。

任何人都可以幫到我嗎？

Answer 1

看到您的一批規範的執行，當你加載你的模型，你需要保持與images, labels, _, _, prediction, accuracy, saver = self._build_graph(1, False)內置圖形並加載重量爲chekpoint值，但不的元圖。我認爲saver.restore(sess, './models/dense_nets_model/dense_nets.ckpt')現在也恢復元圖（抱歉，如果我錯了），所以你只需要恢復它的「數據」部分。

否則，您只是使用圖形進行訓練，其中批量規範中使用的平均值和方差是從批次中獲得的平均值和方差。但是當你測試批量大小爲1時，所以通過批處理的均值和方差進行歸一化總是會使數據爲0，因此是恆定的輸出。

在任何情況下，我建議使用tf.layers.batch_normalization而是用is_training佔位符，你需要喂到你的網絡...

Answer 2

嘗試了很多方法，我解決這個問題，以下是我所做的。

首先感謝@gdelab，我用tf.layers.batch_normalization代替，所以像我的批量規範功能：

def _batch_normalization(self, input_tensor, is_training): 
    return tf.layers.batch_normalization(input_tensor, training=is_training) 

帕拉姆is_training就像是一個佔位符：is_training = tf.placeholder(tf.bool)

構建圖表時，請記住在您的優化中添加此代碼：

extra_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS) 
with tf.control_dependencies(extra_update_ops): 
    train_step = tf.train.AdamOptimizer(self.learning_rate).minimize(cross_entropy) 

因爲tf.layers.batch_normalization增加了日期平均值和方差不會自動作爲列車運行的依賴項添加 - 因此，如果您沒有做任何額外的事情，那麼它們永遠不會運行。

所以begain訓練網絡，後完成培訓，使用這樣的代碼保存模型：

saver = tf.train.Saver(var_list=tf.global_variables()) 
savepath = saver.save(sess, 'here_is_your_personal_model_path') 

注意var_list=tf.global_variables() PARAM確保tensorflow保存所有PARAMS包括全球平均/ VAR這被設置爲不可訓練。

恢復和測試模型時，做這樣的：

# build the graph like training: 
images, labels, _, _, prediction, accuracy, saver = self._build_graph(1, False) 
saver = tf.train.Saver() 
saver.restore(sess, 'here_is_your_personal_model_path') 

現在可以測試他/她的模型，希望它能幫助ü，謝謝！