在TensorFlow中使用make_template（）

Question

我試圖使用make_template（）來避免在我的模型中傳遞重用標誌。但似乎make_template（）在python類中使用時無法正確工作。我粘貼]我的模型代碼和我得到的錯誤。在MNIST數據集上訓練是一個簡單的MLP。

由於代碼有點長，這裏的主要部分是_weights（）函數。我嘗試使用make_template（）來封裝它，然後使用get_variables（）在其中創建並重新使用整個模型中的權重。 _weights（）由_create_dense_layer（）使用，並由_create_model（）用來創建圖形。 train（）函數接受我從數據讀取器獲得的張量。

模型

 
class MLP(object): 
    def __init__(self, hidden=[], biases=False, activation=tf.nn.relu): 
     self.graph = tf.get_default_graph() 
     self.hidden = hidden 
     self.activation = activation 
     self.biases = biases 
     self.n_features = 784 
     self.n_classes = 10 
     self.bsize = 100 
     self.l2 = 0.1 

    def _real_weights(self, shape): 
     initializer=tf.truncated_normal_initializer(stddev=0.1) 
     weights = tf.get_variable('weights', shape, initializer=initializer) 
     return weights 
    # use make_template to make variable reuse transparent 
    _weights = tf.make_template('_weights', _real_weights) 

    def _real_biases(self, shape): 
     initializer=tf.constant_initializer(0.0) 
     return tf.get_variable('biases', shape, initializer=initializer) 
    # use make_template to make variable reuse transparent 
    _biases = tf.make_template('_biases', _real_biases) 

    def _create_dense_layer(self, name, inputs, n_in, n_out, activation=True): 
     with tf.variable_scope(name): 
      weights = self._weights([n_in, n_out]) 
      layer = tf.matmul(inputs, weights) 
      if self.biases: 
       biases = self._biases([n_out]) 
       layer = layer + biases 
      if activation: 
       layer = self.activation(layer) 
      return layer 

    def _create_model(self, inputs): 
     n_in = self.n_features 
     for i in range(len(self.hidden)): 
      n_out = self.hidden[i] 
      name = 'hidden%d' % (i) 
      inputs = self._create_dense_layer(name, inputs, n_in, n_out) 
      n_in = n_out 
     output = self._create_dense_layer('output', inputs, n_in, self.n_classes, activation=False)  
     return output 

    def _create_loss_op(self, logits, labels): 
     cent = tf.nn.softmax_cross_entropy_with_logits(logits, labels) 
     weights = self.graph.get_collection('weights') 
     l2 = (self.l2/self.bsize) * tf.reduce_sum([tf.reduce_sum(tf.square(w)) for w in weights]) 
     return tf.reduce_mean(cent, name='loss') + l2 

    def _create_train_op(self, loss): 
     optimizer = tf.train.AdamOptimizer() 
     return optimizer.minimize(loss) 

    def _create_accuracy_op(self, logits, labels): 
     predictions = tf.nn.softmax(logits) 
     errors = tf.equal(tf.argmax(predictions, 1), tf.argmax(labels, 1)) 
     return tf.reduce_mean(tf.cast(errors, tf.float32)) 

    def train(self, images, labels): 
     logits = model._create_model(images) 
     loss = model._create_loss_op(logits, labels) 
     return model._create_train_op(loss)  

    def accuracy(self, images, labels): 
     logits = model._create_model(images) 
     return model._create_accuracy_op(logits, labels) 

    def predict(self, images): 
     return model._create_model(images) 

錯誤：

 
--------------------------------------------------------------------------- 
TypeError         Traceback (most recent call last) 
in() 
    25  model = MLP(hidden=[128]) 
    26  # define ops 
---> 27  train = model.train(images, labels) 
    28  accuracy = model.accuracy(eval_images, eval_labels) 
    29  # load test data and create a prediction op 

in train(self, images, labels) 
    60 
    61  def train(self, images, labels): 
---> 62   logits = model._create_model(images) 
    63   loss = model._create_loss_op(logits, labels) 
    64   return model._create_train_op(loss) 

in _create_model(self, inputs) 
    39    n_out = self.hidden[i] 
    40    name = 'hidden%d' % (i) 
---> 41    inputs = self._create_dense_layer(name, inputs, n_in, n_out) 
    42    n_in = n_out 
    43   output = self._create_dense_layer('output', inputs, n_in, self.n_classes, activation=False) 

in _create_dense_layer(self, name, inputs, n_in, n_out, activation) 
    25  def _create_dense_layer(self, name, inputs, n_in, n_out, activation=True): 
    26   with tf.variable_scope(name): 
---> 27    weights = self._weights([n_in, n_out]) 
    28    layer = tf.matmul(inputs, weights) 
    29    if self.biases: 

/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/template.py in __call__(self, *args, **kwargs) 
    265   self._unique_name, self._name) as vs: 
    266   self._var_scope = vs 
--> 267   return self._call_func(args, kwargs, check_for_new_variables=False) 
    268 
    269 @property 

/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/template.py in _call_func(self, args, kwargs, check_for_new_variables) 
    206   ops.get_collection(ops.GraphKeys.TRAINABLE_VARIABLES)) 
    207 
--> 208  result = self._func(*args, **kwargs) 
    209  if check_for_new_variables: 
    210   trainable_variables = ops.get_collection(

TypeError: _real_weights() missing 1 required positional argument: 'shape' 

originally defined at: 
    File "", line 1, in 
    class MLP(object): 
    File "", line 17, in MLP 
    _weights = tf.make_template('_weights', _real_weights) 

Answer 1

沒有與此代碼的多個問題，因爲它是在這裏，例如在train,accuracy和predict方法中的model參考文獻。我認爲這是由於從自然棲息地切割代碼。

原因你提到的TypeError，

TypeError: _real_weights() missing 1 required positional argument: 'shape' 

最有可能來自於以下事實：_real_weights本身就是MLP類的實例方法，而不是常規的函數或靜態方法。因此，函數的第一個參數始終是在調用時指向類的實例的self引用（類似於C語言的指針的明確版本，如this指針），如函數聲明中所示：

def _real_weights(self, shape): 
    initializer=tf.truncated_normal_initializer(stddev=0.1) 
    weights = tf.get_variable('weights', shape, initializer=initializer) 
    return weights 

請注意，即使您不使用參數，在這種情況下它仍然是必需的。使用

tf.make_template('_weights', self._real_weights) 

創建函數模板這樣，當你基本上說出您所創建的_weights模板應取兩個位置參數：self和weights（一樣的_real_weights方法）。因此，當你調用從模板創建的功能

weights = self._weights([n_in, n_out]) 

你傳遞數組到self的說法，離開（必填）shape參數不詳。

從什麼樣子，你就會有兩個選項：你既可以使_real_weights的MLP類之外的常規功能，使

def _real_weights(shape): 
    initializer=tf.truncated_normal_initializer(stddev=0.1) 
    weights = tf.get_variable('weights', shape, initializer=initializer) 
    return weights 

class MLP(): 
    # etc. 

這可能不是你想要的，因爲你已經創建了一個類的模型 - 或者你可以明確地使其成爲MLP類的靜態方法，使

class MLP(): 
    @staticmethod 
    def _real_weights(shape): 
     initializer=tf.truncated_normal_initializer(stddev=0.1) 
     weights = tf.get_variable('weights', shape, initializer=initializer) 
     return weights 

自定義靜態方法不上一個類實例進行操作，你可以（一d必須）省略self參考。

你會再創建模板作爲

tf.make_template('_weights', _real_weights) 

在第一種情況下

和

tf.make_template('_weights', MLP._real_weights) 

在第二種情況下，明確指定類MLP爲靜態方法的名稱範圍。無論哪種方式，_real_weights函數/方法和_weights模板現在都只有一個參數，即要創建的變量的形狀。