2016-10-03 42 views
0

我正在嘗試使用張量流來訓練序列模型。我在教程中看到,桶有助於加速培訓。到目前爲止,我只能使用一個桶進行訓練,並且只使用一個gpu和多個桶來使用或多或少的開箱即用代碼,但是當我嘗試使用具有多個gpus的多個桶時,出現錯誤,指出 參數無效:您必須養活佔位符張量「gpu_scope_0/encoder50_gpu0」與D型的值INT32帶桶的Tensorflow錯誤

從錯誤中,我可以告訴大家,我不是正確聲明input_feed,所以它期待的輸入是的每次最大桶的大小。不過,我很困惑爲什麼會出現這種情況,因爲在我正在調整的示例中,它在初始化input_feed的佔位符時會執行相同的操作。據我所知,教程也初始化爲最大容量的桶,但是當我使用教程的代碼時,這個錯誤不會發生。

以下是我認爲是相關的初始化代碼:

self.encoder_inputs = [[] for _ in xrange(self.num_gpus)] 
    self.decoder_inputs = [[] for _ in xrange(self.num_gpus)] 
    self.target_weights = [[] for _ in xrange(self.num_gpus)] 
    self.scope_prefix = "gpu_scope" 
    for j in xrange(self.num_gpus): 
     with tf.device("/gpu:%d" % (self.gpu_offset + j)): 
      with tf.name_scope('%s_%d' % (self.scope_prefix, j)) as scope: 
       for i in xrange(buckets[-1][0]): # Last bucket is the biggest one. 
        self.encoder_inputs[j].append(tf.placeholder(tf.int32, shape=[None], 
                   name="encoder{0}_gpu{1}".format(i,j))) 
       for i in xrange(buckets[-1][1] + 1): 
        self.decoder_inputs[j].append(tf.placeholder(tf.int32, shape=[None], 
                   name="decoder{0}_gpu{1}".format(i,j))) 
        self.target_weights[j].append(tf.placeholder(tf.float32, shape=[None], 
                   name="weight{0}_gpu{1}".format(i,j))) 

    # Our targets are decoder inputs shifted by one. 
    self.losses = [] 
    self.outputs = [] 

    # The following loss computation creates the neural network. The specified 
    # device hosts the trainable tf parameters. 
    bucket = buckets[0] 
    i = 0 
    with tf.device(param_device): 
     output, loss = tf.nn.seq2seq.model_with_buckets(self.encoder_inputs[i], self.decoder_inputs[i], 
                 [self.decoder_inputs[i][k + 1] for k in 
                 xrange(len(self.decoder_inputs[i]) - 1)], 
                 self.target_weights[0], buckets, 
                 lambda x, y: seq2seq_f(x, y, True), 
                 softmax_loss_function=self.softmax_loss_function) 

    bucket = buckets[0] 
    self.encoder_states = [] 
    with tf.device('/gpu:%d' % self.gpu_offset): 
     with variable_scope.variable_scope(variable_scope.get_variable_scope(), 
              reuse=True): 
      self.encoder_outputs, self.encoder_states = get_encoder_outputs(self, 
                      self.encoder_inputs[0]) 

    if not forward_only: 
     self.grads = [] 
    print ("past line 297") 
    done_once = False 
    for i in xrange(self.num_gpus): 
     with tf.device("/gpu:%d" % (self.gpu_offset + i)): 
      with tf.name_scope("%s_%d" % (self.scope_prefix, i)) as scope: 
       with variable_scope.variable_scope(variable_scope.get_variable_scope(), reuse=True): 
        #for j, bucket in enumerate(buckets): 
        output, loss = tf.nn.seq2seq.model_with_buckets(self.encoder_inputs[i], 
                    self.decoder_inputs[i], 
                    [self.decoder_inputs[i][k + 1] for k in 
                    xrange(len(self.decoder_inputs[i]) - 1)], 
                    self.target_weights[i], buckets, 
                    lambda x, y: seq2seq_f(x, y, True), 
                    softmax_loss_function=self.softmax_loss_function) 

        self.losses.append(loss) 
        self.outputs.append(output) 


    # Training outputs and losses. 
    if forward_only: 
     self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(
      self.encoder_inputs, self.decoder_inputs, 
      [self.decoder_inputs[0][k + 1] for k in xrange(buckets[0][1])], 
      self.target_weights, buckets, lambda x, y: seq2seq_f(x, y, True), 
      softmax_loss_function=self.softmax_loss_function) 
     # If we use output projection, we need to project outputs for decoding. 
     if self.output_projection is not None: 
      for b in xrange(len(buckets)): 
       self.outputs[b] = [ 
        tf.matmul(output, self.output_projection[0]) + self.output_projection[1] 
        for output in self.outputs[b] 
        ] 
    else: 
     self.bucket_grads = [] 
     self.gradient_norms = [] 
     params = tf.trainable_variables() 
     opt = tf.train.GradientDescentOptimizer(self.learning_rate) 
     self.updates = [] 
     with tf.device(aggregation_device): 
      for g in xrange(self.num_gpus): 
       for b in xrange(len(buckets)): 
        gradients = tf.gradients(self.losses[g][b], params) 
        clipped_grads, norm = tf.clip_by_global_norm(gradients, max_gradient_norm) 
        self.gradient_norms.append(norm) 
        self.updates.append(
         opt.apply_gradients(zip(clipped_grads, params), global_step=self.global_step)) 

,並在數據餵奶時,以下是相關代碼:眼下

input_feed = {} 
     for i in xrange(self.num_gpus): 
     for l in xrange(encoder_size): 
      input_feed[self.encoder_inputs[i][l].name] = encoder_inputs[i][l] 
     for l in xrange(decoder_size): 
      input_feed[self.decoder_inputs[i][l].name] = decoder_inputs[i][l] 
      input_feed[self.target_weights[i][l].name] = target_weights[i][l] 

     # Since our targets are decoder inputs shifted by one, we need one more. 
     last_target = self.decoder_inputs[i][decoder_size].name 
     input_feed[last_target] = np.zeros([self.batch_size], dtype=np.int32) 

     last_weight = self.target_weights[i][decoder_size].name 
     input_feed[last_weight] = np.zeros([self.batch_size], dtype=np.float32) 
    # Output feed: depends on whether we do a backward step or not. 

    if not forward_only: 
     output_feed = [self.updates[bucket_id], self.gradient_norms[bucket_id], self.losses[bucket_id]] 
    else: 
     output_feed = [self.losses[bucket_id]] # Loss for this batch. 
     for l in xrange(decoder_size): # Output logits. 
      output_feed.append(self.outputs[0][l]) 

我只考慮填充每個輸入達到桶的大小,但我認爲這會失去一些桶裝的優點

回答

0

原來這個問題並沒有在第e佔位符,但後來在我的代碼中我提到了未初始化的佔位符。據我所知,當我修復後面的問題時,這個錯誤停止了。