在while_loop的上下文中使用TensorArrays來累計值

Question

下面我有一個Tensorflow RNN單元的實現，該單元用於模擬本文中的Alex Graves算法ACT：http://arxiv.org/abs/1603.08983。

在通過rnn.rnn調用的序列中的單個時間步驟（帶有靜態sequence_length參數，因此rnn動態展開 - 我使用的是固定批量大小爲20），我們遞歸調用ACTStep，生成大小輸出（1,200）其中RNN單元格的隱藏維度爲200，批量大小爲1.

在Tensorflow中使用while循環，我們進行迭代，直到累計停止概率足夠高。所有這些工作都相當順利，但是我在while循環中存在積累狀態，概率和輸出的問題，爲了創建這些作爲最終單元輸出/狀態的加權組合，我們需要做這些工作。

我嘗試過使用一個簡單的列表，如下所示，但是當圖形編譯爲輸出不在同一個框架中時失敗（是否可以使用control_flow_ops中的「開關」功能將張量轉發到在需要它們的地方，即我們返回值之前的add_n函數？）。我也嘗試過使用TensorArray結構，但是我發現這很難使用，因爲它似乎破壞了形狀信息，並且手動替換它沒有奏效。我也無法找到關於TensorArrays的大量文檔，據推測，他們是，我想，主要是用於TF的內部使用。

任何有關如何正確累積由ACTStep生成的變量的建議將非常感謝。

class ACTCell(RNNCell): 
"""An RNN cell implementing Graves' Adaptive Computation time algorithm""" 
def __init__(self, num_units, cell, epsilon, max_computation): 

    self.one_minus_eps = tf.constant(1.0 - epsilon) 
    self._num_units = num_units 
    self.cell = cell 
    self.N = tf.constant(max_computation) 
@property 
def input_size(self): 
    return self._num_units 
@property 
def output_size(self): 
    return self._num_units 
@property 
def state_size(self): 
    return self._num_units 

def __call__(self, inputs, state, scope=None): 

    with vs.variable_scope(scope or type(self).__name__): 

     # define within cell constants/ counters used to control while loop 
     prob = tf.get_variable("prob", [], tf.float32,tf.constant_initializer(0.0)) 
     counter = tf.get_variable("counter", [],tf.float32,tf.constant_initializer(0.0)) 
     tf.assign(prob,0.0) 
     tf.assign(counter, 0.0) 

     # the predicate for stopping the while loop. Tensorflow demands that we have 
     # all of the variables used in the while loop in the predicate. 
     pred = lambda prob,counter,state,input,\ 
         acc_state,acc_output,acc_probs:\ 
      tf.logical_and(tf.less(prob,self.one_minus_eps), tf.less(counter,self.N)) 

     acc_probs = [] 
     acc_outputs = [] 
     acc_states = [] 


     _,iterations,_,_,acc_states,acc_output,acc_probs = \ 
     control_flow_ops.while_loop(pred, 
     self.ACTStep, 
     [prob,counter,state,input,acc_states,acc_outputs,acc_probs]) 

    # TODO:fix last part of this, need to use the remainder. 
    # TODO: find a way to accumulate the regulariser 

    # here we take a weighted combination of the states and outputs 
    # to use as the actual output and state which is passed to the next timestep. 

    next_state = tf.add_n([tf.mul(x,y) for x,y in zip(acc_probs,acc_states)]) 
    output = tf.add_n([tf.mul(x,y) for x,y in zip(acc_probs,acc_outputs)]) 


    return output, next_state 

def ACTStep(self,prob,counter,state,input, acc_states,acc_outputs,acc_probs): 

    output, new_state = rnn.rnn(self.cell, [input], state, scope=type(self.cell).__name__) 

    prob_w = tf.get_variable("prob_w", [self.cell.input_size,1]) 
    prob_b = tf.get_variable("prob_b", [1]) 
    p = tf.nn.sigmoid(tf.matmul(prob_w,new_state) + prob_b) 

    acc_states.append(new_state) 
    acc_outputs.append(output) 
    acc_probs.append(p) 

    return [tf.add(prob,p),tf.add(counter,1.0),new_state, input,acc_states,acc_outputs,acc_probs] 

Answer 1

我要在這個回答前說，這不是一個完整的解決方案，而是一些關於如何改善你的細胞的評論。

要開始了，你ACTStep功能，你叫rnn.rnn一個時間步長（由[input]定義。如果你正在做一個時間步長，它可能是更有效地使用簡單實際self.cell通話功能。你」 LL看到tensorflow rnncell wrappers

你提到你已經使用TensorArrays試圖用同樣的機制。你打包，並適當解壓tensorarrays？這裏是一個repo在那裏你會model.py下找到tensorarrays打包和拆包正常。

你也問過如果control_flow_ops中有一個功能需要累積所有的張量。我認爲您正在尋找tf.control_dependencies

您可以在control_dependicies中列出所有輸出張量操作，並且需要張量流來計算所有張量到該點。

此外，它看起來像你的counter變量是可訓練的。你確定你想要這樣嗎？如果你在櫃檯上加1，那可能不會產生正確的結果。另一方面，你可以故意保持它的可訓練性，以便在思考成本函數的最後區分它。

而且我相信餘數的函數應該在你的腳本：

remainder = 1.0 - tf.add_n(acc_probs[:-1]) 
#note that there is a -1 in the list as you do not want to grab the last probability 

這裏是我的代碼的版本編輯：

class ACTCell(RNNCell): 
    """An RNN cell implementing Graves' Adaptive Computation time algorithm 
    Notes: https://www.evernote.com/shard/s189/sh/fd165646-b630-48b7-844c-86ad2f07fcda/c9ab960af967ef847097f21d94b0bff7 

    """ 
    def __init__(self, num_units, cell, max_computation = 5.0, epsilon = 0.01): 

     self.one_minus_eps = tf.constant(1.0 - epsilon) #episolon is 0.01 as found in the paper 
     self._num_units = num_units 
     self.cell = cell 
     self.N = tf.constant(max_computation) 

    @property 
    def input_size(self): 
     return self._num_units 
    @property 
    def output_size(self): 
     return self._num_units 
    @property 
    def state_size(self): 
     return self._num_units 

    def __call__(self, inputs, state, scope=None): 

     with vs.variable_scope(scope or type(self).__name__): 

      # define within cell constants/ counters used to control while loop 
      prob = tf.constant(0.0, shape = [batch_size]) 
      counter = tf.constant(0.0, shape = [batch_size]) 

      # the predicate for stopping the while loop. Tensorflow demands that we have 
      # all of the variables used in the while loop in the predicate. 
      pred = lambda prob,counter,state,input,acc_states,acc_output,acc_probs:\ 
       tf.logical_and(tf.less(prob,self.one_minus_eps), tf.less(counter,self.N)) 

      acc_probs, acc_outputs, acc_states = [], [], [] 

      _,iterations,_,_,acc_states,acc_output,acc_probs = \ 
      control_flow_ops.while_loop(
      pred, 
      self.ACTStep, #looks like he purposely makes the while loop here 
      [prob, counter, state, input, acc_states, acc_outputs, acc_probs]) 

     '''mean-field updates for states and outputs''' 
     next_state = tf.add_n([x*y for x,y in zip(acc_probs,acc_states)]) 
     output = tf.add_n([x*y for x,y in zip(acc_probs,acc_outputs)]) 

     remainder = 1.0 - tf.add_n(acc_probs[:-1]) #you take the last off to avoid a negative ponder cost #the problem here is we need to take the sum of all the remainders 
     tf.add_to_collection("ACT_remainder", remainder) #if this doesnt work then you can do self.list based upon timesteps 
     tf.add_to_collection("ACT_iterations", iterations) 
     return output, next_state 

    def ACTStep(self,prob, counter, state, input, acc_states, acc_outputs, acc_probs): 

     '''run rnn once''' 
     output, new_state = rnn.rnn(self.cell, [input], state, scope=type(self.cell).__name__) 

     prob_w = tf.get_variable("prob_w", [self.cell.input_size,1]) 
     prob_b = tf.get_variable("prob_b", [1]) 
     halting_probability = tf.nn.sigmoid(tf.matmul(prob_w,new_state) + prob_b) 


     acc_states.append(new_state) 
     acc_outputs.append(output) 
     acc_probs.append(halting_probability) 

     return [p + prob, counter + 1.0, new_state, input,acc_states,acc_outputs,acc_probs] 


    def PonderCostFunction(self, time_penalty = 0.01): 
     ''' 
     note: ponder is completely different than probability and ponder = roe 

     the ponder cost function prohibits the rnn from cycling endlessly on each timestep when not much is needed 
     ''' 
     n_iterations = tf.get_collection_ref("ACT_iterations") 
     remainder = tf.get_collection_ref("ACT_remainder") 
     return tf.reduce_sum(n_iterations + remainder) #completely different from probability 

這是實現我一直是個複雜的紙在自己工作。我不介意與你合作在Tensorflow中完成它。如果您有興趣，請將我添加到Skype的LeavesBreathe上，我們可以從那裏開始。