不知道爲什麼我的損失值在各個時期增加（tensorflow中的linreg）

Question

我知道，TF對於這類問題是矯枉過正，但這只是我自己介紹語法和TFs培訓過程的方式。

下面是代碼：

data = pd.read_excel("/Users/madhavthaker/Downloads/Reduced_Car_Data.xlsx") 

train = np.random.rand(len(data)) < 0.8 

data_train = data[train] 
data_test = data[~train] 


x_train = data_train.ix[:,0:3].values 
y_train = data_train.ix[:,-1].values 
x_test = data_test.ix[:,0:3].values 
y_test = data_test.ix[:,-1].values 

# Build inference graph. 
# Create Variables W and b that compute y_data = W * x_data + b 
W = tf.Variable(tf.random_normal([3,1]), name='weights') 
b = tf.Variable(tf.random_normal([1]), name='bias') 

# Uncomment the following lines to see what W and b are. 
# print(W) 
# print(b) 

# Create a placeholder we'll use later to feed x's into the graph for training and eval. 
# shape=[None] means we can put in any number of examples. 
# This is used for minibatch training, and to evaluate a lot of examples at once. 
x = tf.placeholder(tf.float32,shape=[x_train.shape[0],3], name='x') 

# Uncomment this line to see what x is 
# print(x) 

# This is the same as tf.add(tf.mul(W, x), b), but looks nicer 
y = tf.matmul(x,W) + b 

# Create a placeholder we'll use later to feed the correct y value into the graph 
y_label = tf.placeholder(shape=[y_train.shape[0],], dtype=tf.float32, name='y_label') 
# print (y_label) 

# Build training graph. 
loss = tf.reduce_mean(tf.square(y - y_label)) # Create an operation that calculates loss. 
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.00001) # Create an optimizer. 
train = optimizer.minimize(loss) # Create an operation that minimizes loss. 

# Uncomment the following 3 lines to see what 'loss', 'optimizer' and 'train' are. 
# print("loss:", loss) 
# print("optimizer:", optimizer) 
# print("train:", train) 
init = tf.global_variables_initializer() 

# Launch the graph 
with tf.Session() as sess: 
    sess.run(init) 

    # Fit all training data 
    for epoch in range(1000): 

     # Display logs per epoch step 
     if (epoch+1) % 50 == 0: 
      cost_val, hy_val, _ = sess.run([loss, y, train], feed_dict={x: x_train, y_label: y_train}) 
      print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(cost_val)) 

    print("Optimization Finished!") 
    training_cost = sess.run(loss, feed_dict={x: x_train, y_label: y_train}) 
    print("Training cost=", training_cost, "W=", sess.run(W), "b=", sess.run(b), '\n') 

有了結果：

Epoch: 0050 cost= 12377621.000000000 
Epoch: 0100 cost= 455768801280.000000000 
Epoch: 0150 cost= 16799577747226624.000000000 
Epoch: 0200 cost= 619229115796003225600.000000000 
Epoch: 0250 cost= 22824834360245537040498688.000000000 
Epoch: 0300 cost= 841322078804629437979012628480.000000000 
Epoch: 0350 cost= 31011140748122347114388001285734400.000000000 
Epoch: 0400 cost= inf 
Epoch: 0450 cost= inf 
Epoch: 0500 cost= inf 
Epoch: 0550 cost= inf 
Epoch: 0600 cost= inf 
Epoch: 0650 cost= inf 
Epoch: 0700 cost= inf 
Epoch: 0750 cost= inf 
Epoch: 0800 cost= inf 
Epoch: 0850 cost= nan 
Epoch: 0900 cost= nan 
Epoch: 0950 cost= nan 
Epoch: 1000 cost= nan 
Optimization Finished! 
Training cost= nan W= [[ nan] 
[ nan] 
[ nan]] b= [ nan] 

我在這個一直盯着了一會兒，我似乎無法弄清楚到底是怎麼回事上。任何幫助將非常感激。

Answer 1

我認爲這是由於您的成本函數的形狀。其實它可能會發生成本增加，看到答案那裏的數學解釋：https://datascience.stackexchange.com/questions/15962/why-is-learning-rate-causing-my-neural-networks-weights-to-skyrocket

也許試圖降低學習率，看看它是否有幫助。

PS：這是正常的，你不要在每個時代調用「sess.run（...）」？

Answer 2

我認爲模型太小而無法逼近所需的映射。我已經按照原樣在隨機數據上運行您的代碼，但損失沒有得到改善。它只有在我爲模型添加了一個圖層時纔有所改進。