如何改進簡單的網絡

我開始玩Keras和簡單的神經網絡。問題是關於正確性，以及下一步如何提高準確性。如何改進簡單的網絡

考慮數據集http://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients 其中有30K的例子和24個功能，目標是預測是否會有一個默認值。我創建了一個簡單的網絡，輸入層有24個輸入源，16個隱藏，還有一個最終的softmax層。損失是binary_crossentropy。試驗爲10％，validation_split爲20％

一個輸入線是

1,20000,2,2,1,24,2,2,-1,-1,-2,-2,3913,3102,689,0,0,0,0,689,0,0,0,0,1

代碼是

import pandas as pd 
from keras.models import Sequential 
from keras.layers.core import Dense, Activation, Dropout 
from keras.optimizers import SGD 
from sklearn.cross_validation import train_test_split 
from keras.utils import np_utils, generic_utils 

# load training in a panda dataframe and skip first line 
train = pd.read_csv('./data/defaulCC.csv', header=1) 
# split X, y 
X = train.iloc[:,:-1].values 
y = train.iloc[:,-1:].values 
dimof_input = X.shape[1] 
dimof_output = len(set(y.flat)) 
print('dimof_input: ', dimof_input) 
print('dimof_output: ', dimof_output) 

X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.9, random_state=0) 
y_train, y_test = [np_utils.to_categorical(x) for x in (y_train, y_test)] 
print X_train.shape, X_test.shape, y_train.shape, y_test.shape 

# Set constants 
batch_size = 128 
dimof_middle = 16 
dropout = 0.2 
countof_epoch = 100 
verbose = 1 
optimizer='sgd' 

print('batch_size: ', batch_size) 
print('dimof_middle: ', dimof_middle) 
print('dropout: ', dropout) 
print('countof_epoch: ', countof_epoch) 
print('verbose: ', verbose) 
print ('optimizer: ', optimizer) 

# this network has dimof_input n the input layer 
# dimof_output in the output layer 
# dimof_middle in the hidden layer 

model = Sequential() 
model.add(Dense(dimof_middle, input_dim=dimof_input, activation='relu')) 
model.add(Dropout(dropout)) 
#model.add(Dense(dimof_middle, activation='relu')) 
#model.add(Dropout(dropout)) 
model.add(Dense(dimof_output, activation='softmax')) 

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True) 
model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy']) 

# Train 
model.fit(
X_train, y_train, 
validation_split=0.2, 
batch_size=batch_size, nb_epoch=countof_epoch, verbose=verbose) 

# Evaluate 
loss, accuracy = model.evaluate(X_test, y_test, verbose=verbose) 
print('loss: ', loss) 
print('accuracy: ', accuracy) 
print()

輸出是

Using Theano backend. 
('dimof_input: ', 24) 
('dimof_output: ', 2) 
(27000, 24) (3000, 24) (27000, 2) (3000, 2) 
('batch_size: ', 128) 
('dimof_middle: ', 16) 
('dropout: ', 0.2) 
('countof_epoch: ', 100) 
('verbose: ', 1) 
('optimizer: ', 'sgd') 
Train on 27000 samples, validate on 5400 samples 
Epoch 1/100 
27000/27000 [==============================] - 0s - loss: 3.6371 - acc: 0.7727 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 2/100 
27000/27000 [==============================] - 0s - loss: 3.5866 - acc: 0.7757 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 3/100 
27000/27000 [==============================] - 0s - loss: 3.6024 - acc: 0.7750 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 4/100 
27000/27000 [==============================] - 0s - loss: 3.5859 - acc: 0.7758 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 5/100 
27000/27000 [==============================] - 0s - loss: 3.5854 - acc: 0.7761 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 6/100 
27000/27000 [==============================] - 0s - loss: 3.5883 - acc: 0.7760 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 7/100 
27000/27000 [==============================] - 0s - loss: 3.5855 - acc: 0.7761 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 8/100 
27000/27000 [==============================] - 0s - loss: 3.5854 - acc: 0.7761 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 9/100 
27000/27000 [==============================] - 0s - loss: 3.5847 - acc: 0.7762 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 10/100 
27000/27000 [==============================] - 0s - loss: 3.5900 - acc: 0.7760 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 11/100 
27000/27000 [==============================] - 0s - loss: 3.5689 - acc: 0.7773 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 12/100 
27000/27000 [==============================] - 0s - loss: 3.5665 - acc: 0.7775 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 13/100 
27000/27000 [==============================] - 0s - loss: 3.5653 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 14/100 
27000/27000 [==============================] - 0s - loss: 3.5701 - acc: 0.7773 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 15/100 
27000/27000 [==============================] - 0s - loss: 3.5582 - acc: 0.7780 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 16/100 
27000/27000 [==============================] - 0s - loss: 3.5682 - acc: 0.7774 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 17/100 
27000/27000 [==============================] - 0s - loss: 3.5665 - acc: 0.7775 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 18/100 
27000/27000 [==============================] - 0s - loss: 3.5648 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 19/100 
27000/27000 [==============================] - 0s - loss: 3.5636 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 20/100 
27000/27000 [==============================] - 0s - loss: 3.5700 - acc: 0.7772 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 21/100 
27000/27000 [==============================] - 0s - loss: 3.5597 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 22/100 
27000/27000 [==============================] - 0s - loss: 3.5597 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 23/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 24/100 
27000/27000 [==============================] - 0s - loss: 3.5590 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 25/100 
27000/27000 [==============================] - 0s - loss: 3.5573 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 26/100 
27000/27000 [==============================] - 0s - loss: 3.5590 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 27/100 
27000/27000 [==============================] - 0s - loss: 3.5621 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 28/100 
27000/27000 [==============================] - 0s - loss: 3.5581 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 29/100 
27000/27000 [==============================] - 0s - loss: 3.5576 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 30/100 
27000/27000 [==============================] - 0s - loss: 3.5590 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 31/100 
27000/27000 [==============================] - 0s - loss: 3.5575 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 32/100 
27000/27000 [==============================] - 0s - loss: 3.5598 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 33/100 
27000/27000 [==============================] - 0s - loss: 3.5604 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 34/100 
27000/27000 [==============================] - 0s - loss: 3.5609 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 35/100 
27000/27000 [==============================] - 0s - loss: 3.5598 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 36/100 
27000/27000 [==============================] - 0s - loss: 3.5575 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 37/100 
27000/27000 [==============================] - 0s - loss: 3.5592 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 38/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 39/100 
27000/27000 [==============================] - 0s - loss: 3.5637 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 40/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 41/100 
27000/27000 [==============================] - 0s - loss: 3.5584 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 42/100 
27000/27000 [==============================] - 0s - loss: 3.5564 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 43/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 44/100 
27000/27000 [==============================] - 0s - loss: 3.5576 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 45/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 46/100 
27000/27000 [==============================] - 0s - loss: 3.5595 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 47/100 
27000/27000 [==============================] - 0s - loss: 3.5581 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 48/100 
27000/27000 [==============================] - 0s - loss: 3.5598 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 49/100 
27000/27000 [==============================] - 0s - loss: 3.5576 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 50/100 
27000/27000 [==============================] - 0s - loss: 3.5610 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 51/100 
27000/27000 [==============================] - 0s - loss: 3.5616 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 52/100 
27000/27000 [==============================] - 0s - loss: 3.5598 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 53/100 
27000/27000 [==============================] - 0s - loss: 3.5569 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 54/100 
27000/27000 [==============================] - 0s - loss: 3.5589 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 55/100 
27000/27000 [==============================] - 0s - loss: 3.5569 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 56/100 
27000/27000 [==============================] - 0s - loss: 3.5563 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 57/100 
27000/27000 [==============================] - 0s - loss: 3.5598 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 58/100 
27000/27000 [==============================] - 0s - loss: 3.5607 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 59/100 
27000/27000 [==============================] - 0s - loss: 3.5611 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 60/100 
27000/27000 [==============================] - 0s - loss: 3.5558 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 61/100 
27000/27000 [==============================] - 0s - loss: 3.5620 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 62/100 
27000/27000 [==============================] - 0s - loss: 3.5592 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 63/100 
27000/27000 [==============================] - 0s - loss: 3.5608 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 64/100 
27000/27000 [==============================] - 0s - loss: 3.5587 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 65/100 
27000/27000 [==============================] - 0s - loss: 3.5586 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 66/100 
27000/27000 [==============================] - 0s - loss: 3.5608 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 67/100 
27000/27000 [==============================] - 0s - loss: 3.5605 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 68/100 
27000/27000 [==============================] - 0s - loss: 3.5598 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 69/100 
27000/27000 [==============================] - 0s - loss: 3.5621 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 70/100 
27000/27000 [==============================] - 0s - loss: 3.5607 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 71/100 
27000/27000 [==============================] - 0s - loss: 3.5609 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 72/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 73/100 
27000/27000 [==============================] - 0s - loss: 3.5586 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 74/100 
27000/27000 [==============================] - 0s - loss: 3.5603 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 75/100 
27000/27000 [==============================] - 0s - loss: 3.5625 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 76/100 
27000/27000 [==============================] - 0s - loss: 3.5573 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 77/100 
27000/27000 [==============================] - 0s - loss: 3.5590 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 78/100 
27000/27000 [==============================] - 0s - loss: 3.5608 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 79/100 
27000/27000 [==============================] - 0s - loss: 3.5613 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 80/100 
27000/27000 [==============================] - 0s - loss: 3.5564 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 81/100 
27000/27000 [==============================] - 0s - loss: 3.5638 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 82/100 
27000/27000 [==============================] - 0s - loss: 3.5609 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 83/100 
27000/27000 [==============================] - 0s - loss: 3.5591 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 84/100 
27000/27000 [==============================] - 0s - loss: 3.5599 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 85/100 
27000/27000 [==============================] - 0s - loss: 3.5615 - acc: 0.7776 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 86/100 
27000/27000 [==============================] - 0s - loss: 3.5616 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 87/100 
27000/27000 [==============================] - 0s - loss: 3.5616 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 88/100 
27000/27000 [==============================] - 0s - loss: 3.5586 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 89/100 
27000/27000 [==============================] - 0s - loss: 3.5615 - acc: 0.7777 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 90/100 
27000/27000 [==============================] - 0s - loss: 3.5581 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 91/100 
27000/27000 [==============================] - 0s - loss: 3.5580 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 92/100 
27000/27000 [==============================] - 0s - loss: 3.5586 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 93/100 
27000/27000 [==============================] - 0s - loss: 3.5611 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 94/100 
27000/27000 [==============================] - 0s - loss: 3.5589 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 95/100 
27000/27000 [==============================] - 0s - loss: 3.5595 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 96/100 
27000/27000 [==============================] - 0s - loss: 3.5623 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 97/100 
27000/27000 [==============================] - 0s - loss: 3.5623 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 98/100 
27000/27000 [==============================] - 0s - loss: 3.5605 - acc: 0.7779 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 99/100 
27000/27000 [==============================] - 0s - loss: 3.5612 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
Epoch 100/100 
27000/27000 [==============================] - 0s - loss: 3.5612 - acc: 0.7778 - val_loss: 3.6157 - val_acc: 0.7744 
3000/3000 [==============================] - 0s  
('loss: ', 3.4197844168345135) 
('accuracy: ', 0.78666666682561237) 
()

來源

2016-04-21 ScientistBoy

損耗不降低，這是一個問題。有幾個你可以做的改進：

規範化你的數據。在您的具體示例中，所有功能都具有不同的範圍，因此規範化它們是神經網絡成功學習的必備條件。爲了標準化，您可以計算每個要素的平均值和標準偏差，並將平均值和除以標準偏差，這將使特徵保留在近似[-1,1]的範圍內。最小 - 最大標準化也應該起作用。
標準化後，您可以通過添加更多圖層來增加模型的容量。你也可以嘗試使用不同的非線性像sigmoid。
增加正則化。你使用p = 0.2的Dropout，你可以將它增加到p = 0.5。或者，您可以使用批量標準化。

這個答案很通用，但是這個建議應該適用於多種數據。

來源

2016-04-23 19:55:40

如何改進簡單的網絡

回答

相關問題