我第一次使用keras + tensorflow。我想指定correlation coefficient作爲損失函數。這是有道理的,以便它是一個介於0和1之間的數字,其中0是不好的,1是好的。如何指定相關係數作爲keras中的損失函數
我的基本代碼,目前看起來像:
def baseline_model():
model = Sequential()
model.add(Dense(4000, input_dim=n**2, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')
return model
estimators = []
estimators.append(('standardize', StandardScaler()))
estimators.append(('mlp', KerasRegressor(build_fn=baseline_model, epochs=100, batch_size=32, verbose=2)))
pipeline = Pipeline(estimators)
kfold = KFold(n_splits=10, random_state=0)
results = cross_val_score(pipeline, X, Y, cv=kfold)
print("Standardized: %.2f (%.2f) MSE" % (results.mean(), results.std()))
我怎樣才能改變這種做法,它優化,而不是減少平方相關係數?
我試過如下:
def correlation_coefficient(y_true, y_pred):
pearson_r, _ = tf.contrib.metrics.streaming_pearson_correlation(y_pred, y_true)
return 1-pearson_r**2
def baseline_model():
# create model
model = Sequential()
model.add(Dense(4000, input_dim=n**2, kernel_initializer='normal', activation='relu'))
# model.add(Dense(2000, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss=correlation_coefficient, optimizer='adam')
return model
但這與崩潰:
Traceback (most recent call last):
File "deeplearning-det.py", line 67, in <module>
results = cross_val_score(pipeline, X, Y, cv=kfold)
File "/home/user/.local/lib/python3.5/site-packages/sklearn/model_selection/_validation.py", line 321, in cross_val_score
pre_dispatch=pre_dispatch)
File "/home/user/.local/lib/python3.5/site-packages/sklearn/model_selection/_validation.py", line 195, in cross_validate
for train, test in cv.split(X, y, groups))
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 779, in __call__
while self.dispatch_one_batch(iterator):
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 625, in dispatch_one_batch
self._dispatch(tasks)
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 588, in _dispatch
job = self._backend.apply_async(batch, callback=cb)
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 111, in apply_async
result = ImmediateResult(func)
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 332, in __init__
self.results = batch()
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 131, in __call__
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/home/user/.local/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 131, in <listcomp>
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/home/user/.local/lib/python3.5/site-packages/sklearn/model_selection/_validation.py", line 437, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File "/home/user/.local/lib/python3.5/site-packages/sklearn/pipeline.py", line 259, in fit
self._final_estimator.fit(Xt, y, **fit_params)
File "/home/user/.local/lib/python3.5/site-packages/keras/wrappers/scikit_learn.py", line 147, in fit
history = self.model.fit(x, y, **fit_args)
File "/home/user/.local/lib/python3.5/site-packages/keras/models.py", line 867, in fit
initial_epoch=initial_epoch)
File "/home/user/.local/lib/python3.5/site-packages/keras/engine/training.py", line 1575, in fit
self._make_train_function()
File "/home/user/.local/lib/python3.5/site-packages/keras/engine/training.py", line 960, in _make_train_function
loss=self.total_loss)
File "/home/user/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/home/user/.local/lib/python3.5/site-packages/keras/optimizers.py", line 432, in get_updates
m_t = (self.beta_1 * m) + (1. - self.beta_1) * g
File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 856, in binary_op_wrapper
y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 611, in convert_to_tensor
as_ref=False)
File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 676, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 121, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/home/user/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 364, in make_tensor_proto
raise ValueError("None values not supported.")
ValueError: None values not supported.
更新1
繼答案BEL現在代碼運行。不幸的是,correlation_coefficient
和correlation_coefficient_loss
函數給出了彼此不同的值,我不確定它們中的任何一個與您從1- scipy.stats.pearsonr()[0] ** 2獲得的值是否相同。
Why are loss functions giving the wrong outputs and how can they be corrected to give the same values as
1 - scipy.stats.pearsonr()[0]**2
would give?
這裏是完全獨立的代碼,應該只運行:2
我已經放棄了correlation_coefficient
功能,我現在只使用correlation_coefficient_loss
一個
import numpy as np
import sys
import math
from scipy.stats import ortho_group
from scipy.stats import pearsonr
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import tensorflow as tf
from keras import backend as K
def permanent(M):
n = M.shape[0]
d = np.ones(n)
j = 0
s = 1
f = np.arange(n)
v = M.sum(axis=0)
p = np.prod(v)
while (j < n-1):
v -= 2*d[j]*M[j]
d[j] = -d[j]
s = -s
prod = np.prod(v)
p += s*prod
f[0] = 0
f[j] = f[j+1]
f[j+1] = j+1
j = f[0]
return p/2**(n-1)
def correlation_coefficient_loss(y_true, y_pred):
x = y_true
y = y_pred
mx = K.mean(x)
my = K.mean(y)
xm, ym = x-mx, y-my
r_num = K.sum(xm * ym)
r_den = K.sum(K.sum(K.square(xm)) * K.sum(K.square(ym)))
r = r_num/r_den
return 1 - r**2
def correlation_coefficient(y_true, y_pred):
pearson_r, update_op = tf.contrib.metrics.streaming_pearson_correlation(y_pred, y_true)
# find all variables created for this metric
metric_vars = [i for i in tf.local_variables() if 'correlation_coefficient' in i.name.split('/')[1]]
# Add metric variables to GLOBAL_VARIABLES collection.
# They will be initialized for new session.
for v in metric_vars:
tf.add_to_collection(tf.GraphKeys.GLOBAL_VARIABLES, v)
# force to update metric values
with tf.control_dependencies([update_op]):
pearson_r = tf.identity(pearson_r)
return 1-pearson_r**2
def baseline_model():
# create model
model = Sequential()
model.add(Dense(4000, input_dim=no_rows**2, kernel_initializer='normal', activation='relu'))
# model.add(Dense(2000, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss=correlation_coefficient_loss, optimizer='adam', metrics=[correlation_coefficient])
return model
no_rows = 8
print("Making the input data using seed 7", file=sys.stderr)
np.random.seed(7)
U = ortho_group.rvs(no_rows**2)
U = U[:, :no_rows]
# U is a random orthogonal matrix
X = []
Y = []
print(U)
for i in range(40000):
I = np.random.choice(no_rows**2, size = no_rows)
A = U[I][np.lexsort(np.rot90(U[I]))]
X.append(A.ravel())
Y.append(-math.log(permanent(A)**2, 2))
X = np.array(X)
Y = np.array(Y)
estimators = []
estimators.append(('standardize', StandardScaler()))
estimators.append(('mlp', KerasRegressor(build_fn=baseline_model, epochs=100, batch_size=32, verbose=2)))
pipeline = Pipeline(estimators)
X_train, X_test, y_train, y_test = train_test_split(X, Y,
train_size=0.75, test_size=0.25)
pipeline.fit(X_train, y_train)
更新如下面的JulioDanielReyes所給出的。但是,這仍然是錯誤的,或者keras顯着過度擬合。甚至當我有:
def baseline_model():
model = Sequential()
model.add(Dense(40, input_dim=no_rows**2, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
model.compile(loss=correlation_coefficient_loss, optimizer='adam', metrics=[correlation_coefficient_loss])
return model
我得到的損失,例如,0.6653後100時代,但0.857當我測試訓練的模型。
How can it be overfitting which such a tiny number of nodes in the hidden layer?
您是否嘗試過1-K.square(pearson_r)? –
@DanielMöller不,我沒有。你能介紹一下你想要的嗎? – eleanora
準確地說,而不是'1 - pearson_r ** 2'。 –