如何在張量流中將矢量和矩陣相乘而不重新變形？

Question

此：

import numpy as np 
a = np.array([1, 2, 1]) 
w = np.array([[.5, .6], [.7, .8], [.7, .8]]) 

print(np.dot(a, w)) 
# [ 2.6 3. ] # plain nice old matrix multiplication n x (n, m) -> m 

import tensorflow as tf 

a = tf.constant(a, dtype=tf.float64) 
w = tf.constant(w) 

with tf.Session() as sess: 
    print(tf.matmul(a, w).eval()) 

結果：

C:\_\Python35\python.exe C:/Users/MrD/.PyCharm2017.1/config/scratches/scratch_31.py 
[ 2.6 3. ] 
# bunch of errors in windows... 
Traceback (most recent call last): 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 671, in _call_cpp_shape_fn_impl 
    input_tensors_as_shapes, status) 
    File "C:\_\Python35\lib\contextlib.py", line 66, in __exit__ 
    next(self.gen) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 466, in raise_exception_on_not_ok_status 
    pywrap_tensorflow.TF_GetCode(status)) 
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 2 but is rank 1 for 'MatMul' (op: 'MatMul') with input shapes: [3], [3,2]. 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "C:/Users/MrD/.PyCharm2017.1/config/scratches/scratch_31.py", line 14, in <module> 
    print(tf.matmul(a, w).eval()) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1765, in matmul 
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 1454, in _mat_mul 
    transpose_b=transpose_b, name=name) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op 
    op_def=op_def) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 2329, in create_op 
    set_shapes_for_outputs(ret) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 1717, in set_shapes_for_outputs 
    shapes = shape_func(op) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 1667, in call_with_requiring 
    return call_cpp_shape_fn(op, require_shape_fn=True) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 610, in call_cpp_shape_fn 
    debug_python_shape_fn, require_shape_fn) 
    File "C:\_\Python35\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 676, in _call_cpp_shape_fn_impl 
    raise ValueError(err.message) 
ValueError: Shape must be rank 2 but is rank 1 for 'MatMul' (op: 'MatMul') with input shapes: [3], [3,2]. 

Process finished with exit code 1 

（不知道爲什麼同樣異常的處理內提出）

該解決方案在Tensorflow exception with matmul建議正在重塑向量矩陣但是這會導致不必要的複雜代碼 - 是否仍然沒有其他方法將矢量與矩陣相乘？

順便提及使用expand_dims用默認參數（如在上面的鏈接建議的）提出了一個ValueError - 這不是在docs提到並且擊敗具有默認參數的目的。

Answer 1

Matmul被編碼爲等級2或更高的張量。不確定爲什麼老實說numpy有它，它也允許矩陣向量乘法。

import numpy as np 
a = np.array([1, 2, 1]) 
w = np.array([[.5, .6], [.7, .8], [.7, .8]]) 

print(np.dot(a, w)) 
# [ 2.6 3. ] # plain nice old matix multiplication n x (n, m) -> m 
print(np.sum(np.expand_dims(a, -1) * w , axis=0)) 
# equivalent result [2.6, 3] 

import tensorflow as tf 

a = tf.constant(a, dtype=tf.float64) 
w = tf.constant(w) 

with tf.Session() as sess: 
    # they all produce the same result as numpy above 
    print(tf.matmul(tf.expand_dims(a,0), w).eval()) 
    print((tf.reduce_sum(tf.multiply(tf.expand_dims(a,-1), w), axis=0)).eval()) 
    print((tf.reduce_sum(tf.multiply(a, tf.transpose(w)), axis=1)).eval()) 

    # Note tf.multiply is equivalent to "*" 
    print((tf.reduce_sum(tf.expand_dims(a,-1) * w, axis=0)).eval()) 
    print((tf.reduce_sum(a * tf.transpose(w), axis=1)).eval()) 

Answer 2

tf.einsum給你做的正是你在簡潔，直觀的形式需要什麼樣的能力：

with tf.Session() as sess: 
    print(tf.einsum('n,nm->m', a, w).eval()) 
    # [ 2.6 3. ] 

你甚至可以寫你的意見明確n x (n, m) -> m。我認爲它更具可讀性和直觀性。

我最喜歡使用的情況是，當你想乘一批矩陣與權重向量：

n_in = 10 
n_step = 6 
input = tf.placeholder(dtype=tf.float32, shape=(None, n_step, n_in)) 
weights = tf.Variable(tf.truncated_normal((n_in, 1), stddev=1.0/np.sqrt(n_in))) 
Y_predict = tf.einsum('ijk,kl->ijl', input, weights) 
print(Y_predict.get_shape()) 
# (?, 6, 1) 

所以，你可以輕鬆地乘權在沒有轉換或重複的所有批次。這不能通過像其他答案一樣擴展維度來完成。所以你避免tf.matmul要求有匹配的批量尺寸和其他外部尺寸：

The inputs must, following any transpositions, be tensors of rank >= 2 where the inner 2 dimensions specify valid matrix multiplication arguments, and any further outer dimensions match.