tensorflow操作在哪裏運行？ CPU或GPU？

Question

看代碼：

import tensorflow as tf 

with tf.device('/gpu:0'): 
    with tf.device('/cpu:0'): 
     x = tf.constant(0,name='x') 
     x = x * 2 
    y = x + 2 
    config = tf.ConfigProto(log_device_placement=True) 
    with tf.Session(config=config) as sess: 
     sess.run(y) 

當你運行它，你會得到的結果。

mul: (Mul): /job:localhost/replica:0/task:0/cpu:0 
2017-08-11 21:38:23.953846: I c:\tf_jenkins\home\workspace\release-win\m\windows 
-gpu\py\35\tensorflow\core\common_runtime\simple_placer.cc:847] mul: (Mul)/job:l 
ocalhost/replica:0/task:0/cpu:0 
add: (Add): /job:localhost/replica:0/task:0/gpu:0 
2017-08-11 21:38:23.954846: I c:\tf_jenkins\home\workspace\release-win\m\windows 
-gpu\py\35\tensorflow\core\common_runtime\simple_placer.cc:847] add: (Add)/job:l 
ocalhost/replica:0/task:0/gpu:0 
add/y: (Const): /job:localhost/replica:0/task:0/gpu:0 
2017-08-11 21:38:23.954846: I c:\tf_jenkins\home\workspace\release-win\m\windows 
-gpu\py\35\tensorflow\core\common_runtime\simple_placer.cc:847] add/y: (Const)/j 
ob:localhost/replica:0/task:0/gpu:0 
mul/y: (Const): /job:localhost/replica:0/task:0/cpu:0 
2017-08-11 21:38:23.954846: I c:\tf_jenkins\home\workspace\release-win\m\windows 
-gpu\py\35\tensorflow\core\common_runtime\simple_placer.cc:847] mul/y: (Const)/j 
ob:localhost/replica:0/task:0/cpu:0 
x: (Const): /job:localhost/replica:0/task:0/cpu:0 
2017-08-11 21:38:23.954846: I c:\tf_jenkins\home\workspace\release-win\m\windows 
-gpu\py\35\tensorflow\core\common_runtime\simple_placer.cc:847] x: (Const)/job:l 
ocalhost/replica:0/task:0/cpu:0 

這意味着cpu和add上gpu運行的mul運行。所以我得出結論，where does ops or tensors define where does ops or tensors run。

而當我查看Inception時，我感到困惑。

with slim.arg_scope([slim.variables.variable], device='/cpu:0'): 
    # Calculate the loss for one tower of the ImageNet model. This 
    # function constructs the entire ImageNet model but shares the 
    # variables across all towers. 
    loss = _tower_loss(images_splits[i], labels_splits[i], num_classes, 
         scope, reuse_variables) 

的tower_loss定義上的CPU，這意味着每個GPU將根據我的結論運行在CPU損失。但我認爲每個GPU應該在gpu上運行副本 。我誤解了嗎？

Answer 1

父設備分配被子設備分配覆蓋。

在以下代碼中，函數_tower_loss在/gpu:i（如果您查看實現）中有另一個設備分配。損失正在使用GPU的計算，但損失聚集並在cpu.

loss = _tower_loss(images_splits[i], labels_splits[i], num_classes, 
        scope, reuse_variables)