1
我正在嘗試使用CNN進行文本分類。下面是我的一些標籤：dog，cat，bird，football，basketball ......由於這些類是莫名其妙太細顆粒具有良好的精度，加上訓練數據相對較少，我將其整合到animal，sports。多任務學習網絡結構設計
然後，我設計了一個簡單的多任務學習結構，如下所示，但它並沒有改善我的細粒度標籤上的最終性能。
18 data = mx.symbol.Variable('data')
19 softmax_label = mx.symbol.Variable('softmax_label')
20 softmax_label_finegrained = mx.symbol.Variable('softmax_label_finegrained')
21
22 # embedding layer
23 if not with_embedding:
24 word_embed = mx.symbol.Embedding(data=data, input_dim=vocab_size,
25 output_dim=embedding_size, name='word_embedding')
26 conv_input = mx.symbol.Reshape(data=word_embed, target_shape=(batch_size, 1, sentence_size, embedding_size)) # convolution layer needs 4D input.
27 else:
28 logging.info('with pretrained embedding.')
29 conv_input = data
30
31 # convolution and pooling layer
32 pooled_outputs = []
33 for i, filter_size in enumerate(filter_list):
34 convi = mx.symbol.Convolution(data=conv_input, kernel=(filter_size, embedding_size), num_filter=num_filter)
35 acti = mx.symbol.Activation(data=convi, act_type='relu')
36 pooli = mx.symbol.Pooling(data=acti, pool_type='max', kernel=(sentence_size - filter_size + 1, 1), stride=(1,1)) # max pooling on entire sentence feature ma p.
37 pooled_outputs.append(pooli)
38
39 # combine all pooled outputs
40 num_feature_maps = num_filter * len(filter_list)
41 concat = mx.symbol.Concat(*pooled_outputs, dim=1) # max-overtime pooling. concat all feature maps into a long feature before feeding into final dropout and full y connected layer.
42 h_pool = mx.symbol.Reshape(data=concat, shape=(batch_size, num_feature_maps)) # make it flat/horizontal
43
44 # dropout
45 if dropout > 0.0:
46 logging.info('use dropout.')
47 drop = mx.symbol.Dropout(data=h_pool, p=dropout)
48 else:
49 logging.info('Do not use dropout.')
50 drop = h_pool
51
52 # fully connected and softmax output.
53 logging.info('num_classes: %d', num_classes)
54 logging.info('num_fine_classes: %d', num_fine_classes)
55 fc = mx.symbol.FullyConnected(data=drop, num_hidden= num_classes, name='fc')
56 fc_fine = mx.symbol.FullyConnected(data=drop, num_hidden= num_fine_classes, name='fc_fine')
57 softmax = mx.symbol.SoftmaxOutput(data= fc, label= softmax_label)
58 softmax_fine = mx.symbol.SoftmaxOutput(data= fc_fine, label= softmax_label_finegrained)
59
60 return mx.symbol.Group([softmax, softmax_fine])
我還試圖通過增加內部SoftmaxActivation層將更多的信息fc後沒有工作：
52 # fully connected and softmax output.
53 logging.info('num_classes: %d', num_classes)
54 logging.info('num_fine_classes: %d', num_fine_classes)
55 fc = mx.symbol.FullyConnected(data=drop, num_hidden= num_classes, name='fc')
56 softmax = mx.symbol.SoftmaxOutput(data=fc, label= softmax_label)
57 softmax_act = mx.symbol.SoftmaxActivation(data=fc)
58 # make softmax_domain a internal layer for emitting activation, which we take it as a input into downstream task.
59 drop_act = mx.symbol.Concat(drop, softmax_act, dim=1)
60 fc_fine = mx.symbol.FullyConnected(data=drop_act, num_hidden= num_fine_classes, name='fc_fine')
61 softmax_fine = mx.symbol.SoftmaxOutput(data=fc_fine, label= softmax_label_finegrained)
62
63 return mx.symbol.Group([softmax, softmax_fine])
64
所以請你們在設計這樣的網絡中的任何想法或經驗？任何想法都歡迎，謝謝〜