Python音頻信號分類MFCC功能神經網絡

Question

我想分類從語音到情緒的音頻信號。爲此，我提取音頻信號的MFCC特徵，並將它們饋入一個簡單的神經網絡（用PyBrain的BackpropTrainer訓練的FeedForwardNetwork）。不幸的是結果非常糟糕。從5個班級看來，網絡似乎總是會有相同的班級。

我有5類情感和7000左右標記的音頻文件，我劃分，使每個類的80％被用來訓練網絡和20％來測試網絡。

想法是使用小窗口並從這些窗口中提取MFCC特徵以生成大量訓練示例。在評估中，評估一個文件中的所有窗口，並通過大多數投票決定預測標籤。

Training examples per class: 
{0: 81310, 1: 60809, 2: 58262, 3: 105907, 4: 73182} 

Example of scaled MFCC features: 
[ -6.03465056e-01 8.28665733e-01 -7.25728303e-01 2.88611116e-05 
1.18677218e-02 -1.65316583e-01 5.67322809e-01 -4.92335095e-01 
3.29816126e-01 -2.52946780e-01 -2.26147779e-01 5.27210979e-01 
-7.36851560e-01] 

Layers________________________: 13 20 5 (also tried 13 50 5 and 13 100 5) 
Learning Rate_________________: 0.01 (also tried 0.1 and 0.3) 
Training epochs_______________: 10 (error rate does not improve at all during training) 

Truth table on test set: 
[[ 0. 4. 0. 239. 99.] 
[ 0. 41. 0. 157. 23.] 
[ 0. 18. 0. 173. 18.] 
[ 0. 12. 0. 299. 59.] 
[ 0. 0. 0. 85. 132.]] 

Success rate overall [%]: 34.7314201619 
Success rate Class 0 [%]: 0.0 
Success rate Class 1 [%]: 18.5520361991 
Success rate Class 2 [%]: 0.0 
Success rate Class 3 [%]: 80.8108108108 
Success rate Class 4 [%]: 60.8294930876 

好吧，現在，你可以看到，結果在類上的分佈非常糟糕。 0級和2級從未預測過。我假設，這暗示了我的網絡或更可能是我的數據的問題。

我可以在這裏發佈大量的代碼，但我認爲在下面的圖像中顯示所有我正在使用的MFCC功能的步驟更有意義。請注意，我使用整個信號而沒有窗口只是爲了說明。這看起來好嗎？ MFCC的價值非常巨大，它們不應該小得多嗎？ （I比例下來在所有數據爲[-2,2]，也嘗試[0,1]它們與minmaxscaler送入網絡之前）

這是我使用的代碼這是我一個離散餘弦變換之前直接套用Melfilter銀行提取MFCC特徵（我從這裏得到它：stackoverflow）：

def freqToMel(freq): 
    ''' 
    Calculate the Mel frequency for a given frequency 
    ''' 
    return 1127.01048 * math.log(1 + freq/700.0) 

def melToFreq(mel): 
    ''' 
    Calculate the frequency for a given Mel frequency 
    ''' 
    return 700 * (math.exp(freq/1127.01048 - 1)) 

def melFilterBank(blockSize): 
    numBands = int(mfccFeatures) 
    maxMel = int(freqToMel(maxHz)) 
    minMel = int(freqToMel(minHz)) 

    # Create a matrix for triangular filters, one row per filter 
    filterMatrix = numpy.zeros((numBands, blockSize)) 

    melRange = numpy.array(xrange(numBands + 2)) 

    melCenterFilters = melRange * (maxMel - minMel)/(numBands + 1) + minMel 

    # each array index represent the center of each triangular filter 
    aux = numpy.log(1 + 1000.0/700.0)/1000.0 
    aux = (numpy.exp(melCenterFilters * aux) - 1)/22050 
    aux = 0.5 + 700 * blockSize * aux 
    aux = numpy.floor(aux) # Arredonda pra baixo 
    centerIndex = numpy.array(aux, int) # Get int values 

    for i in xrange(numBands): 
    start, centre, end = centerIndex[i:i + 3] 
    k1 = numpy.float32(centre - start) 
    k2 = numpy.float32(end - centre) 
    up = (numpy.array(xrange(start, centre)) - start)/k1 
    down = (end - numpy.array(xrange(centre, end)))/k2 

    filterMatrix[i][start:centre] = up 
    filterMatrix[i][centre:end] = down 

    return filterMatrix.transpose() 

我能做些什麼，以獲得更好的預測結果？

Answer 1

在這裏，我構思了一個從言語中識別性別的例子。本例中我使用了Hyke-dataset 。這只是一個快速的例子。如果想要進行認真的性別鑑定，可能會做得更好。但在一般的錯誤率降低：

Build up data... 
Train network... 
Number of training patterns: 94956 
Number of test patterns:  31651 
Input and output dimensions: 13 2 
Train network... 
epoch: 0 train error: 62.24% test error: 61.84% 
epoch: 1 train error: 34.11% test error: 34.25% 
epoch: 2 train error: 31.11% test error: 31.20% 
epoch: 3 train error: 30.34% test error: 30.22% 
epoch: 4 train error: 30.76% test error: 30.75% 
epoch: 5 train error: 30.65% test error: 30.72% 
epoch: 6 train error: 30.81% test error: 30.79% 
epoch: 7 train error: 29.38% test error: 29.45% 
epoch: 8 train error: 31.92% test error: 31.92% 
epoch: 9 train error: 29.14% test error: 29.23% 

我從scikits.talkbox使用MFCC實行。也許下面的代碼可以幫助你。 （性別鑑定肯定比情感檢測一個更容易的任務......也許你需要更多的和不同的功能。）

import glob 

from scipy.io.wavfile import read as wavread 
from scikits.talkbox.features import mfcc 

from pybrain.datasets   import ClassificationDataSet 
from pybrain.utilities   import percentError 
from pybrain.tools.shortcuts  import buildNetwork 
from pybrain.supervised.trainers import BackpropTrainer 
from pybrain.structure.modules import SoftmaxLayer 

def report_error(trainer, trndata, tstdata): 
    trnresult = percentError(trainer.testOnClassData(), trndata['class']) 
    tstresult = percentError(trainer.testOnClassData(dataset=tstdata), tstdata['class']) 
    print "epoch: %4d" % trainer.totalepochs, " train error: %5.2f%%" % trnresult, " test error: %5.2f%%" % tstresult 

def main(auido_path, coeffs=13): 
    dataset = ClassificationDataSet(coeffs, 1, nb_classes=2, class_labels=['male', 'female']) 
    male_files = glob.glob("%s/male_audio/*/*_1.wav" % auido_path) 
    female_files = glob.glob("%s/female_audio/*/*_1.wav" % auido_path) 

    print "Build up data..." 
    for sex, files in enumerate([male_files, female_files]): 
     for f in files: 
      sr, signal = wavread(f) 
      ceps, mspec, spec = mfcc(signal, nwin=2048, nfft=2048, fs=sr, nceps=coeffs) 
      for i in range(ceps.shape[0]): 
       dataset.appendLinked(ceps[i], [sex]) 

    tstdata, trndata = dataset.splitWithProportion(0.25) 
    trndata._convertToOneOfMany() 
    tstdata._convertToOneOfMany() 

    print "Number of training patterns: ", len(trndata) 
    print "Number of test patterns:  ", len(tstdata) 
    print "Input and output dimensions: ", trndata.indim, trndata.outdim 

    print "Train network..." 
    fnn = buildNetwork(coeffs, int(coeffs*1.5), 2, outclass=SoftmaxLayer, fast=True) 
    trainer = BackpropTrainer(fnn, dataset=trndata, learningrate=0.005) 

    report_error(trainer, trndata, tstdata) 
    for i in range(100): 
     trainer.trainEpochs(1) 
     report_error(trainer, trndata, tstdata) 

if __name__ == '__main__': 
    main("/path/to/hyke/audio_data") 

---

阿扎瑞斯Reda的，SAURABH Panjwani和愛德華Cutrell： Hyke：一種低 - 用於發展中地區的遠程考勤跟蹤系統，第五屆ACM開發區網絡化系統研討會（NSDR）。