從數據框中刪除停用詞

Question

dataframe['Text'] = dataframe['Text'].apply(lambda x : ' '.join([item for item in string.split(x.lower()) if item not in stopwords])) 

我正在從數據框中刪除停用詞。邏輯工作正常，但是當有一些空行時，它會給出錯誤。

我已經使用了dropna（），但它會刪除整行，而不是其他列中有數據。

如何在上述邏輯列的文字要你的邏輯之前不爲空

Answer 1

使用，

dataframe.dropna(subset=['Text'], how='all') 

Answer 2

可以更換NaN空list什麼是不容易的添加條件 - 通過使用mask或combine_firstSeries由空創建lists：

pos_tweets = [('I love this car', 'positive'), 
('This view is amazing', 'positive'), 
('I feel great this morning', 'positive'), 
('I am so excited about the concert', 'positive'), 
(None, 'positive')] 

df = pd.DataFrame(pos_tweets, columns= ["Text","col2"]) 
print (df) 
           Text  col2 
0     I love this car positive 
1    This view is amazing positive 
2   I feel great this morning positive 
3 I am so excited about the concert positive 
4        None positive 

stopwords = ['love','car','amazing'] 
s = pd.Series([[]], index=df.index) 
df["Text"] = df["Text"].str.lower().str.split().mask(df["Text"].isnull(), s) 
print (df) 
             Text  col2 
0      [i, love, this, car] positive 
1     [this, view, is, amazing] positive 
2   [i, feel, great, this, morning] positive 
3 [i, am, so, excited, about, the, concert] positive 
4           [] positive 

df['Text']=df['Text'].apply(lambda x:' '.join([item for item in x if item not in stopwords])) 
print (df) 
           Text  col2 
0        i this positive 
1      this view is positive 
2   i feel great this morning positive 
3 i am so excited about the concert positive 
4          positive 

另一種解決方案：

stopwords = ['love','car','amazing'] 
df["Text"]=df["Text"].str.lower().str.split().combine_first(pd.Series([[]], index=df.index)) 
print (df) 
             Text  col2 
0      [i, love, this, car] positive 
1     [this, view, is, amazing] positive 
2   [i, feel, great, this, morning] positive 
3 [i, am, so, excited, about, the, concert] positive 
4           [] positive