星火斯卡拉如何使用替換功能在RDD

我有一個鳴叫文件星火斯卡拉如何使用替換功能在RDD

396124436845178880,"When's 12.4k gonna roll around",Matty_T_03 
396124437168537600,"I really wish I didn't give up everything I did for you.  I'm so mad at my self for even letting it get as far as it did.",savava143 
396124436958412800,"I really need to double check who I'm sending my  snapchats to before sending it ",juliannpham 
396124437218885632,"@Darrin_myers30 I feel you man, gotta stay prayed up.  Year is important",Ful_of_Ambition 
396124437558611968,"tell me what I did in my life to deserve this.",_ItsNotBragging 
396124437499502592,"Too many fine men out here...see me drooling",LolaofLife 
396124437722198016,"@jaiclynclausen will do",I_harley99

我想讀文件到RDD後，以取代所有特殊字符，

val fileReadRdd = sc.textFile(fileInput) 
    val fileReadRdd2 = fileReadRdd.map(x => x.map(_.replace(","," "))) 
    val fileFlat = fileReadRdd.flatMap(rec => rec.split(" "))

我得到以下錯誤

Error:(41, 57) value replace is not a member of Char 
    val fileReadRdd2 = fileReadRdd.map(x => x.map(_.replace(",","")))

來源

2017-03-20 Ravinder Karra

我懷疑：

x => x.map(_.replace(",",""))

是治療你的字符串的字符序列，而你真正想要

x => x.replace(",", "")

（即你不需要映射字符的'序列'）

來源

2017-03-20 16:37:39

謝謝布萊恩。 val stripCurly =「[{〜，！，@，＃，$，％，^，＆，*，（，），_，=， - ，'，：，'，？，/，<,>，。}] 「 val fileReadRdd2 = fileReadRdd.map（x => stripCurly.replaceAll（x，」「）） –

星火斯卡拉如何使用替換功能在RDD

回答

相關問題