使用BeautifulSoup刮掉whoscall.in上的問題4

Question

我的python腳本使用BeautifulSoup，似乎無法把握頁面上div的單詞，是否有特定的原因呢？我可以抓取個人資料圖片來計算消息的數量，但不是文本本身。

（僅供參考，我已經使用這個頁面：http://whoscall.in/1/2392247496/）

if(website == "1"): 
    reqInput = "http://whoscall.in/1/%s/" % (teleWho) 
    urlfile = urllib2.Request(reqInput) 
    print (reqInput) 
    time.sleep(1) 
    requestRec = requests.get(reqInput) 
    soup = BeautifulSoup(requestRec.content, "lxml") 
    noMatch = soup.find(text=re.compile(r"no reports yet on the phone number")) 
    print(requestRec.content)# #only if needed# 
    type(noMatch) is str 
    if noMatch is None: 
    worksheet.write(idx+1, 2, "Got a hit") 
    howMany = soup.find_all('img',{'src':'/default-avatar.gif'}) 
    howManyAreThere = len(howMany) 
    worksheet.write(idx+1,1,howManyAreThere) 
    print (howManyAreThere) 
    scamNum = soup.find_all(text=("scam"),recursive=True) 
    #,'scam','Scammer','scammer'# 
    scamCount = len(scamNum) 
    print(scamNum) 
    searchTerms = {scamCount:scamCount} 
    sentiment = max(searchTerms, key=searchTerms.get) 
    worksheet.write(idx+1,3,sentiment) 

我似乎無法拉文本「騙局」關閉頁面

我不確定它爲什麼拒絕找到那個文本，就像其他美麗的湯碼一樣完美。

https://github.com/GarnetSunset/Haircuttery/

Answer 1

改變這一行：

scamNum = soup.find_all(text=("scam"),recursive=True) 

到：

scamNum = [ div.text for div in soup.find_all('div', {'style':'font-size:14px; margin:10px; overflow:hidden'}) if 'scam' in div.text.lower() ] 

試試這個多個單詞：

words = [ 'word1', 'word2', ... ] 
scamNum = [ div.text for div in soup.find_all('div', {'style':'font-size:14px; margin:10px; overflow:hidden'}) if any(word for word in words if word in div.text.lower()) ]