主亞馬遜網站搜索術語（解析）

Question

嗨所以亞馬遜主搜索條有下列信息

<input type="submit" class="nav-input" value="Go" tabindex="7"> 

，我想建立一個功能，將看amazon.co.uk 找到這個標籤，然後搜索任何關鍵詞，我給

要做到這一點，我已經做到了這一點：

path = 'https://www.amazon.co.uk/' 

values = {'s': 'what-I-want-to-search', 
      'submit':'search'} 
data = urllib.parse.urlencode(values) 
data = data.encode('utf-8') 


driver = webdriver.PhantomJS() 
driver.get(path) 
html = driver.page_source 

driver = webdriver.PhantomJS() 
driver.get(path, data) 

html = driver.page_source 

，其中的麻煩全部的想法是，繼sentdex政黨成員rial，我編碼搜索條件，然後將它們發送到html路徑，並且我一直使用硒來對抗動態加載的網頁，但在這種情況下，我認爲它應該沒問題，但無論如何，我需要知道如何讓python搜索主網站上的內容，並將其帶到搜索結果頁面，有幫助嗎？

因緣會使用請求和BS4

Answer 1

出現在你面前的朋友，你只需要通過正確的PARAMS，你可以看到，如果你在Chrome瀏覽器開發工具，看看網絡標籤：

In [4]: from bs4 import BeautifulSoup  
In [5]: import requests  
In [6]: params = {"url": "search-alias=", 
    ...:   "field-keywords": "python"} 

In [7]: with requests.Session() as s: 
    ...:   url = "https://www.amazon.co.uk/s" 
    ...:   r = s.get(url, params=params) 
    ...:   soup = BeautifulSoup(r.content,"lxml") 
    ...:   for a in cont: 
    ...:    print(a.select_one("a")["title"]) 
    ...:   
Python Programming for the Absolute Beginner 
Python: The Ultimate Beginner's Guide! 
Automate the Boring Stuff with Python: Practical Programming for Total Beginners 
Python: Learn Python in One Day and Learn It Well. Python for Beginners with Hands-on Project. (Learn Coding Fast with Hands-On Project Book 1) 
Python Crash Course: A Hands-On, Project-Based Introduction to Programming 
Learning Python 
Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython 
Python Cookbook 
Python for Informatics: Exploring Information 
Fluent Python 
Python Playground: Geeky Projects for the Curious Programmer 
Python in easy steps 
Learn Python the Hard Way: A Very Simple Introduction to the Terrifyingly Beautiful World of Computers and Code (Zed Shaw's Hard Way) 
Python: The Ultimate Beginners Guide: Start Coding Today 
Programming the Raspberry Pi, Second Edition: Getting Started with Python 
Data Science from Scratch: First Principles with Python 

打破代碼到功能，並獲得所有T他的網頁，我們只需要不斷循環，直到與ID pagnNextLink錨是不可見的：

from bs4 import BeautifulSoup 
import requests 
from urlparse import urljoin 
# from urllib.parse import urljoin -> python 3 


def parse(soup): 
    yield [a["title"] for a in soup.select("a.a-link-normal.s-access-detail-page.a-text-normal")] 

def get(term): 
    params = {"url": "search-alias=", 
       "field-keywords": term} 

    with requests.Session() as s: 
     head = {"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36"} 
     url = "https://www.amazon.co.uk/s" 
     r = s.get(url, params=params) 
     soup = BeautifulSoup(r.content, "lxml") 
     nxt = soup.select_one("#pagnNextLink") 
     while nxt: 
      cont = requests.get(urljoin("https://www.amazon.co.uk/", nxt["href"]), headers=head) 
      soup = BeautifulSoup(cont.content,"lxml") 
      for t in parse(soup): 
       print(t) 
      nxt = soup.select_one("#pagnNextLink") 

如果我們跑了幾個迭代代碼：

In [5]: get("python") 
['Python Machine Learning', 'Effective Python: 59 Specific Ways to Write Better Python (Effective Software Development)', 'Black Hat Python: Python Programming for Hackers and Pentesters', 'Doing Math with Python: Use Programming to Explore Algebra, Statistics, Calculus, and More!', 'Think Python: How to Think Like a Computer Scientist', 'Python Basics, Level 1 (Coding Club) (Coding Club, Level 1)', 'Python for Finance: Analyze Big Financial Data', 'Violent Python: A Cookbook for Hackers, Forensic Analysts, Penetration Testers and Security Engineers', "Python Essential Reference (Developer's Library)", 'Learn Web Scraping With Python In A Day: The Ultimate Crash Course to Learning the Basics of Web Scraping With Python In No Time (Python, Python ... Python Books, Python for Beginners)', 'Programming Python', 'QPython - Python on Android', 'Coding Club Python: Next Steps Level 2', "Python: Programming, Master's Handbook; A TRUE Beginner's Guide! Problem Solving, Code, Data Science, Data Structures & Algorithms (Code like a PRO ... engineering, r programming, iOS development)", 'Python: Complete Crash Course for Becoming an Expert in Python Programming', 'Coding Club Python: Building Big Apps Level 3'] 
['High Performance Python: Practical Performant Programming for Humans', '25ft Python No Spill Clean And Fill', 'Learning Python with Raspberry Pi', 'Web Scraping with Python: Collecting Data from the Modern Web', 'Invent Your Own Computer Games with Python, 3rd Edition', 'More Python Programming for the Absolute Beginner', 'Python for Kids: A Playful Introduction to Programming', "Monty Python's Life of Brian", 'Python 3 Object-oriented Programming - Second Edition', 'Introduction to Computation and Programming Using Python', 'Evolution of The Silly Walks T Shirt - Funny TV Ministry - Various Colours and Sizes XS - 3XL', "Hacking Secret Ciphers with Python: A beginner's guide to cryptography and computer programming with Python", 'Monty Python Fluxx', 'MASTER LOCK 8417DPRO Python Cable 1.80 m x 5 mm 2 Keys', "Learn Python: A beginner's guide book to programming python, learning the basics and start coding easily", 'Master Lock Python Disc Cylinder Key Adjustable Braided Steel Cable Lock, 10 x 1800 mm - Black'] 

In [6]: get("c programming") 
['C Programming', 'C# 6.0 in a Nutshell: The Definitive Reference', 'PIC microcontrollers Programming in C with examples', 'C++: The Ultimate Crash Course to Learning the Basics of C++ In No Time (c plus plus, C++ for beginners, programming computer, how to program) (HTML, Javascript, ... Java, C++ Course, C++ Development Book 3)', 'Java: The Best Guide to Master Java Programming Fast (Java for Beginners, Java for Dummies, how to program, java app, java programming): Volume 2 (C Programming, HTML, Javascript)', 'A Book on C.: Programming in C.', "Learn C the Hard Way: Practical Exercises on the Computational Subjects You Keep Avoiding (Like C) (Zed Shaw's Hard Way Series)", 'C++: C++ and Hacking for dummies. A smart way to learn C plus plus and beginners guide to computer hacking: Volume 10 (C Programming, HTML, Javascript, Programming, Coding, CSS, Java, PHP)', 'Introduction to Algorithms', 'Programming: Computer Programming for Beginners: Learn the Basics of Java, SQL & C++ - 2. Edition (Coding, C Programming, Java Programming, SQL Programming, JavaScript, Python, PHP)', '21st Century C: C Tips from the New School', 'C For Dummies', 'Learn C# Programming Training DVD - Tutorial Video', 'GT01-C30R2-6P Programming PLC Cable 2.5M for Mitsubishi Melsec A970', 'Programming In C', 'Get Coding!: Learn HTML, CSS & JavaScript & build a website, app & game'] 
['Hewlett Packard [HP] Calculator Financial Platinum RPN Algebraic Programmable Ref HP12C PLATINUM', 'C: Easy C Programming for Beginners, Your Step-By-Step Guide To Learning C Programming (C Programming Series)', '4.9M RS232 DB9 F/M PLC Programming Cable Adapter White for Omron CQM1 C200HE HG', 'KOREAN COSMETICS, LG Household & Health Care_ SUM37, Secret Programming Eye C...', 'C++: C++ and Python. C++ for Beginners and Python for Dummies to Learn Fast (C Programming, Programming for beginners, c plus plus, programming ... Developers, Coding, CSS, Java, PHP)', '1:8 Brushless Combo BLC-150C Plus + Ripper 2000KV motor + programming Board', 'Lonely Planet Italian Phrasebook & Audio', 'Full Forgiveness - Let Go of Hurt & Offense With Guided Imagery, Self Hypnosis and Neuro-linguistic Programming (NLP)', 'Accelerated C++: Practical Programming by Example (C++ in Depth Series)', 'Gardena Water Computer C1060plus 1864-20', 'Learning To Build Apps For iPhone and iPad - Training DVD', 'Practical C Programming (A Nutshell handbook)', 'Prince Brat and the Whipping Boy', 'English: Practice Test Papers (Letts Key Stage 2 Success) (Letts Key Stage 1 Success)', 'Arabic For Dummies: Audio Set', 'The Actor and the Text (Applause Acting Series)'] 

你可以做任何你喜歡解析，我只是拉動標題，讓我們很容易看到我們正在獲取正確的數據。我還會考慮在請求之間添加睡眠。