2014-12-07 33 views
0

我有一個python webscrapping代碼,運行得非常好,如果我不插入任何結果在數據庫中。即當我註釋掉的代碼TypeError:並非在字符串格式化過程中轉換的所有參數

「」「 連接到數據庫,並把數據放入 ‘’」

db= MySQLdb.connect("localhost","XXX","XXX","hmm_Raw_Data") 
cursor=db.cursor() 


#checking phase to stop scrapping 


sql = """SELECT Short_link FROM RentalWanted WHERE Short_link=%s""" 

rows = cursor.execute(sql,(link_result)) 

if rows>=1: 
    duplicate_count+=1 
    print duplicate_count 

    # if duplicate_count>=15: 
    # print "The program has started getting duplicates now- The program is terminating" 
    # sys.exit() 
else: 
    query="""INSERT INTO RentalWanted 
    (Sale_Rent, 
    Type, 
    Area, 
    Nearby, 
    Title, 
    Price, 
    PricePerSqrFt, 
    Bedroom, 
    Agency_Fee, 
    Bathroom, 
    Size, 
    ZonedFor, 
    Freehold, 
    Prop_ref, 
    Furnished_status, 
    Rent_payment, 
    Building_info, 
    Amenities, 
    Trade_name, 
    Licence, 
    RERA_ID, 
    Phone_info, 
    Short_link) 
    values(
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s, 
    %s)""" 
    cursor.execute(query,(
    Sale_Rent_result, 
    Type_result, 
    area_result, 
    nearby_result, 
    title_result, 
    price_result, 
    Pricepersq_result, 
    bedroom_result, 
    agencyfee_result, 
    bathroom_result, 
    size_result, 
    Zoned_for_result, 
    Freehold_result, 
    propertyref_result, 
    furnished_result, 
    rent_is_paid_result, 
    building_result, 
    Amenities_result, 
    tradename_result, 
    licencenum_result, 
    reraid_result, 
    phone_result, 
    link_result)) 


db.commit() 
cursor.close() 
db.close() 

把上面的代碼時,我得到的錯誤,這部分是這樣的:

Traceback (most recent call last): File "RentalWanted.py", line 461, in <module> 
    getting_urls_of_all_pages() File "RentalWanted.py", line 45, in getting_urls_of_all_pages 
    every_property_in_a_page_data_extraction(a['href']) File "RentalWanted.py", line 365, in every_property_in_a_page_data_extraction 
    rows = cursor.execute(sql,(link_result)) File "/usr/lib/python2.6/site-packages/MySQL_python-1.2.5-py2.6-linux-x86_64.egg/MySQLdb/cursors.py", line 187, in execute 
    query = query % tuple([db.literal(item) for item in args]) TypeError: not all arguments converted during string formatting 

我認爲我正在做的查詢有問題。

任何人都可以幫我弄清楚哪一部分需要修復。我花了幾個小時,但不知道我在哪裏錯了

感謝

+0

有人嗎?幫助我PLZ。謝謝 – Newbie 2014-12-07 14:10:33

回答

0

你真的有23個獨立的變量?最好把所有的東西都放到一本字典裏,這樣它就更清楚了,什麼屬於一起,而且你不必非常重視。錯誤是,執行需要一個列表作爲最後一個參數,並且link_result可能是一個包含多個字符的字符串,例如與多個元素的列表:

result = { 
    "Sale_Rent": Sale_Rent_result, 
    "Type": Type_result, 
    "Area": area_result, 
    "Nearby": nearby_result, 
    "Title": title_result, 
    "Price": price_result, 
    "PricePerSqrFt": Pricepersq_result, 
    "Bedroom": bedroom_result, 
    "Agency_Fee": agencyfee_result, 
    "Bathroom": bathroom_result, 
    "Size": size_result, 
    "ZonedFor": Zoned_for_result, 
    "Freehold": Freehold_result, 
    "Prop_ref": propertyref_result, 
    "Furnished_status": furnished_result, 
    "Rent_payment": rent_is_paid_result, 
    "Building_info": building_result, 
    "Amenities": Amenities_result, 
    "Trade_name": tradename_result, 
    "Licence": licencenum_result, 
    "RERA_ID": reraid_result, 
    "Phone_info": phone_result, 
    "Short_link": link_result, 
} 

db= MySQLdb.connect("localhost","XXX","XXX","hmm_Raw_Data") 
cursor=db.cursor() 


#checking phase to stop scrapping 


sql = """SELECT Short_link FROM RentalWanted WHERE Short_link=%s""" 

rows = cursor.execute(sql,(result["Short_link"],)) 

if rows>=1: 
    duplicate_count+=1 
    print duplicate_count 

    # if duplicate_count>=15: 
    # print "The program has started getting duplicates now- The program is terminating" 
    # sys.exit() 
else: 
    query = """INSERT INTO RentalWanted ({fields}) VALUES ({values})""" 
    query = query.format(fields=','.join(result), values=','.join(['%s']*len(result))) 
    cursor.execute(query, result.values()) 

db.commit() 
cursor.close() 
db.close() 

,更好的使列Short_link獨特捕獲錯誤,如果你嘗試插入另一行具有相同的鏈接,而不是通過手工檢查約束:

db= MySQLdb.connect("localhost","XXX","XXX","hmm_Raw_Data") 
cursor=db.cursor() 
try: 
    query = """INSERT INTO RentalWanted ({fields}) VALUES ({values})""" 
    query = query.format(fields=','.join(result), values=','.join(['%s']*len(result))) 
    cursor.execute(query, result.values()) 
except mysql.connector.IntegrityError: 
    duplicate_count+=1 
    print duplicate_count 
else: 
    db.commit() 
cursor.close() 
db.close() 
+0

謝謝。有用。我在另一臺服務器上運行相同的腳本,它運行良好。是否因爲新服務器中的python版本過時?舊版本是Python Python 2.7.3,新的服務器是Python 2.6.6 – Newbie 2014-12-07 16:57:01

0

顯然在MySQL-python 1.2.5版本中存在向後兼容性問題,當你調用execute時,它期望一個元組而不是字符串。

試試這個:

rows = cursor.execute(sql,([link_result])) 
相關問題