保存數據到sqlite從tweepy

Question

我有這個代碼保存數據在cvs文件現在我想保存該數據到sqlite或mysql，但它應該保存像它保存在cvs數據庫中的一個一個記錄 和一個問題 這個數據量將每月 我的想法是做一個網站，該網站有趨勢的所有統計每天30-40MB的數據庫我應該在我有1GB使用MySQL或PostgreSQL的事業+數據庫，我想使用在Django

import tweepy 
import csv #Import csv 
auth = tweepy.auth.OAuthHandler('XXXXXX', 'XXXXXXX') 
auth.set_access_token('XXX-XXX', 'XXX') 

api = tweepy.API(auth) 
# Open/Create a file to append data 
csvFile = open('result.csv', 'a') 
#Use csv Writer 
csvWriter = csv.writer(csvFile) 

for tweet in tweepy.Cursor(api.search, 
        q="google", 
        since="2016-05-22", 
        until="2016-05-23", 
        lang="en").items(): 
    #Write a row to the csv file/ I use encode utf-8 
    csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8')]) 
    print tweet.created_at, tweet.text 
csvFile.close() 

Answer 1

SQLITE3解決方案：

首先，創建一個數據庫，在這個例子中我使用了twitter2.db。基本連接到SQLITE3和創建表：

import sqlite3 

conn = sqlite3.connect('C:\\....\\twitter2.db') 
c = conn.cursor() 

c.execute("CREATE TABLE tweeter(text VARCHAR, created_at DATETIME") 

做的工作：

import tweepy 
import sqlite3 


conn = sqlite3.connect('C:\\....\\Desktop\\twitter2.db') 
x = conn.cursor() 
consumer_key="..." 
consumer_secret=".." 
access_token=".." 
access_token_secret=".." 

auth = tweepy.OAuthHandler(consumer_key, consumer_secret) 
auth.set_access_token(access_token, access_token_secret) 
api = tweepy.API(auth) 

class CustomStreamListener(tweepy.StreamListener): 
    def __init__(self, api): 
     self.api = api 
     super(tweepy.StreamListener, self).__init__() 


    def on_status(self, status): 
      x.execute("""INSERT INTO tweeter (text,created_at) VALUES(?,?)""", 
      (status.text, status.created_at)) 
      conn.commit() 

    def on_error(self, status_code): 
     print >> sys.stderr, 'Encountered error with status code:', status_code 
     return True 

    def on_timeout(self): 
     print >> sys.stderr, 'Timeout...' 
     return True 


sapi = tweepy.streaming.Stream(auth, CustomStreamListener(api)) 
sapi.filter(track=['google']) 

關閉連接：

conn.close()