缺少循環寫入文件

Question

import urllib.request 
import re 
import csv 
import pandas as pd 
from bs4 import BeautifulSoup 

stocklist = ['aapl','goog','fb','amzn','COP'] 
for stocklist in stocklist: 
    optionsUrl = urllib.request.urlopen('http://finance.yahoo.com/q?s='+stocklist).read() 
    soup = BeautifulSoup(optionsUrl) 
    stocksymbol = ['Symbol:',''+stocklist+''] 
    optionsTable = [stocksymbol]+[ 
     [x.text for x in y.parent.contents] 
     for y in soup.findAll('td', attrs={'class': 'yfnc_tabledata1','rtq_table': ''}) 
    ] 
    print(optionsTable) 
    my_df = pd.DataFrame(optionsTable).T 
    my_df.to_csv('test.csv', index=False, header=False) 

我有這段代碼。有人建議我使用熊貓。我能夠將列表中的數據寫入CSV文件。但是，CSV文件僅爲COP提供數據，而不是用於其他股票（csv文件只有一行數據，我假設它是覆蓋）。有人可以告訴我我失蹤或修復此代碼嗎？打印（optionsTable）打印4行，但..

這裏是輸出：

[['Symbol:', 'aapl'], ['Prev Close:', '99.65'], ['Open:', '98.51'], ['Bid:', '98.95 x 1700'], ['Ask:', '98.96 x 1200'], ['1y Target Est:', '124.90'], ['Beta:', '1.48679'], ['Earnings Date:', 'Jul 19 - Jul 25 (Est.)'], ["Day's Range:", '98.48 - 99.35'], ['52wk Range:', '89.47 - 132.97'], ['Volume:', '28,454,663'], ['Avg Vol (3m):', '38,261,900'], ['Market Cap:', '541.57B'], ['P/E (ttm):', '11.01'], ['EPS (ttm):', '8.98'], ['Div & Yield:', '2.28 (2.30%) '], ['Forward P/E (1 yr):', '10.86'], ['P/S (ttm):', '2.40'], ['Ex-Dividend Date:', '05-May-16'], ['Annual EPS Est\n      (Sep-16)\n     :', '8.28'], ['Quarterly EPS Est\n      (Jun-16)\n     :', '1.39'], ['Mean Recommendation*:', '1.8'], ['PEG Ratio (5 yr expected):', '1.30']] 
[['Symbol:', 'goog'], ['Prev Close:', '728.58'], ['Open:', '719.47'], ['Bid:', '717.60 x 400'], ['Ask:', '717.96 x 100'], ['1y Target Est:', '924.83'], ['Beta:', '1.032'], ['Next Earnings Date:', 'N/A'], ["Day's Range:", '716.43 - 725.86'], ['52wk Range:', '515.18 - 789.87'], ['Volume:', '1,050,710'], ['Avg Vol (3m):', '1,781,050'], ['Market Cap:', '493.43B'], ['P/E (ttm):', '29.25'], ['EPS (ttm):', '24.58'], ['Div & Yield:', 'N/A (N/A) '], ['Forward P/E (1 yr):', 'N/A'], ['P/S (ttm):', '6.41'], ['Ex-Dividend Date:', 'N/A'], ['Annual EPS Est\n      (Dec-16)\n     :', 'N/A'], ['Quarterly EPS Est\n      (Jun-16)\n     :', 'N/A'], ['Mean Recommendation*:', '1.8'], ['PEG Ratio (5 yr expected):', 'N/A']] 
[['Symbol:', 'fb'], ['Prev Close:', '118.56'], ['Open:', '117.52'], ['Bid:', '116.39 x 800'], ['Ask:', '116.40 x 500'], ['1y Target Est:', '142.87'], ['Beta:', '0.840485'], ['Earnings Date:', 'Jul 27 - Aug 1 (Est.)'], ["Day's Range:", '116.26 - 118.11'], ['52wk Range:', '72.00 - 121.08'], ['Volume:', '17,257,639'], ['Avg Vol (3m):', '25,746,700'], ['Market Cap:', '333.25B'], ['P/E (ttm):', '71.26'], ['EPS (ttm):', '1.64'], ['Div & Yield:', 'N/A (N/A) '], ['Forward P/E (1 yr):', '25.25'], ['P/S (ttm):', '17.16'], ['Ex-Dividend Date:', 'N/A'], ['Annual EPS Est\n      (Dec-16)\n     :', 'N/A'], ['Quarterly EPS Est\n      (Jun-16)\n     :', 'N/A'], ['Mean Recommendation*:', '1.7'], ['PEG Ratio (5 yr expected):', 'N/A']] 
[['Symbol:', 'amzn'], ['Prev Close:', '727.65'], ['Open:', '722.35'], ['Bid:', '716.25 x 500'], ['Ask:', '716.50 x 100'], ['1y Target Est:', '800.92'], ['Beta:', '1.6465'], ['Earnings Date:', 'Jul 21 - Jul 25 (Est.)'], ["Day's Range:", '714.21 - 724.98'], ['52wk Range:', '422.64 - 731.50'], ['Volume:', '3,161,899'], ['Avg Vol (3m):', '3,948,360'], ['Market Cap:', '338.47B'], ['P/E (ttm):', '295.70'], ['EPS (ttm):', '2.43'], ['Div & Yield:', 'N/A (N/A) '], ['Forward P/E (1 yr):', '72.29'], ['P/S (ttm):', '3.03'], ['Ex-Dividend Date:', 'N/A'], ['Annual EPS Est\n      (Dec-16)\n     :', '5.38'], ['Quarterly EPS Est\n      (Jun-16)\n     :', '1.10'], ['Mean Recommendation*:', '1.8'], ['PEG Ratio (5 yr expected):', '2.43']] 
[['Symbol:', 'COP'], ['Prev Close:', '46.57'], ['Open:', '45.90'], ['Bid:', '44.47 x 1300'], ['Ask:', '44.48 x 2300'], ['1y Target Est:', '51.23'], ['Beta:', '1.42252'], ['Earnings Date:', 'Jul 28 - Aug 1 (Est.)'], ["Day's Range:", '44.26 - 46.12'], ['52wk Range:', '31.05 - 64.13'], ['Volume:', '8,217,057'], ['Avg Vol (3m):', '8,947,330'], ['Market Cap:', '55.11B'], ['P/E (ttm):', 'N/A'], ['EPS (ttm):', '-4.98'], ['Div & Yield:', '1.98 (4.16%) '], ['Forward P/E (1 yr):', '143.48'], ['P/S (ttm):', '2.11'], ['Ex-Dividend Date:', '18-May-16'], ['Annual EPS Est\n      (Dec-16)\n     :', '-2.26'], ['Quarterly EPS Est\n      (Jun-16)\n     :', '-0.67'], ['Mean Recommendation*:', '2.5'], ['PEG Ratio (5 yr expected):', '0.37']] 

Answer 1

您可以附加到文件做

with open('test.csv', 'a') as f: 
    my_df.to_csv(f, header=False) 

這

Answer 2

你每次循環覆蓋您的csv。你應該收集的所有數據和循環後其寫入到CSV：

stocklist = ['aapl','goog','fb','amzn','COP'] 
columns = [] 
data = [] 
for s in stocklist: 
    optionsUrl = urllib.request.urlopen('http://finance.yahoo.com/q?s='+s).read() 
    soup = BeautifulSoup(optionsUrl, "html.parser") 
    stocksymbol = ['Symbol:', s] 
    optionsTable = [stocksymbol]+[ 
    [x.text for x in y.parent.contents] 
    for y in soup.findAll('td', attrs={'class': 'yfnc_tabledata1','rtq_table': ''}) 
    ] 

    if not columns: 
     columns = [o[0] for o in optionsTable] 
    data.append(o[1] for o in optionsTable) 

# create DataFrame from data 
df = pd.DataFrame(data, columns=columns) 
df.to_csv('test.csv', index=False) 

Answer 3

我會建議你使用pandas-datareader，這是專爲你準備做的事情。

這裏是一個小的演示：

from datetime import datetime 
import pandas_datareader.data as wb 

stocklist = ['AAPL','GOOG','FB','AMZN','COP'] 

start = datetime(2016,6,8) 
end = datetime(2016,6,11) 

p = wb.DataReader(stocklist, 'yahoo',start,end) 

p - 是熊貓panel，使我們可以做有趣的事情：

讓我們來看看我們有什麼在我們的面板

In [388]: p.axes 
Out[388]: 
[Index(['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close'], dtype='object'), 
DatetimeIndex(['2016-06-08', '2016-06-09', '2016-06-10'], dtype='datetime64[ns]', name='Date', freq='D'), 
Index(['AAPL', 'AMZN', 'COP', 'FB', 'GOOG'], dtype='object')] 

In [389]: p.keys() 
Out[389]: Index(['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close'], dtype='object') 

選擇數據

In [390]: p['Adj Close'] 
Out[390]: 
       AAPL  AMZN  COP   FB  GOOG 
Date 
2016-06-08 98.940002 726.640015 47.490002 118.389999 728.280029 
2016-06-09 99.650002 727.650024 46.570000 118.559998 728.580017 
2016-06-10 98.830002 717.909973 44.509998 116.620003 719.409973 

In [391]: p['Volume'] 
Out[391]: 
        AAPL  AMZN  COP   FB  GOOG 
Date 
2016-06-08 20812700.0 2200100.0 9596700.0 14368700.0 1582100.0 
2016-06-09 26419600.0 2163100.0 5389300.0 13823400.0 985900.0 
2016-06-10 31462100.0 3409500.0 8941200.0 18412700.0 1206000.0 

In [394]: p[:,:,'AAPL'] 
Out[394]: 
       Open  High  Low  Close  Volume Adj Close 
Date 
2016-06-08 99.019997 99.559998 98.680000 98.940002 20812700.0 98.940002 
2016-06-09 98.500000 99.989998 98.459999 99.650002 26419600.0 99.650002 
2016-06-10 98.529999 99.349998 98.480003 98.830002 31462100.0 98.830002 

In [395]: p[:,'2016-06-10'] 
Out[395]: 
      Open  High   Low  Close  Volume Adj Close 
AAPL 98.529999 99.349998 98.480003 98.830002 31462100.0 98.830002 
AMZN 722.349976 724.979980 714.210022 717.909973 3409500.0 717.909973 
COP 45.900002 46.119999 44.259998 44.509998 8941200.0 44.509998 
FB 117.540001 118.110001 116.260002 116.620003 18412700.0 116.620003 
GOOG 719.469971 725.890015 716.429993 719.409973 1206000.0 719.409973