安排每列一個項目的CSV文件的行scrapy蟒蛇

Question

我不得不從一個站點刮項目，我把他們安置在以JSON文件，如下面

{ 
"author": ["TIM ROCK"], 
"book_name": ["Truk Lagoon, Pohnpei & Kosrae Dive Guide"], 
"category": "Travel", 
} 
{ 
"author": ["JOY"], 
"book_name": ["PARSER"], 
"category": "Accomp", 
} 

我想將它們存儲在CSV一個字典文件每一行在每一個項目一個列如下

| author |  book_name  | category | 
| TIM ROCK | Truk Lagoon ... |  Travel | 
|  JOY  | PARSER   |  Accomp | 

我得到一排一個字典的項目，但與所有的列組合

我pipeline.py代碼

導入CSV

class Blurb2Pipeline(object): 

    def __init__(self): 
     self.brandCategoryCsv = csv.writer(open('blurb.csv', 'wb')) 
     self.brandCategoryCsv.writerow(['book_name', 'author','category']) 

    def process_item(self, item, spider): 
     self.brandCategoryCsv.writerow([item['book_name'].encode('utf-8'), 
            item['author'].encode('utf-8'), 
            item['category'].encode('utf-8'), 
            ]) 
     return item   

Answer 1

的要點是，這是非常簡單，csv.DictWriter：

>>> inputs = [{ 
... "author": ["TIM ROCK"], 
... "book_name": ["Truk Lagoon, Pohnpei & Kosrae Dive Guide"], 
... "category": "Travel", 
... }, 
... { 
... "author": ["JOY"], 
... "book_name": ["PARSER"], 
... "category": "Accomp", 
... } 
... ] 
>>> 
>>> from csv import DictWriter 
>>> from cStringIO import StringIO 
>>> 
>>> buf=StringIO() 
>>> c=DictWriter(buf, fieldnames=['author', 'book_name', 'category']) 
>>> c.writeheader() 
>>> c.writerows(inputs) 
>>> print buf.getvalue() 
author,book_name,category 
['TIM ROCK'],"['Truk Lagoon, Pohnpei & Kosrae Dive Guide']",Travel 
['JOY'],['PARSER'],Accomp 

倒不如加入的東西的數組，但是因爲元素可以是列表或字符串，這有點棘手。告訴某些東西是字符串還是其他可迭代的東西是Python中爲數不多的直接類型檢查很有意義的情況之一。

>>> for row in inputs: 
...  for k, v in row.iteritems(): 
...   if not isinstance(v, basestring): 
...    try: 
...     row[k] = ', '.join(v) 
...    except TypeError: 
...     pass 
...  c.writerow(row) 
... 
>>> print buf.getvalue() 
author,book_name,category 
TIM ROCK,"Truk Lagoon, Pohnpei & Kosrae Dive Guide",Travel 
JOY,PARSER,Accomp