在預先存在的sqllite表中防止與sqlalchemy重複的條目

Question

我有一個預先存在的sqllite表，我正在使用sqlalchemy訪問。我意識到存在許多重複的「案例」數字。如果我理解正確的話，似乎你不能表的創建與sqllite後唯一約束添加到表中，除去使用複本後：

DELETE FROM mytable 
WHERE id NOT IN 
(
SELECT MIN(id) 
FROM judgements 
GROUP BY "case" 

我決定使用SQLAlchemy防止添加額外的DUP。我與scrapy工作，並有管線元件，看起來像：

class DynamicSQLlitePipeline(object): 

    def __init__(self,table_name): 
     db_path = "sqlite:///"+settings.SETTINGS_PATH+"\\data.db" 
     _engine = create_engine(db_path) 
     _connection = _engine.connect() 
     _metadata = MetaData() 
     _stack_items = Table(table_name, _metadata, 
          Column("id", Integer, primary_key=True), 
          Column("case", Text , unique=True), 
           ....) 
     _metadata.create_all(_engine) 
     self.connection = _connection 
     self.stack_items = _stack_items 



    def process_item(self, item, spider): 

     try: 
      ins_query = self.stack_items.insert().values(
      case=item['case'], 
      .... 
      ) 
      self.connection.execute(ins_query) 
     except IntegrityError: 
       print('THIS IS A DUP') 
     return item 

我所做的唯一變化是唯一=真加列「案例」。但是在測試中，dups仍然被添加/我如何才能使這個工作？

Answer 1

下面的代碼片段使用python版本2.7和sqlalchemy版本1.0.9和sqlite版本3.15.2工作。

from sqlalchemy import create_engine, MetaData, Column, Integer, Table, Text 
from sqlalchemy.exc import IntegrityError 


class DynamicSQLlitePipeline(object): 

    def __init__(self, table_name): 
     db_path = "sqlite:///data.db" 
     _engine = create_engine(db_path) 
     _connection = _engine.connect() 
     _metadata = MetaData() 
     _stack_items = Table(table_name, _metadata, 
          Column("id", Integer, primary_key=True), 
          Column("case", Text, unique=True),) 
     _metadata.create_all(_engine) 
     self.connection = _connection 
     self.stack_items = _stack_items 

    def process_item(self, item): 

     try: 
      ins_query = self.stack_items.insert().values(case=item['case']) 
      self.connection.execute(ins_query) 
     except IntegrityError: 
       print('THIS IS A DUP') 
     return item 

if __name__ == '__main__': 

    d = DynamicSQLlitePipeline("pipeline") 
    item = { 
     'case': 'sdjwaichjkneirjpewjcmelkdfpoewrjlkxncdsd' 
    } 
    print d.process_item(item) 

和輸出爲第二次運行將是這樣的：

THIS IS A DUP 
{'case': 'sdjwaichjkneirjpewjcmelkdfpoewrjlkxncdsd'} 

我沒有看到你的代碼邏輯多大區別。唯一的區別可能是我猜的版本。