多個CSV文件複製到Postgres的

Question

我寫一個SQL腳本多個.CSV文件複製到一個Postgres數據庫是這樣的：

COPY product(title, department) from 'ys.csv' CSV HEADER; 

我有多個文件，我想複製我不想：

COPY product(title, department) from 'ys1.csv' CSV HEADER; 
COPY product(title, department) from 'ys2.csv' CSV HEADER; 
COPY product(title, department) from 'ys3.csv' CSV HEADER; 
COPY product(title, department) from 'ys4.csv' CSV HEADER; 
COPY product(title, department) from 'ys5.csv' CSV HEADER; 

我想用for循環來代替多個拷貝命令。這可能嗎？謝謝

Answer 1

在linux中管道輸出文件列表到psql。讓copy使用標準輸入：

cat /path_to/ys*.csv | psql -c 'COPY product(title, department) from stdin CSV HEADER' 

尋找其他操作系統

Answer 2

相當於我嘗試了上述答案，但有一個以上的文件，工作時我得到了一個錯誤。我認爲在第二個文件沒有切斷標題。

這工作FOT我：

# get filenames 
IMPFILES=(path/FileNamepart.csv) 

# import the files 
for i in ${IMPFILES[@]} 
    do 
     psql -U user -d database -c "\copy TABLE_NAME from '$i' DELIMITER ';' CSV HEADER" 
     # move the imported file 
     mv $i /FilePath 
    done 

在我來說，我每移動文件AFER是進口的。如果發生錯誤，我知道在哪裏尋找。如果有新文件放在該位置，我可以再次運行該腳本。

Answer 3

您可以使用pg_ls_dir遍歷文件名。

DO $$ 

DECLARE file_path TEXT; -- Path where your CSV files are 
DECLARE fn_i TEXT; -- Variable to hold name of current CSV file being inserted 
DECLARE mytable TEXT; -- Variable to hold name of table to insert data into 

BEGIN 

    file_path := 'C:/Program Files/PostgreSQL/9.6/data/my_csvs/'; -- Declare the path to your CSV files. You probably need to put this in your PostgreSQL file path to avoid permission issues. 
    mytable := 'product(title,department)'; -- Declare table to insert data into. You can give columns too since it's just going into an execute statement. 

    CREATE TEMP TABLE files AS 
    SELECT file_path || pg_ls_dir AS fn -- get all of the files in the directory, prepending with file path 
    FROM pg_ls_dir(file_path); 

    LOOP  
     fn_i := (select fn from files limit 1); -- Pick the first file 
     raise notice 'fn: %', fn_i; 
     EXECUTE 'COPY ' || mytable || ' from ''' || fn_i || ''' with csv header'; 
     DELETE FROM files WHERE fn = fn_i; -- Delete the file just inserted from the queue 
     EXIT WHEN (SELECT COUNT(*) FROM files) = 0; 
    END LOOP; 

END $$;