建立批量插入語句powershell到sql

Question

我有一個powershell腳本，它從特定目錄遞歸地寫入每個文件及其屬性。這可以工作，但目錄可能有多達1,000,000個文件。我想要做的是每個事務以1000個插入點進行批量處理。原來這裏是PS：

$server = "" 
$Database = "" 
$Path = "C:\Test" 

$Connection = New-Object System.Data.SQLClient.SQLConnection 
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;" 
$Connection.Open() 
$Command = New-Object System.Data.SQLClient.SQLCommand 
$Command.Connection = $Connection 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object Name,Length,Mode, Directory,CreationTime, LastAccessTime, LastWriteTime) { 
    $fileName = $file.Name 
    $fileSize = ([int]$file.Length) 
    $fileMode = $file.Mode 
    $fileDirectory = $file.Directory 
    $fileCreationTime = [datetime]$file.CreationTime 
    $fileLastAccessTime = [datetime]$file.LastAccessTime 
    $fileLastWriteTime = [datetime]$file.LastWriteTime 

    $sql = " 
       begin 
        insert TestPowerShell 
        select '$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime' 
       end 
      " 
    $Command.CommandText = $sql 
    echo $sql 
    $Command.ExecuteNonQuery() 
} 

$Connection.Close() 

我的想法是實現某種計數器，將繼續追加插入，直到它到達1000，然後跳出循環並執行。我無法弄清楚這個當前的設置如何在1000進行批處理，然後執行get-childitem循環，然後選擇備份。

Answer 1

任何有興趣 - 這是做這件事：

function WriteBatch { 
    echo $sql 
    $Command.CommandText = $sql 
     $Command.ExecuteNonQuery() 
} 

$server = "" 
$Database = "" 
$Path = "" 
$Counter = 0 

$Connection = New-Object System.Data.SQLClient.SQLConnection 
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;" 
$Connection.Open() 
$Command = New-Object System.Data.SQLClient.SQLCommand 
$Command.Connection = $Connection 
[string]$sql =  " 
       begin 
        insert into TestPowerShell(NameString, FileSize, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) 
        values " 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object Name, Length, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) { 
    $fileName = $file.Name 
    $fileSize = ([int]$file.Length) 
    $fileMode = $file.Mode 
    $fileDirectory = $file.Directory 
    $fileCreationTime = [datetime]$file.CreationTime 
    $fileLastAccessTime = [datetime]$file.LastAccessTime 
    $fileLastWriteTime = [datetime]$file.LastWriteTime 
    $sql = $sql + "('$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime')," 
    $sql += "`n" 
    $Counter++ 

    If($Counter -eq 900) { 
     $sql = $sql.Trim().Trim(',') 
     $sql = $sql + " End" 
     WriteBatch 
     $Counter = 0 
     $sql = " 
       begin 
        insert into TestPowerShell(NameString, FileSize, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) 
        values " 
    } 

} 

if ($Counter -gt 0){ 
    $sql = $sql.Trim().Trim(',') 
    $sql = $sql + " End" 
    WriteBatch 
} 

$Connection.Close() 

Answer 2

像這樣的東西應該做的：

function Execute-SqlQuery($query){ 
    Write-Host "Executing query:" 
    Write-Host $query; 
} 

$data = @(1,2,3,4,5,6,7,8,9,10,11); 
$batchSize = 2; 
$counter = 0; 
$sql = ""; 

foreach($item in $data){ 

    if($counter -eq $batchSize){ 
     Execute-SqlQuery $sql; 
     $counter = 0; 
     $sql = ""; 
    } 

    $sql += "insert into myTable(id) values($item) `n"; 
    $counter += 1;  
} 

Execute-SqlQuery $sql; 

Answer 3

$server = "" 
$Database = "" 
$Path = "C:\Test" 

$Connection = New-Object System.Data.SQLClient.SQLConnection 
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;" 
$Connection.Open() 
$Command = New-Object System.Data.SQLClient.SQLCommand 
$Command.Connection = $Connection 

# new variables to handle batching 
$batchcounter=0 
$batchsize=1000 
$sqlValues = New-Object Collections.ArrayList 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object Name,Length,Mode, Directory,CreationTime, LastAccessTime, LastWriteTime) { 
    $fileName = $file.Name 
    $fileSize = ([int]$file.Length) 
    $fileMode = $file.Mode 
    $fileDirectory = $file.Directory 
    $fileCreationTime = [datetime]$file.CreationTime 
    $fileLastAccessTime = [datetime]$file.LastAccessTime 
    $fileLastWriteTime = [datetime]$file.LastWriteTime 

    $sqlValues.Add("('$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime')") 

    $batchcounter++ 

    # if the counter hits batchsize, run the insert, using lots of: 
    # insert into table 
    # values (1,2,3) 
    #  , (4,5,6) 
    #  , (7,8,9) 

    if ($batchcounter % $batchsize -eq 0) { 
     $sql = "insert TestPowerShell values {0}" -f ($sqlValues.ToArray() -join "`r`n,") 
     $Command.CommandText = $sql 
     Write-Host $sql 
     $Command.ExecuteNonQuery() 
     $sqlValues.Clear() 
    } 
} 

# catch any remaining files 
if ($batchcounter -gt 0) { 
    $sql = "insert TestPowerShell values {0}" -f ($sqlValues.ToArray() -join "`r`n,") 
    $Command.CommandText = $sql 
    Write-Host $sql 
    $Command.ExecuteNonQuery() 
    $sqlValues.Clear() 
} 

$Connection.Close()