PowerShell的：獲取內容從大文件（服務器列表）

Question

我從文本文件（serverlist.txt）服務器100,000名單

當我在一個鏡頭中運行它會爆我的內存和CPU以及時間需要更長的時間（大約3天）才能完成DNS查找掃描。

我試圖拆分下面包含20k個服務器列表的文件，並且可以完成爲每個文件掃描最多10分鐘。

serverlist1.txt 
serverlist2.txt 
serverlist3.txt 
serverlist4.txt 
serverlist5.txt 

---

$objContainer = @() 
$values = @() 
$domains = Get-Content -path "serverlist1.txt" 
$named = 0 
$timestamp= get-date 

$domains | ForEach-Object { 
    $domain = $_ 
    nslookup $domain 2>&1 | ForEach-Object { 
     if ($_ -match '^Name:\s*(.*)$') { 
      $values += $matches[1] 
      $named = 1; 
     } elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) { 
      $values += $matches[1] 
     } elseif ($_ -match '^Aliases:\s*(.*)$') { 
      $values += $matches[1] 
     } 
    } 

    $obj = New-Object -TypeName PSObject 
    #$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain 
    $obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0] 
    $obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1] 
    $obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2] 
    $obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp 
    $objContainer += $obj 

    $values = @() 
    $named = 0 
} 

Write-Output $objContainer 
$objContainer | Export-csv "dnslog_$((Get-Date).ToString('MM-dd-yyyy_hh-mm-ss')).csv" -NoTypeInformation 

我的問題是，如何執行一次循環，並從文本文件輸入生成dnslog（日期時間）後.csv文件

如：

1. 運行powershell腳本。\ filename.ps1
2. 從serverlist1.txt輸入
3. 輸出dnslog（日期時間）.csv文件
4. 輸入從serverlist2.txt
5. 輸出dnslog（日期時間）.csv文件從serverlist3.txt
6. 輸出dnslog（日期時間）
7. 輸入.csv文件
8. 輸入從serverlist4.txt
9. 輸出dnslog（日期時間）的.csv從serverlist5.txt
10. 輸入
11. 輸出dnslog（日期時間）的.csv

完成！

如果我有多於5個文本文件列表，它將繼續從輸入文件循環直到完成。

Answer 1

添加到克里斯的回答我還要補充一個ReadCount標誌的獲取內容如下所示：

Get-Content -path "serverlist1.txt" -ReadCount 1 | % { 

這將節省不必將整個文件讀入內存中。

Answer 2

你應該考慮運行這個並行批處理作業。你已經嘗試過嗎？

您可以通過刪除所有對內存的提交（使用+ =進行變量賦值和數組重寫）來處理RAM破壞問題。

$timestamp = get-date 

Get-Content -path "serverlist1.txt" | ForEach-Object { 
    $domain = $_ 

    # You can clear this here. 
    $values = @() 
    $named = 0 

    # There are potentially better options than nslookup. 
    # Needs a bit of care to understand what's an alias here though. 
    # [System.Net.Dns]::GetHostEntry($domain) 
    # And if you don't like that, quite a few of us have written equivalent tools in PowerShell. 
    nslookup $domain 2>&1 | ForEach-Object { 
     if ($_ -match '^Name:\s*(.*)$') { 
      $values += $matches[1] 
      $named = 1; 
     } elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) { 
      $values += $matches[1] 
     } elseif ($_ -match '^Aliases:\s*(.*)$') { 
      $values += $matches[1] 
     } 
    } 

    # Leave the output object in the output pipeline 
    # If you're running PowerShell 3 or better: 
    [PSCustomObject]@{ 
     Domain  = $domain 
     Name   = $values[0] 
     'IP Address' = $values[1] 
     Alias  = $values[2] 
     TimeStamp = $timestamp 
    } 
    # PowerShell 2 is less flexible. This or Select-Object. 
    #$obj = New-Object -TypeName PSObject 
    ##$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain 
    #$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0] 
    #$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1] 
    #$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2] 
    #$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp 
    # To leave this in the output pipeline, uncomment this 
    # $obj 

    # No version of PowerShell needs you to do this. It's a good way to ramp up memory usage 
    # for large data sets. 
    # $objContainer += $obj 
} | Export-Csv "dnslog_$(Get-Date -Format 'MM-dd-yyyy_hh-mm-ss').csv" -NoTypeInformation