谷歌雲扳手批量插入過程中的瞬態故障異常

Question

由於我預計隨着時間的推移，行數將增長到數以百萬計，因此我目前正在測試Google Cloud Spanner作爲一個項目的MySQL替代方案。數據庫需要響應速度非常快，並在幾秒鐘內返回查詢結果，所以我想我會給Spanner一個嘗試。

我試圖批量加載樣本數據來我的扳手DB，但我不斷收到此錯誤：

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> Google.Cloud.Spanner.Data.SpannerException: The operation was aborted. ---> Grpc.Core.RpcException: Status(StatusCode=Aborted, Detail="Aborted due to transient fault")

我使用的是性能稍微修改後的代碼從這裏：https://github.com/GoogleCloudPlatform/dotnet-docs-samples/blob/master/spanner/api/Program.cs

下面是代碼修改在InsertSampleData：

public static object InsertSampleData(string projectId, 
     string instanceId, string databaseId) 
    { 
     // I get about 100k rows here 
     List<Data> data = get_data(); 

     // how many runs I need to make if I split the data by 100 rows 
     int rows = 100; 
     double cnt = (double)data.Count/rows; 
     cnt = Math.Ceiling(cnt); 

     // process the data part by part 
     for (int i = 0; i < cnt; i++) 
     { 
      // returns part of the data based on offset and amount 
      List<Data> data_part = get_part(data, i, rows); 

      var response = InsertTradesAsync(
      projectId, instanceId, databaseId, data_part); 
      s_logger.Info("Waiting for operation to complete..."); 
      response.Wait(); 
      s_logger.Info($"Operation status: {response.Status}"); 
     } 

     return ExitCode.Success; 
    } 

InsertTradesAsync是一樣的，在回購（除了當然參數列表）。

當我運行的代碼，我總是得到以下錯誤：

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> Google.Cloud.Spanner.Data.SpannerException: The operation was aborted. ---> Grpc.Core.RpcException: Status(StatusCode=Aborted, Detail="Aborted due to transient fault") 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) 
    at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass0_0`2.<<WithRetry>b__0>d.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.V1.Internal.ExecuteHelper.<WithSessionChecking>d__0`1.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.V1.TransactionPool.<RunFinalMethodAsync>d__9`1.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.Data.SpannerTransaction.<<CommitAsync>b__29_0>d.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.Data.ExecuteHelper.<WithErrorTranslationAndProfiling>d__2`1.MoveNext() 
    --- End of inner exception stack trace --- 
    at Google.Cloud.Spanner.Data.ExecuteHelper.<WithErrorTranslationAndProfiling>d__2`1.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.Data.EphemeralTransaction.<>c__DisplayClass2_0.<<ExecuteMutationsAsync>b__1>d.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.Data.ExecuteHelper.<WithErrorTranslationAndProfiling>d__2`1.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at Google.Cloud.Spanner.Data.SpannerCommand.<ExecuteMutationsAsync>d__49.MoveNext() 
--- End of stack trace from previous location where exception was thrown --- 
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
    at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() 
    at GoogleCloudSamples.Spanner.Program.<InsertTradesAsync>d__25.MoveNext() in c:\Users\user\Documents\Dev\spanner\api\Program.cs:line 1298 
    --- End of inner exception stack trace --- 
    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) 
    at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) 
    at System.Threading.Tasks.Task.Wait() 
    at GoogleCloudSamples.Spanner.Program.InsertSampleData(String projectId, String instanceId, String databaseId) in c:\Users\user\Documents\Dev\spanner\api\Program.cs:line 1585 
    at GoogleCloudSamples.Spanner.Program.<>c__DisplayClass45_0.<Main>b__2(InsertSampleDataOptions opts) in c:\Users\user\Documents\Dev\spanner\api\Program.cs:line 1932 
    at CommandLine.ParserResultExtensions.MapResult[T1,T2,T3,T4,T5,T6,T7,T8,T9,T10,T11,T12,T13,T14,T15,T16,TResult](ParserResult`1 result, Func`2 parsedFunc1, Func`2 parsedFunc2, Func`2 parsedFunc3, Func`2 parsedFunc4, Func`2 parsedFunc5, Func`2 parsedFunc6, Func`2 parsedFunc7, Func`2 parsedFunc8, Func`2 parsedFunc9, Func`2 parsedFunc10, Func`2 parsedFunc11, Func`2 parsedFunc12, Func`2 parsedFunc13, Func`2 parsedFunc14, Func`2 parsedFunc15, Func`2 parsedFunc16, Func`2 notParsedFunc) 
    at GoogleCloudSamples.Spanner.Program.Main(String[] args) in c:\Users\user\Documents\Dev\spanner\api\Program.cs:line 1910 

我想這可能與限制和配額（https://cloud.google.com/spanner/quotas），但是異常不同量的行被插入後拋出我的桌子（這似乎是隨機的，有時發生在68次運行後，每次100行，然後是28x100,52x100等）。該表有30列，PK由2列（沒有索引）組成，我正在處理100個數據，所以我認爲我沒有達到極限。

如果我將cmd.CommandTimeout設置爲非常高的數字，我得到更高的插入行（大約400x100） - 我假設客戶端庫重用連接？但是我找不到任何有關C＃庫的信息。即使插入了更多行，錯誤仍然會發生。

任何幫助將非常感激。

謝謝！

Answer 1

Google開發者在這裏。 我有幾點建議可以幫助你。

首先是您應該使用單個事務並儘可能多地添加寫入。它看起來像你一次做100個？但你可以做更多。你可以看到有多少寫在這裏可以批量限制（我相信這是20,000）：

https://github.com/GoogleCloudPlatform/dotnet-docs-samples/blob/master/spanner/api/Program.cs#L1242

第二項建議直接解決你的問題。因爲如何扳手的作品，您需要使用您的交易圍繞重如在這裏例如：

https://github.com/GoogleCloudPlatform/dotnet-docs-samples/blob/master/spanner/api/Program.cs#L1259

（點擊此處下載瞬時故障應用程序的NuGet）： https://www.nuget.org/packages/EnterpriseLibrary.TransientFaultHandling/

你需要這個，因爲扳手可能偶爾會遇到一個僵局，迫使您重新運行整個交易。我們提供例外的擴展方法「IsTransientSpannerFault」，使其更容易構建您重試的政策，如：

internal class CustomTransientErrorDetectionStrategy 
     : ITransientErrorDetectionStrategy 
    { 
     public bool IsTransient(Exception ex) => 
      ex.IsTransientSpannerFault(); 
    } 

希望這有助於！

編輯：我只是注意到，你不是在等待異步調用的結果InsertTradesAsync。你可能想至少在最後做一個Task.WaitAll。請注意，每個批次的100次寫入操作都可能與前一批次並行運行。這很可能是增加了導致重試的僵局的可能性。

如果你這樣做的目的，以提高性能，在幕後池扳手ADO.NET庫將打開多個連接等於SpannerConnection.SpannerOptions.MaximumGrpcChannels。所以你會看到一定的收穫。您可以增加此值來調整您的表現。

關於