Dataflow因java.lang.NoSuchMethodError失敗：io.grpc.protobuf.ProtoUtils.marshaller（Lcom/google/protobuf/Message;）

Question

我試圖讓Dataflow作業在Google Cloud上運行。它總是失敗：

java.lang.NoSuchMethodError：io.grpc.protobuf.ProtoUtils.marshaller（LCOM /谷歌/的protobuf /消息;）LIO/GRPC/$的methodDescriptor的Marshaller

這是一個Maven項目，這裏是我的依賴：

<dependencies> 
    <dependency> 
    <groupId>com.google.cloud.dataflow</groupId> 
    <artifactId>google-cloud-dataflow-java-sdk-all</artifactId> 
    <version>1.8.0</version> 
    </dependency> 
    <dependency> 
    <groupId>com.google.cloud</groupId> 
    <artifactId>google-cloud-pubsub</artifactId> 
    <version>0.4.0</version> 
    </dependency>  
    <dependency> 
    <groupId>com.google.protobuf</groupId> 
    <artifactId>protobuf-java</artifactId> 
    <version>3.0.0</version> 
    </dependency>  
</dependencies> 

我試過一堆不同的依賴版本。我錯過了什麼？

無論我通過exec：java還是通過陰影罐運行，結果都是一樣的。

完整堆棧跟蹤：

(e8dbd0c1b8b8a22): java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lcom/google/protobuf/Message;)Lio/grpc/MethodDescriptor$Marshaller; at com.google.iam.v1.IAMPolicyGrpc.(IAMPolicyGrpc.java:56) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.(PublisherSettings.java:487) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.createDefault(PublisherSettings.java:508) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.access$000(PublisherSettings.java:402) at com.google.cloud.pubsub.spi.v1.PublisherSettings.defaultBuilder(PublisherSettings.java:224) at com.google.cloud.pubsub.spi.DefaultPubSubRpc.(DefaultPubSubRpc.java:138) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubRpcFactory.create(PubSubOptions.java:60) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubRpcFactory.create(PubSubOptions.java:54) at com.google.cloud.ServiceOptions.rpc(ServiceOptions.java:399) at com.google.cloud.pubsub.PubSubImpl.(PubSubImpl.java:115) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubFactory.create(PubSubOptions.java:43) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubFactory.create(PubSubOptions.java:38) at com.google.cloud.ServiceOptions.service(ServiceOptions.java:391) at com.google.lindsaysmith.titan.DataflowBulkLoadNodes$SendPubSub.sendPubsubMessage(DataflowBulkLoadNodes.java:41) at com.google.lindsaysmith.titan.DataflowBulkLoadNodes$SendPubSub.processElement(DataflowBulkLoadNodes.java:32) at com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:139) at com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:190) at com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47) at com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:55) at

Answer 1

在這種情況下，通常的答案是，你真的必須使用GRPC和協議緩衝的完全版本，在數據流SDK的依賴性聲明。這包括所有的傳遞依賴關係，因此您可能必須禁止其他庫的gRPC或protobuf依賴關係，因此它們不會干涉。

您可以看到版本here (gRPC)和here (protobuf)。我將他們從這個答案中刪除，以免它過時。