當使用sbt構建一個胖罐子時，build.sbt中的重複數據刪除錯誤

Question

我正在嘗試創建一個組合框架的胖子，但我不斷收到此重複數據刪除錯誤。

這是運行SBT裝配錯誤：

java.lang.RuntimeException: deduplicate: different file contents found in the following: 
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA 
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA 
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA 
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA 
      at sbtassembly.Plugin$Assembly$.sbtassembly$Plugin$Assembly$$applyStrategy$1(Plugin.scala:253) 
      at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:270) 
      at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:267) 
      at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) 
      at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) 
      at scala.collection.Iterator$class.foreach(Iterator.scala:727) 
      at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) 
      at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) 
      at scala.collection.AbstractIterable.foreach(Iterable.scala:54) 
      at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251) 
      at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105) 
      at sbtassembly.Plugin$Assembly$.applyStrategies(Plugin.scala:272) 
      at sbtassembly.Plugin$Assembly$.x$4$lzycompute$1(Plugin.scala:172) 
      at sbtassembly.Plugin$Assembly$.x$4$1(Plugin.scala:170) 
      at sbtassembly.Plugin$Assembly$.stratMapping$lzycompute$1(Plugin.scala:170) 
      at sbtassembly.Plugin$Assembly$.stratMapping$1(Plugin.scala:170) 
      at sbtassembly.Plugin$Assembly$.inputs$lzycompute$1(Plugin.scala:214) 
      at sbtassembly.Plugin$Assembly$.inputs$1(Plugin.scala:204) 
      at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:230) 
      at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:373) 
      at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:370) 
      at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47) 
      at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42) 
      at sbt.std.Transform$$anon$4.work(System.scala:64) 
      at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237) 
      at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237) 
      at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18) 
      at sbt.Execute.work(Execute.scala:244) 
      at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237) 
      at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237) 
      at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160) 
      at sbt.CompletionService$$anon$2.call(CompletionService.scala:30) 
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
      at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
      at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
      at java.lang.Thread.run(Thread.java:724) 
    [error] (*:assembly) deduplicate: different file contents found in the following: 
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA 
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA 
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA 
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA 
    [error] Total time: 8 s, completed 19/09/2014 11:50:50 AM 

這是我plugins.sbt文件

logLevel := Level.Warn 

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2") 

這是我build.sbt文件

import AssemblyKeys._ // put this at the top of the file 

name := "Spark_sbt2" 

version := "1.1" 

scalaVersion := "2.10.4" 

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0" 

libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.1.0" 

libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.1.0" 

libraryDependencies += "org.apache.httpcomponents" % "httpclient" % "4.3.5" 

libraryDependencies += "org.restlet.jse" % "org.restlet" % "2.2.2" 

libraryDependencies += "org.restlet.jse" % "org.restlet.ext.jackson" % "2.2.2" 

resolvers += "Akka Repository" at "http://repo.akka.io/releases/" 

resolvers += "Restlet" at "http://maven.restlet.org" 

assemblySettings 

jarName in assembly := "spark-assembly.jar" 

我不知道如何解決它。感謝任何人都可以幫忙！

Answer 1

問題是sbt-assembly無法選擇要拾取哪個META-INF/ECLIPSEF.RSA文件。試試這個排除所有這些文件：

excludedFiles in assembly <<= 
(excludedFiles in assembly) { 
    (old) => (bases) => 
    old(bases) ++ (bases flatMap (base => 
     (base/"META-INF/ECLIPSEF.RSA").get)) 
} 

如果你真的需要裏面的jar一些RSA密鑰：

assembledMappings in assembly <<= (assembledMappings in assembly) map { (old: File => Seq[(File, String)]) => 
    (f: File) => old(f) :+ (file("path/to/choosen/rsa/key"), "META-INF/ECLIPSEF.RSA") 
} 

（或不提它excludedFiles適當基地）