2016-03-14 40 views
0

我正在使用apache spark,並且想要將運行我的spark-submit命令時拋出的java.net.BindException: Address already in use.異常靜音。 log4j屬性設置在下面貼出的單獨文件中。如何將log4j中的類靜音java.net.BindException

java.net.BindException: Address already in use 
    at sun.nio.ch.Net.bind0(Native Method) 
    at sun.nio.ch.Net.bind(Net.java:433) 
    at sun.nio.ch.Net.bind(Net.java:425) 
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) 
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) 
    at org.spark-project.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187) 
    at org.spark-project.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316) 
    at org.spark-project.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265) 
    at org.spark-project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) 
    at org.spark-project.jetty.server.Server.doStart(Server.java:293) 
    at org.spark-project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) 
    at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$1(JettyUtils.scala:252) 
    at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:262) 
    at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:262) 
    at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1988) 
    at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) 
    at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1979) 
    at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:262) 
    at org.apache.spark.ui.WebUI.bind(WebUI.scala:136) 
    at org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:481) 
    at org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:481) 
    at scala.Option.foreach(Option.scala:236) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:481) 
    at com.mypackage.myclass.MyUtil$class.build_context(MyUtil.scala:500) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:497) 
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) 
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) 
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) 
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) 
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 
    16/03/14 11:13:01 WARN Utils: Service 'SparkUI' could not bind on port 4050. Attempting port 4051. 

這裏是我的log4j.properties

log4j.rootCategory=WARN, console 
log4j.appender.console=org.apache.log4j.ConsoleAppender 
log4j.appender.console.target=System.err 
log4j.appender.console.layout=org.apache.log4j.PatternLayout 
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n 
log4j.logger.org.eclipse.jetty=WARN 
log4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERROR 
log4j.logger.com.jiwire=INFO 
log4j.logger.com.nd=INFO 

任何想法,我該怎麼做呢?

回答

0

該錯誤是由於Spark試圖在已經使用的端口上託管其web interface而導致的。

您可能同時運行Spark的兩個實例,使第一個實例獲取端口。

抑制這種錯誤通常是一種不好的做法。

Read here瞭解更多關於Spark配置的信息。

+0

我明白異常的原因,我仍然想從我的日誌中改變它。任何方式來做到這一點? – CruncherBigData