2016-10-15 100 views
3

我想在eclipse中使用log4j生成日誌來構建一個基於scala的jar文件。它在控制檯中完美地打印出來,但是當我嘗試使用log4j.properties文件將其寫入日誌文件時,沒有任何反應。Scala:用log4j寫日誌到文件

項目結構如下

Maven project with scala nature

loggerTest.scala

package scala.n*****.n**** 

import org.apache.log4j.Logger 

object loggerTest extends LogHelper { 
    def main(args : Array[String]){ 
    log.info("This is info"); 
    log.error("This is error"); 
    log.warn("This is warn"); 
    } 
} 

trait LogHelper { 
    lazy val log = Logger.getLogger(this.getClass.getName) 
} 

log4j.properties

# Root logger option 
log4j.rootLogger=WARN, stdout, file 

# Redirect log messages to console 
log4j.appender.stdout=org.apache.log4j.ConsoleAppender 
log4j.appender.stdout.Target=System.out 
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout 
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n 

# Redirect log messages to a log file 
log4j.appender.file=org.apache.log4j.RollingFileAppender 
log4j.appender.file.File=/home/abc/test/abc.log 
log4j.appender.file.encoding=UTF-8 
log4j.appender.file.MaxFileSize=2kB 
log4j.appender.file.MaxBackupIndex=1 
log4j.appender.file.layout=org.apache.log4j.PatternLayout 
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n 

的pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
<modelVersion>4.0.0</modelVersion> 
<groupId>loggerTest</groupId> 
<artifactId>loggerTest</artifactId> 
<version>0.0.1-SNAPSHOT</version> 
<name>loggerTest</name> 
<description>loggerTest</description> 

<repositories> 
    <repository> 
    <id>scala-tools.org</id> 
    <name>Scala-tools Maven2 Repository</name> 
    <url>http://scala-tools.org/repo-releases</url> 
    </repository> 
    <repository> 
    <id>maven-hadoop</id> 
    <name>Hadoop Releases</name> 
    <url>https://repository.cloudera.com/content/repositories/releases/</url> 
    </repository> 
    <repository> 
    <id>cloudera-repos</id> 
    <name>Cloudera Repos</name> 
    <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url> 
    </repository> 
</repositories> 

<pluginRepositories> 
    <pluginRepository> 
    <id>scala-tools.org</id> 
    <name>Scala-tools Maven2 Repository</name> 
    <url>http://scala-tools.org/repo-releases</url> 
    </pluginRepository> 
</pluginRepositories> 

<properties> 
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> 
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> 
</properties> 

<build> 
    <plugins> 
     <!-- mixed scala/java compile --> 
     <plugin> 
      <groupId>org.scala-tools</groupId> 
      <artifactId>maven-scala-plugin</artifactId> 
      <executions> 
       <execution> 
        <id>compile</id> 
        <goals> 
         <goal>compile</goal> 
        </goals> 
        <phase>compile</phase> 
       </execution> 
       <execution> 
        <id>test-compile</id> 
        <goals> 
         <goal>testCompile</goal> 
        </goals> 
        <phase>test-compile</phase> 
       </execution> 
       <execution> 
        <phase>process-resources</phase> 
        <goals> 
         <goal>compile</goal> 
        </goals> 
       </execution> 
      </executions> 
     </plugin> 
     <plugin> 
      <artifactId>maven-compiler-plugin</artifactId> 
      <configuration> 
       <source>1.7</source> 
       <target>1.7</target> 
      </configuration> 
     </plugin> 

     <plugin> 
      <groupId>org.apache.maven.plugins</groupId> 
      <artifactId>maven-jar-plugin</artifactId> 
      <configuration> 
       <archive> 
        <manifest> 
         <addClasspath>true</addClasspath> 
         <mainClass>fully.qualified.MainClass</mainClass> 
        </manifest> 
       </archive> 
      </configuration> 
     </plugin> 
    </plugins> 
    <pluginManagement> 
     <plugins> 
      <!--This plugin's configuration is used to store Eclipse m2e settings 
       only. It has no influence on the Maven build itself. --> 
      <plugin> 
       <groupId>org.eclipse.m2e</groupId> 
       <artifactId>lifecycle-mapping</artifactId> 
       <version>1.0.0</version> 
       <configuration> 
        <lifecycleMappingMetadata> 
         <pluginExecutions> 
          <pluginExecution> 
           <pluginExecutionFilter> 
            <groupId>org.scala-tools</groupId> 
            <artifactId> 
             maven-scala-plugin 
            </artifactId> 
            <versionRange> 
             [2.15.2,) 
            </versionRange> 
            <goals> 
             <goal>compile</goal> 
             <goal>testCompile</goal> 
            </goals> 
           </pluginExecutionFilter> 
           <action> 
            <execute></execute> 
           </action> 
          </pluginExecution> 
         </pluginExecutions> 
        </lifecycleMappingMetadata> 
       </configuration> 
      </plugin> 
     </plugins> 
    </pluginManagement> 
</build> 

<dependencies> 
<dependency> 
<groupId>org.apache.spark</groupId> 


<artifactId>spark-core_2.10</artifactId> 
    <version>1.5.0</version> 
    </dependency> 

    <dependency> 
    <groupId>org.apache.spark</groupId> 
    <artifactId>spark-hive_2.10</artifactId> 
    <version>1.5.0</version> 
    </dependency> 

    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-sql_2.10</artifactId> 
     <version>1.5.0</version> 
    </dependency> 
     <dependency> 
      <groupId>org.scala-lang</groupId> 
      <artifactId>scala-library</artifactId> 
      <version>2.10.4</version> 
     </dependency> 
     </dependencies> 

    </project> 

當我將它作爲maven build運行時,它會在「target」文件夾中生成一個jar文件。

我複製的jar到/ home/ABC /測試

我跑火花外殼,罐子命令

$ spark-submit --class scala.n*****.n*****.loggerTest loggerTest-0.0.1-SNAPSHOT.jar 

控制檯出來沒事,但它應該寫在/ home的文件/ abc/log它沒有。

有人可以幫忙嗎?

+0

可你只是嘗試有了這個 ? log4j.rootLogger = WARN,文件 – BDR

+3

我認爲Spark有自己的默認log4j屬性,它們會覆蓋你的屬性,因爲你通過'spark-submit'運行這個屬性。嘗試單獨運行代碼(例如,'java -jar loggerTest-0.0.1-SNAPSHOT.jar'或'scala loggerTest-0.0.1-SNAPSHOT.jar')以查看您的配置,然後閱讀關於使用Spark-提交:http://spark.apache.org/docs/latest/configuration.html#configuring-logging –

回答

2

你好,而你正在部署你的應用程序,你應該定義log4j的文件執行者和駕駛員一步的解決方案如下

spark-submit --class MAIN_CLASS --driver-java-options "-Dlog4j.configuration=file:PATH_OF_LOG4J" --conf "spark.executor.extraJavaOptions=-Dlog4j.configuration=file:PATH_OF_LOG4J" --master MASTER_IP:PORT JAR_PATH 

欲瞭解更多細節和步驟中,您可以檢查此鏈接https://blog.knoldus.com/2016/02/23/logging-spark-application-on-standalone-cluster/

+0

該行工作,但只有當我把屬性文件放在與jar相同的目錄。但我似乎無法給出src/main/resources中的道具文件的路徑。你能幫我解決嗎? – Amber

+0

你部署你的應用程序與客戶端模式soo你應該總是把參數不能設置在你的spark conf中的java額外選項,因爲直到jvm開始,如果你通過參數傳遞它,你不能訪問你的jar中的log4j –

+0

你可以把它在某個地方給它的路徑不正確? –