掃描並從依賴關係jar中讀取文件

Question

我正在開發一個使用大型數據集的spring引導命令行應用程序。數據集很麻煩，所以我想將它與我的代碼分開，但我不想讓用戶不得不管理數據。

我認爲最理想的解決方案是創建一個數據jar並從中讀取主應用程序。不幸的是，當它在一個依賴關係jar中時，我無法成功讀取數據。

我做了一個示例應用程序來證明什麼，我至今試圖

https://github.com/LewisWatson/java-data-jar

. ├── data-jar │   ├── pom.xml │   └── src │   └── main │   └── resources │   └── data │   ├── hellos │   │   └── hello.txt │   └── loremIpsums │   └── loremIpsum.txt ├── data-jar-reader │   ├── pom.xml │   └── src │   └── main │   ├── java │   │   └── com │   │   └── example │   │   └── datajarreader │   │   └── DataJarReaderApplication.java │   └── resources │   └── application.properties ├── LICENSE ├── pom.xml └── README.md

它有兩個模塊：

• data-jar包含兩個文本文件在maven的標準資源目錄中的一個data目錄中有兩個目錄。
• data-jar-reader是彈簧引導命令行應用程序，它依賴於data-jar並嘗試訪問其數據文件。

，做閱讀類是DataJarReaderApplication

@SpringBootApplication 
public class DataJarReaderApplication implements CommandLineRunner { 

    private static final Logger log = LoggerFactory.getLogger(DataJarReaderApplication.class); 

    FileSystem fileSystem; 

    public static void main(String[] args) { 
    SpringApplication.run(DataJarReaderApplication.class, args); 
    } 

    @Override 
    public void run(String... args) throws Exception { 
    Path path = getTestDataPath(); 
    List<String> data = loadTestData(path); 
    log.info("data {}", data); 
    } 

    private Path getTestDataPath() throws URISyntaxException, IOException { 

    Path path; 

    URI uri = this.getClass().getResource("/data").toURI(); 
    log.info("uri: {}", uri); 

    if (uri.getScheme().equals("jar")) { 
     fileSystem = FileSystems.newFileSystem(uri, Collections.<String, Object>emptyMap()); 
     log.info("fileSystem: {}", fileSystem); 
     path = fileSystem.getPath("/data"); 
    } else { // not in a jar, probably running in iDE 
     path = Paths.get(uri); 
    } 

    return path; 
    } 

    private List<String> loadTestData(Path testDataDirectory) throws IOException { 
    try (Stream<Path> path = Files.walk(testDataDirectory).sorted()) { 
     return loadTrackData(path); 
    } 
    } 

    private List<String> loadTrackData(Stream<Path> walk) throws IOException { 

    List<String> data = new ArrayList<>(); 

    for (Iterator<Path> it = walk.iterator(); it.hasNext();) { 

     Path path = it.next(); 

     if (path.getFileName().toString().endsWith(".txt")) { 

     log.info("text file path: {}", path); 

     List<String> dataFromFile = getData(path); 
     data.addAll(dataFromFile); 

     } else { 

     log.info("non text file path: {}", path); 

     } 

    } 

    return data; 
    } 

    private List<String> getData(Path path) throws IOException { 

    List<String> data = new ArrayList<>(); 

    try (BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) { 

     String line; 

     while ((line = reader.readLine()) != null) { 
     data.add(line); 
     } 

    } 
    return data; 
    } 
} 

輸出我得到的，當我運行的應用表明，它的查找數據在數據罐子

jar:file:/home/lewis/workspace/java-data-jar/data-jar-reader/target/data-jar-reader-0.0.1-SNAPSHOT.jar!/BOOT-INF/lib/data-jar-1.0-SNAPSHOT.jar!/data 

但是當它試圖訪問數據我得到一個沒有這樣的文件例外...

$ java -jar data-jar-reader/target/data-jar-reader-0.0.1-SNAPSHOT.jar 

    . ____   _   __ _ _ 
/\\/___'_ __ _ _(_)_ __ __ _ \ \ \ \ 
(()\___ | '_ | '_| | '_ \/ _` | \ \ \ \ 
\\/ ___)| |_)| | | | | || (_| | )))) 
    ' |____| .__|_| |_|_| |_\__, |//// 
=========|_|==============|___/=/_/_/_/ 
:: Spring Boot ::  (v1.5.7.RELEASE) 

2017-10-16 09:35:46.238 INFO 18756 --- [   main] c.e.d.DataJarReaderApplication   : Starting DataJarReaderApplication v0.0.1-SNAPSHOT on mir with PID 18756 (/home/lewis/workspace/java-data-jar/data-jar-reader/target/data-jar-reader-0.0.1-SNAPSHOT.jar started by lewis in /home/lewis/workspace/java-data-jar) 
2017-10-16 09:35:46.249 INFO 18756 --- [   main] c.e.d.DataJarReaderApplication   : No active profile set, falling back to default profiles: default 
2017-10-16 09:35:46.376 INFO 18756 --- [   main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@443b7951: startup date [Mon Oct 16 09:35:46 BST 2017]; root of context hierarchy 
2017-10-16 09:35:47.049 INFO 18756 --- [   main] o.s.j.e.a.AnnotationMBeanExporter  : Registering beans for JMX exposure on startup 
2017-10-16 09:35:47.056 INFO 18756 --- [   main] c.e.d.DataJarReaderApplication   : uri: jar:file:/home/lewis/workspace/java-data-jar/data-jar-reader/target/data-jar-reader-0.0.1-SNAPSHOT.jar!/BOOT-INF/lib/data-jar-1.0-SNAPSHOT.jar!/data 
2017-10-16 09:35:47.094 INFO 18756 --- [   main] c.e.d.DataJarReaderApplication   : fileSystem: /home/lewis/workspace/java-data-jar/data-jar-reader/target/data-jar-reader-0.0.1-SNAPSHOT.jar 
2017-10-16 09:35:47.098 INFO 18756 --- [   main] utoConfigurationReportLoggingInitializer : 

Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled. 
2017-10-16 09:35:47.111 ERROR 18756 --- [   main] o.s.boot.SpringApplication    : Application startup failed 

java.lang.IllegalStateException: Failed to execute CommandLineRunner 
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:735) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:716) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:703) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:304) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    at com.example.datajarreader.DataJarReaderApplication.main(DataJarReaderApplication.java:34) [classes!/:0.0.1-SNAPSHOT] 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131] 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131] 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131] 
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131] 
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [data-jar-reader-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] 
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [data-jar-reader-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] 
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [data-jar-reader-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] 
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [data-jar-reader-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] 
Caused by: java.nio.file.NoSuchFileException: /data 
    at com.sun.nio.zipfs.ZipPath.getAttributes(ZipPath.java:666) ~[zipfs.jar:1.8.0_131] 
    at com.sun.nio.zipfs.ZipFileSystemProvider.readAttributes(ZipFileSystemProvider.java:294) ~[zipfs.jar:1.8.0_131] 
    at java.nio.file.Files.readAttributes(Files.java:1737) ~[na:1.8.0_131] 
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) ~[na:1.8.0_131] 
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) ~[na:1.8.0_131] 
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) ~[na:1.8.0_131] 
    at java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:72) ~[na:1.8.0_131] 
    at java.nio.file.Files.walk(Files.java:3574) ~[na:1.8.0_131] 
    at java.nio.file.Files.walk(Files.java:3625) ~[na:1.8.0_131] 
    at com.example.datajarreader.DataJarReaderApplication.loadTestData(DataJarReaderApplication.java:63) [classes!/:0.0.1-SNAPSHOT] 
    at com.example.datajarreader.DataJarReaderApplication.run(DataJarReaderApplication.java:40) [classes!/:0.0.1-SNAPSHOT] 
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:732) [spring-boot-1.5.7.RELEASE.jar!/:1.5.7.RELEASE] 
    ... 14 common frames omitted 

Answer 1

如果您在IDE中運行應用程序，則不必以不同方式訪問數據。這些數據文件總是資源，例如文件在類路徑中，應該作爲資源加載。

Spring有一個方便的資源加載器，可以幫助發現類路徑中的數據文件：PathMatchingResourcePatternResolver。

例如，讓所有.txt文件中data包：

ClassLoader cl = this.getClass().getClassLoader(); 
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl); 
Resource[] resources = resolver.getResources("classpath:/data/**/*.txt") ; 
for (Resource resource: resources){ 
    logger.info(resource.getFilename()); 
} 

然後可以使用getInputStream每個資源閱讀您的文件。

Answer 2

假設你真的想使用該JAR，你有幾種選擇：

• 可以列出類路徑的數據罐子和使用ClassLoader.getResourceAsStream（relativePathWithinJar）。
• 您可以將數據jar放入某個目錄，並使用Java API將其作爲常規ZIP/JAR歸檔打開。

在第二種情況下，看看JarFile類。要打開它，你必須提供jar文件的路徑。您可以把它變成硬編碼目錄，到當前目錄或通過傳遞命令行參數，系統屬性或類似的東西，一些任意路徑。打開它之後，可以通過它們的相對（jar）路徑將各個文件作爲JarEntries獲取。

與任何大型文件，最好避免使用整個數據加載到內存中的方法和堅持，對於連續的，按需提供閱讀的InputStream方法。