如何從Java中的S3讀取Snappy壓縮文件

Question

目前我們在Hadoop中運行MapReduce作業，其輸出壓縮爲SnappyCompression。然後我們將輸出文件移至S3。現在我想通過Java從S3讀取Compressed文件。

Answer 1

我發現答案從S3讀快速壓縮文件。首先，您應該從S3獲取對象內容。然後解壓縮文件。

S3Object s3object = s3Client.getObject(new GetObjectRequest(bucketName,Path)); 
    InputStream inContent = s3object.getObjectContent(); 
    CompressionCodec codec = (CompressionCodec) ReflectionUtils.newInstance(SnappyCodec.class, new Configuration()); 
    InputStream inStream = codec.createInputStream(new BufferedInputStream(inContent)); 
    InputStreamReader inRead = new InputStreamReader(inStream); 
    BufferedReader br = new BufferedReader(inRead); 
    String line=null; 
    while ((line = br.readLine()) != null){ 
     system.out.println(line); 
    }