Google App Engine - 從Blobstore一行一行讀取BlobstoreInputStream

我真的希望你能幫助我！Google App Engine - 從Blobstore一行一行讀取BlobstoreInputStream

目標是能夠通過我的（java - 谷歌應用程序引擎）應用程序上傳數據文件（.tsv）並獲取其內容並將其寫入到mysql數據庫中。

經過一番研究，blobstore似乎是最好的選擇，因爲該文件將在70-100MB左右。

所以我幾天前開始使用這個示例項目來讓我開始https://github.com/crhym3/java-blobstore-gcs-sample。

我得到了項目並運行沒有問題。由於我想訪問內容，並且不下載文件（這就是示例項目正在做的事情），所以我嘗試使用BlobstoreInputStream來完成此操作，正如幾個來源中所建議的那樣。所以我從示例項目中修改的唯一文件是ServeBlobServlet。我試圖通過BlobstoreInputStream獲得第一行，而不是服務整個blob。

public class ServeBlobServlet extends HttpServlet { 

BlobstoreService blobstore = BlobstoreServiceFactory.getBlobstoreService(); 

private static final Logger log = 
     Logger.getLogger(ServeBlobServlet.class.getName()); 

@Override 
protected void doGet(HttpServletRequest req, HttpServletResponse resp) 
     throws ServletException, IOException { 

    String[] parts = req.getRequestURI().split("/"); 
    String gsObjectName = URLDecoder.decode(parts[parts.length - 1], "UTF-8"); 

    log.info("Serving GCS object: " + gsObjectName); 
    BlobKey blobKey = blobstore.createGsBlobKey(gsObjectName); 

    //blobstore.serve(blobKey, resp); 
    System.out.println(getFirstLine(blobKey)); 
} 

private String getFirstLine(BlobKey blobkey) throws IOException { 
    BufferedReader reader = new BufferedReader(new InputStreamReader(new BlobstoreInputStream(blobkey))); 
    String line = reader.readLine(); 
    reader.close(); 
    return line; 
}

正如你所看到的。我實際上只是取消了註釋blobstore.serve行並添加了標準輸出行（並編寫了getFirstLine方法）。

雖然我使用blobstore.serve（它工作正常）相同的BlobKey，我得到這個錯誤。

Uncaught exception from servlet 
com.google.appengine.api.blobstore.BlobstoreInputStream$BlobstoreIOException: BlobstoreInputStream received an invalid blob key: AMIfv97t_VDPdp2Qg7UXkE3oc6nRe7yrr86jo6th4-cCU2w8HnjOF5iSOYyAlsrQDkWVLsOMYMdHlbl-AE14swvD5n03N4XStmumVlNeNtNANXI8OWNpodBmJrBB4QS7Ru_AXK5rYvV4-KizTi6kPznYvGQ6gCd1oizh0xtRwO0jkuC-K4l648XpGhZF65H-eb1JJtxWDEsMlQ9LbmRzEPJZHV3tfZ5YRLFnlzOEGnWZxj92BpORnBq8Ly_PMTA-xSNDuFsMBAJwYzdDwG3WVwDpnkxxkqvTLfTRxXKm1WgMZEtlP-0w4NbVwuJ8QNaTAzsDLCs14PFFvWV4pVMd9coXvF3KmC-Z6T4S8yC0nGK9ST0x8o0-CNI 
    at com.google.appengine.api.blobstore.BlobstoreInputStream.<init>(BlobstoreInputStream.java:121) 
    at com.google.appengine.api.blobstore.BlobstoreInputStream.<init>(BlobstoreInputStream.java:92) 
    at com.google.appengine.api.blobstore.BlobstoreInputStream.<init>(BlobstoreInputStream.java:104) 
    at com.moviemap.server.ServeBlobServlet.getFirstLine(ServeBlobServlet.java:43) 
    at com.moviemap.server.ServeBlobServlet.doGet(ServeBlobServlet.java:39) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) 
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) 
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:37) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) 
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) 
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) 
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) 
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) 
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:260) 
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) 
    at org.mortbay.jetty.Server.handle(Server.java:326) 
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) 
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) 
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:78) 
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) 
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:148) 
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:469) 
    at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437) 
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444) 
    at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:234) 
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308) 
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300) 
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441) 
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:235) 
    at java.lang.Thread.run(Thread.java:745)

我試過幾種不同的方法和BlobKey構造函數，沒有任何工作。同樣的錯誤不斷回來。

在此先感謝任何花時間幫助的人！欣賞它！

來源

2015-11-20 mikefedi

你正在閱讀一個GCS通過Blob存儲區API文檔。您應該使用https://cloud.google.com/appengine/docs/java/googlecloudstorageclient/getstarted。還要注意'createGsBlobKey'的文檔說：'爲Google存儲文件創建一個BlobKey。由filename表示的文件的存在性未被檢查，因此可以爲當前不存在的文件創建BlobKey。「情況會是這樣嗎？ – konqi

是的，你是完全正確的。 – mikefedi

下面是答案

孔雀河是在他的上面完全正確的問題發表評論。我開始的項目（鏈接在問題中）不適合直接上傳和讀取blob。正如konqi所說，我所做的（該項目的作用）是將一個blob上傳到谷歌雲存儲。

所以回答：如果你想上傳一個blob到谷歌blobstore並直接閱讀，你應該在這裏按照教程https://cloud.google.com/appengine/docs/java/blobstore/#Java_Using_the_Blobstore，但不是提供blob（參見教程中的段落），你可以閱讀它像即：

String keyString = req.getParameter("blob-key"); 
    BlobKey blobkey = new BlobKey(keyString); 
    BufferedReader reader = new BufferedReader(new InputStreamReader(new BlobstoreInputStream(blobkey))); 
    String line = null; 
    while((line = reader.readLine()) != null) { 
     //Do whatever you like with the line 
    } 
    reader.close();

希望這可以幫助別人

來源

2015-11-27 15:21:22 mikefedi

Google App Engine - 從Blobstore一行一行讀取BlobstoreInputStream

回答

相關問題