使用TIKA從URL中提取文本

Question

是否可以使用Tika從URL中提取文本？任何鏈接將不勝感激。或TIKA僅適用於pdf，文字和其他媒體文件？

Answer 1

這是lucid：

InputStream input = new FileInputStream(new File(resourceLocation)); 
ContentHandler textHandler = new BodyContentHandler(); 
Metadata metadata = new Metadata(); 
PDFParser parser = new PDFParser(); 
parser.parse(input, textHandler, metadata); 
input.close(); 
out.println("Title: " + metadata.get("title")); 
out.println("Author: " + metadata.get("Author")); 
out.println("content: " + textHandler.toString()); 

而不是創建一個PDFParser可以使用Tika的AutoDetectParser來自動處理文件的DIFF類型：

Parser parser = new AutoDetectParser(); 

Answer 2

檢查documentation - 是的，你可以。

例

java -jar tika-app-0.9.jar -t http://stackoverflow.com/questions/6656849/extract-the-text-from-url-using-tika 

會告訴你此頁面上的文本。

Answer 3

從本地文件不是從URL中提取內容使用此代碼：

byte[] raw = content.getContent(); 
    ContentHandler handler = new BodyContentHandler(); 
    Metadata metadata = new Metadata(); 
    Parser parser = new AutoDetectParser(); 
    parser.parse(new ByteArrayInputStream(raw), handler, metadata, new ParseContext()); 
    LOG.info("content: " + handler.toString()); 

Answer 4

是的，你可以做到這一點。這是代碼。該代碼使用Apache http client

HttpGet httpget = new HttpGet("http://url.here"); 
    HttpEntity entity = null; 
    HttpClient client = new DefaultHttpClient(); 
    HttpResponse response = client.execute(httpget); 
    entity = response.getEntity(); 
    if (entity != null) { 
     InputStream instream = entity.getContent(); 
     BodyContentHandler handler = new BodyContentHandler(); 
     Metadata metadata = new Metadata(); 
     Parser parser = new AutoDetectParser(); 
     parser.parse(instream, handler, metadata, new ParseContext()); 
     String plainText = handler.toString(); 
     FileWriter writer = new FileWriter("/scratch/cache/output.txt"); 
     writer.write(plainText); 
     writer.close(); 
     System.out.println("done"); 
    }