如何使用Java在mongodb上載json文件？

Question

我想使用Java將大的JSON文件（newclicklogs.json）上傳到MongoDB中。這裏是我的JSON文件看起來像：

{"preview":false,"result":{"search_term":"rania","request_time":"Sat Apr 01 12:47:04 -0400 2017","request_ip":"127.0.0.1","stats_type":"stats","upi":"355658761","unit":"DR","job_title":"Communications Officer","vpu":"INP","organization":"73","city":"Wash","country":"DC","title":"Tom","url":"www.demo.com","tab_name":"People-Tab","page_name":"PEOPLE","result_number":"5","page_num":"0","session_id":"df234f468cb3fe8be","total_results":"5","filter":"qterm=rina","_time":"2017-04-01T12:47:04.000-0400"}} 
{"preview"......} 
{"preview"......} 
.... 

這裏是我的Java代碼：

import java.io.File; 
import java.io.IOException; 
import java.util.ArrayList; 
import java.util.List; 
import org.apache.commons.io.FileUtils; 
import org.bson.Document; 
import com.mongodb.DBObject; 
import com.mongodb.MongoClient; 

public class Main { 

    public static void main(String[] args) throws IOException { 

     String jsonString = FileUtils.readFileToString(new File("data/newclicklogs.json"), "UTF-8"); 

     Document doc = Document.parse(jsonString); 
     List<Document> list = new ArrayList<>(); 
     list.add(doc); 

     new MongoClient().getDatabase("test2").getCollection("collection1").insertMany(list); 

    } 
} 

當我詢問我的MongoDB集合，只有一個文件獲取添加。如何將我的文件中的所有文檔添加到mongodb集合中。我是MongoDB的新手。任何幫助表示讚賞。

Answer 1

您應該嘗試使用帶緩衝讀取器的批量寫入。

下面的代碼將從文件，一行（文檔）中讀取json數據，然後在將數據寫入數據庫之前解析json到Document和批量請求。

MongoClient client = new MongoClient("localhost", 27017); 
MongoDatabase database = client.getDatabase("test2"); 
MongoCollection<Document> collection = database.getCollection("collection1"); 

int count = 0; 
int batch = 100; 

List<InsertOneModel<Document>> docs = new ArrayList<>(); 

try (BufferedReader br = new BufferedReader(new FileReader("data/newclicklogs.json"))) { 
     String line; 
     while ((line = br.readLine()) != null) { 
     docs.add(new InsertOneModel<>(Document.parse(line))); 
     count++; 
     if (count == batch) { 
      collection.bulkWrite(docs, new BulkWriteOptions().ordered(false)); 
      docs.clear(); 
      count = 0; 
     } 
    } 
} 

if (count > 0) { 
    collection.bulkWrite(docs, new BulkWriteOptions().ordered(false)); 
} 

當你對整個JSON你基本上是通過覆蓋所有以前的的減少了文件最後文件運行Document.parse。

這裏更多

http://mongodb.github.io/mongo-java-driver/3.4/driver/tutorials/bulk-writes/