故障解析JSON與mongoimport

我得到一個Assertion: 10340:Failure parsing JSON string錯誤，在運行中管mongoimport在Github的API，如下所示：故障解析JSON與mongoimport

[email protected]:~/rails/github/gitwatcher$ curl https://api.github.com/users/lgs/repos | mongoimport -h localhost -d gitwatch_dev -c repo -f repositories 
connected to: localhost 
% Total % Received % Xferd Average Speed Time Time  Time Current 
           Dload Upload Total Spent Left Speed 
0  0 0  0 0  0  0  0 --:--:-- --:--:-- --:--:--  0Mon Jun 20 00:56:01 Assertion: 10340:Failure parsing JSON string near: [ 
100 22303 100 22303 0  0 31104  0 --:--:-- --:--:-- --:--:-- 111k 
0x816d8a1 0x8118814 0x84b357a 0x84b5bb8 0x84adc65 0x84b2ee1 0x60bbd6 0x80f5bc1 
mongoimport(_ZN5mongo11msgassertedEiPKc+0x221) [0x816d8a1] 
mongoimport(_ZN5mongo8fromjsonEPKcPi+0x3b4) [0x8118814] 
mongoimport(_ZN6Import9parseLineEPc+0x7a) [0x84b357a] 
mongoimport(_ZN6Import3runEv+0x1a98) [0x84b5bb8] 
mongoimport(_ZN5mongo4Tool4mainEiPPc+0x1ce5) [0x84adc65] 
mongoimport(main+0x51) [0x84b2ee1] 
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0x60bbd6] 
mongoimport(__gxx_personality_v0+0x3f1) [0x80f5bc1] 
exception:Failure parsing JSON string near: [ 
[ 
... 
... 
Mon Jun 20 00:45:20 Assertion: 10340:Failure parsing JSON string near: "name": "t 
0x816d8a1 0x8118814 0x84b357a 0x84b5bb8 0x84adc65 0x84b2ee1 0x126bd6 0x80f5bc1 
mongoimport(_ZN5mongo11msgassertedEiPKc+0x221) [0x816d8a1] 
mongoimport(_ZN5mongo8fromjsonEPKcPi+0x3b4) [0x8118814] 
mongoimport(_ZN6Import9parseLineEPc+0x7a) [0x84b357a] 
mongoimport(_ZN6Import3runEv+0x1a98) [0x84b5bb8] 
mongoimport(_ZN5mongo4Tool4mainEiPPc+0x1ce5) [0x84adc65] 
mongoimport(main+0x51) [0x84b2ee1] 
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0x126bd6] 
mongoimport(__gxx_personality_v0+0x3f1) [0x80f5bc1] 
exception:Failure parsing JSON string near: "name": "t 
"name": "tentacles" 
... 
...

這裏看到完整的跟蹤：http://pastie.org/2093486。無論如何，JSON格式我回來從Github的API似乎確定（捲曲https://api.github.com/users/lgs/repos）：

[ 
{ 
    "open_issues": 0, 
    "watchers": 3, 
    "homepage": "http://scrubyt.org", 
    "language": null, 
    "forks": 1, 
    "pushed_at": "2009-02-25T22:49:08Z", 
    "created_at": "2009-02-25T22:22:40Z", 
    "fork": true, 
    "url": "https://api.github.com/repos/lgs/scrubyt", 
    "private": false, 
    "size": 188, 
    "description": "A simple to learn and use, yet powerful web scraping toolkit!", 
    "owner": { 
    "avatar_url": "https://secure.gravatar.com/avatar/9c7d80ebc20ab8994e51b9f7518909ae?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2 
Fgravatar-140.png", 
    "login": "lgs", 
    "url": "https://api.github.com/users/lgs", 
    "id": 1573 
    }, 
    "name": "scrubyt", 
    "html_url": "https://github.com/lgs/scrubyt" 
}, 
... 
... 
]

這裏是一個片段：http://www.pastie.org/2093524。

如果我嘗試指定CSV格式它的工作原理：

[email protected]:~/rails/github/gitwatcher$ curl https://api.github.com/users/lgs/repos | mongoimport -h localhost -d gitwatch_dev -c repo -f repositories --type csv 
connected to: localhost 
    % Total % Received % Xferd Average Speed Time Time  Time Current 
           Dload Upload Total Spent Left Speed 
100 22303 100 22303 0  0 23914  0 --:--:-- --:--:-- --:--:-- 106k 
imported 640 objects 
[email protected]:~/rails/github/gitwatcher$

來源

2011-06-19 Luca G. Soave

請勿使用像pastebin這樣的易失性服務來引用SO帖子中的代碼/數據。 –

...不穩定？ ... pastebin？以前的代碼鏈接不是pastebin也不是易變的;-)感謝downvoting！ –

我可以在1.8.1 64b上覆制它。研究可能發生的事情。我將輸出粘貼到JSON驗證器中，並且JSON有效。 – lobster1234

好吧，這裏是可以回事。首先，我刪除了JSON中的所有換行符，以減少從n（其中n =行數）到1的錯誤數量。然後，事實證明，我必須將JSON數組包裝到另一個變量中，並且此後它才起作用。我認爲mongoimport被設計爲與mongoexport一起工作，所以很可能你不能用它來導入任意的JSON。但是，如果你願意，在調用導入實用程序之前，我所做的是你在代碼中必須做的事情。

我只用1個紀錄，而我測試。這裏是沒有換行符的記錄。

[{"url":"https://api.github.com/repos/lgs/scrubyt", "pushed_at": "2009-02-25T22:49:08Z","homepage": "http://scrubyt.org", "forks": 1,"language": null,"fork": true,"html_url": "https://github.com/lgs/scrubyt","created_at": "2009-02-25T22:22:40Z", "open_issues": 0,"private": false,"size": 188,"watchers": 3,"owner": {"url": "https://api.github.com/users/lgs","login": "lgs","id": 1573,"avatar_url": "https://secure.gravatar.com/avatar/9c7d80ebc20ab8994e51b9f7518909ae?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png"},"name": "scrubyt","description": "A simple to learn and use, yet powerful web scraping toolkit!"}]

然後，我somedata包裹它（你可以使用這裏的任何名稱）：

{somedata:[{"url":"https://api.github.com/repos/lgs/scrubyt", "pushed_at": "2009-02-25T22:49:08Z","homepage": "http://scrubyt.org", "forks": 1,"language": null,"fork": true,"html_url": "https://github.com/lgs/scrubyt","created_at": "2009-02-25T22:22:40Z", "open_issues": 0,"private": false,"size": 188,"watchers": 3,"owner": {"url": "https://api.github.com/users/lgs","login": "lgs","id": 1573,"avatar_url": "https://secure.gravatar.com/avatar/9c7d80ebc20ab8994e51b9f7518909ae?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png"},"name": "scrubyt","description": "A simple to learn and use, yet powerful web scraping toolkit!"}]}

，我能夠看到蒙戈記錄。

> db.repo.findOne() 
{ 
    "_id" : ObjectId("4dff91d29c73f72483e82ef2"), 
    "somedata" : [ 
     { 
      "url" : "https://api.github.com/repos/lgs/scrubyt", 
      "pushed_at" : "2009-02-25T22:49:08Z", 
      "homepage" : "http://scrubyt.org", 
      "forks" : 1, 
      "language" : null, 
      "fork" : true, 
      "html_url" : "https://github.com/lgs/scrubyt", 
      "created_at" : "2009-02-25T22:22:40Z", 
      "open_issues" : 0, 
      "private" : false, 
      "size" : 188, 
      "watchers" : 3, 
      "owner" : { 
       "url" : "https://api.github.com/users/lgs", 
       "login" : "lgs", 
       "id" : 1573, 
       "avatar_url" : "https://secure.gravatar.com/avatar/9c7d80ebc20ab8994e51b9f7518909ae?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png" 
      }, 
      "name" : "scrubyt", 
      "description" : "A simple to learn and use, yet powerful web scraping toolkit!" 
     } 
    ] 
}

希望這有助於！

來源

2011-06-20 18:37:54 lobster1234

這真的很棒。很酷的故障排除非常感謝。 –

它爲我用「mongoimport --jsonArray ...」

來源

2012-01-20 11:10:10

謝謝！我的頭撞了一個小時後，爲什麼它說我的JSON無效。 – occasl

是的，這工作得很好，即使在那裏有新的字符。 –

...這是偉大的，但 –

這個工作我沒意見之後，我刪除任何「\ n」。你可以在linux中使用trr cat file.json | TR -d「\ n」> file.json

來源

2012-10-04 06:20:19 Paul

同時使用由@Daniel和@ lobster1234提供的答案我創建了我使用的JSON條目導入蒙戈的腳本。

#!/bin/sh 

if [ -z "$1" ] ; 
then 
    echo "missing argument" 
    exit -1 
fi 

FILE=${1%%.json} 

echo $FILE 

cat $FILE.json | tr -d '\n' > $FILE.import.json 

mongoimport --collection collection --db main --file $FILE.import.json --jsonArray --upsert

來源

2013-02-24 13:39:28

故障解析JSON與mongoimport

回答

相關問題