2017-09-04 41 views
0

我試圖請求多個subreddits的json頁面,並從大學項目的每個頁面的標題和鏈接。這裏是有問題的代碼:「未定義的方法[]」當分析的reddit api

require 'rufus-scheduler' 
require 'json' 
require 'httparty' 
ENV['TZ'] = 'Europe/Dublin' 

scheduler = Rufus::Scheduler::singleton 


scheduler.every '12h00m', :first_at => Time.now + 10 do 

array_of_subreddits = ["pics", "memes", "funny", "aww", "memes", 
"birdswitharms"] 
array_of_subreddits.each do |category| 
sleep 10 #wait 10 seconds between each request 

@response = JSON.parse(HTTParty.get("http://reddit.com/r/#{category}/.json?limit=25").body) 

    @response['data']['children'].each do |data| 
     @link = data['data']['url'] 
     @title = data['data']['title'] 
     @category = category 
     Pic.create([{:title => "#{@title}", :link => "#{@link}", :category => "#{@category}"}]) 
    end 
    end 
end 

這有效地完美有時,它會貫穿每一個和結束,因爲它應該。往往不是但是它會給我這個消息一分兩次之後:

NoMethodError (undefined method `[]' for nil:NilClass): 
 
    app/controllers/home_controller.rb:17:in `block in index' 
 
    app/controllers/home_controller.rb:9:in `each' 
 
    app/controllers/home_controller.rb:9:in `index' 
 

 

 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/_source.erb (4.8ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/_trace.html.erb (2.2ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/_request_and_response.html.erb (1.2ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/diagnostics.html.erb within rescues/layout (66.2ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/_markup.html.erb (0.4ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/_inner_console_markup.html.erb within layouts/inlined_string (0.3ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/_prompt_box_markup.html.erb within layouts/inlined_string (0.3ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/style.css.erb within layouts/inlined_string (0.5ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/console.js.erb within layouts/javascript (51.6ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/main.js.erb within layouts/javascript (0.3ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/error_page.js.erb within layouts/javascript (0.5ms) 
 
    Rendered /Users/conorbreen/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/web-console-2.3.0/lib/web_console/templates/index.html.erb (124.8ms)

+2

您正在HTTP調用。很可能您在Reddit上遇到速率限制或任何其他服務錯誤,因爲@response ['data']'爲零,這引發了上述異常。檢查'@response ['data']'和'@response ['data'] ['children']的http響應/存在,以使腳本高效。 – kiddorails

+1

不要盲目地認爲'HTTParty.get(...)'成功了。將其分配給一個變量'response = HTTParty.get(...)',並在處理其數據之前檢查'response.success?'或'response.code'。 – Stefan

回答

1

當你得到這樣的錯誤,你應該始終傾倒的實際響應,因此您可以檢查它。事實上,你得到了一個nil的錯誤,並且代碼執行諸如['data']['children']這樣的代碼意味着Id猜測你得到了一個JSON響應,但缺少其中一個首選項(例如['data']返回nil)。

不只是假設每個請求都是成功的,許多事情都可能導致HTTP失敗。它可能會返回一個有效的JSON響應,而不是您期望的響應,例如,一個錯誤消息,會告訴您問題

而且即使以10秒延遲,你可能會打一個速率限制(沒有測試過reddit的個人),但讀了rules

許多默認的用戶代理(如「巨蟒/ urllib的」或「 Java「)受到極大限制,以鼓勵獨特的描述性用戶代理字符串。

+0

這是問題的孩子。通過爲每個請求使用不同的用戶代理程序,問題得到解決。 我發現這個寶石相當有用: https://github.com/asconix/user-agent-randomizer 非常感謝Fire Lancer! –

+0

真的,你應該這樣做,他們問。如果你遇到問題,他們可能會採取更激烈的行動來對付你。 「更改您的客戶的用戶代理... **您的用戶名**作爲聯繫信息,格式如下: ':<版本號>(通過/ u/)'」。我不認爲它應該是隨機的,並且他們一眼就看不到它。 –

2

創建客戶端類是一種更好的方式與httparty工作:

class RedditClient 
    include HTTParty 

    format :json 

    base_uri "http://reddit.com/r/" 

    def self.get_category(category, *opts) 
    opts.reverse_merge(limit: 25) 
    get("/#{category}.json", opts) 
    end 
end 

這樣HTTParty處理JSON解析爲我們和不嘗試將轉換一個空的響應。它也更容易分開測試。

但是你還是應該檢查響應嘗試使用它之前是成功的:

@response = RedditClient.get_category(category) 
if @response.success? 
    attrs = @response['data']['children'].map do |child| 
    { 
     category: category, 
     link: child['data']['url'], 
     title: child['data']['title'] 
    } 
    end 
    Pic.create!(attrs) 
else 
    # log it or raise some sort of error 
end 

請注意,您通過包含一個散列.create陣列組成,其中。您可以改爲傳遞哈希數組,並將記錄插入到單個SQL插入語句中。

0

這種錯誤在ruby或rails中最常見。可以用多種方式處理。正如@Stefan建議你可以使用任何波紋管。

最簡單的這樣

response = HTTParty.get('http://reddit.com/r/#{category}/.json?limit=25') 
if response.success? 
    response_body = response.body 
    # continue 
end 

response = HTTParty.get('http://reddit.com/r/#{category}/.json?limit=25') 

case response.code 
    when 200 
    puts "Good!" 
    # Continue your parsing 
    when 404 
    puts "NOT FOUND!" 
    when 500...600 
    puts "ERROR #{response.code}" 
end 

begin 
    HTTParty.get('http://reddit.com/r/#{category}/.json?limit=25') 
rescue HTTParty::Error 
    # HTTParty errors like Not found 
rescue StandardError 
    # StandardError like Timeout 
else 
    # continue 
end