同步一個方法不同的Sidekiq線程並等待

Question

問題： 我有幾個sidekiq線程和一個函數，只能從任何線程調用一次。

原因： 我們正在查詢AdWords API以獲取一些數據。當涉及到費率限制時，它們的限制性很強。只有一個線程可能會調用該函數來一次獲取數據。

現在一些代碼：

# Public: Get estimates for a set of keywords. If there is an error, retry 
# several times. If not successful, raise an error 
# 
# keywords: The keyword objects to get estimates for. 
# save: Boolean to indicate whether the keyword objects should be saved to 
# the database 
# 
def repeatedly_try_get_estimates(keywords: [], save: true, sleep_delay: 150) 
    return keywords if keywords.empty? 
    func = -> { get_estimates(keywords, !save) } 
    retry_operation(function: func, max_tries: 15, sleep_delay: sleep_delay) 
end 

• 正如你所看到的，現在我有一個巨大的sleep_delay解決該問題。
• 代碼調用retry_operation函數， get_estimates函數作爲參數。然後它將重試 get_estimates函數幾次，直到出現API 異常。

的retry_function：

# Private: Retry a function X times and wait X seconds. If it does not work X times, 
# raise an error. If successful return the functions results. 
# 
# - max_tries: The maximum tries to repeat the function 
# - sleep_delay: The seconds to wait between each iteration. 
# - function: The lambda function to call each iteration 
# 
def retry_operation(max_tries: 5, sleep_delay: 30, function: nil, current_try: 0, result: nil) 

    # Can't call, no function 
    if function.nil? 
    return 
    end 

    # Abort, tried too frequently. 
    if current_try > max_tries 
    raise "Failed function too often" 
    end 

    # Check if there is an exception 
    exception = true 
    begin 
    result = function.call 
    exception = false 
    rescue => e 
    Rails.logger.info "Received error when repeatedly calling function #{e.message.to_s}" 
    end 

    if exception 
    sleep sleep_delay if sleep_delay > 0 
    retry_operation(max_tries: max_tries, sleep_delay: sleep_delay, function: function, current_try: current_try + 1) 
    else 
    result 
    end 
end 

的get_estimates_function是在這裏：https://gist.github.com/a14868d939ef0e34ef9f。這太長了，以防萬一。

我想我需要做到以下幾點：

1. 調整代碼在repeatedly_try_get_estimates功能。
2. 在課堂上使用互斥鎖。
3. 如果正在使用互斥鎖，請搶救異常。
4. 只有當互斥是免費的，運行rety_operation，否則睡眠一段時間

感謝您的幫助:)

Answer 1

在這裏，我們去，得到它的工作：

# Public: Get estimates for a set of keywords. If there is an error, retry 
# several times. If not successful, raise an error 
# 
# keywords: The keyword objects to get estimates for. 
# save: Boolean to indicate whether the keyword objects should be saved to 
# the database 
# 
def repeatedly_try_get_estimates(keywords: [], save: true, sleep_delay: 40) 
    return keywords if keywords.empty? 
    func = -> { get_estimates(keywords, save_keywords: true) } 
    exception = nil 
    result = nil 
    initial_sleep = 0 

    estimates_mutex.synchronize do 
    since_last_request = Time.now.to_i - last_adwords_api_request 
    if since_last_request <= 30 
     Rails.logger.info "AdWords: Last request was only few seconds ago - sleeping #{since_last_request}." 
     initial_sleep = since_last_request 
    end 
    begin 
     result = retry_operation(function: func, max_tries: 15, sleep_delay: sleep_delay, initial_sleep: initial_sleep) 
    rescue => e 
     exception = e 
    end 
    @@last_adwords_api_request = Time.now.to_i 
    end 
    if exception 
    raise exception 
    end 
    result 
end