速度限制你自己的過載外部API的

Question

我發現了很多信息和腳本示例，顯示瞭如何限制API的用戶速度，但我無法找到任何示例如何限制您自己的請求一個API，當這些限制被強加。

我一直速率限制我的腳本的代碼，如sleep或usleep命令，但感覺像做事情的方式效率極低，尤其是當API端點具有相當高的速率限制和錘擊API的，直到你遇到限度，也效率低下。

例如，Google的API限制根據您使用的API而有所不同，並且可以增加/減少，在這種情況下，硬編碼到代碼中的固定費率限制看起來像是原始猜測工作！

我錯過了非常明顯的東西嗎？或者這只是不如我預期的那麼普遍？

Answer 1

好吧，對於咯咯我一起扔了一個限制類，可以讓你指定每秒，每分鐘和每小時的限制。我無法拒絕有一個很好的理由來使用循環隊列！

如果您有多個進程正在進行消耗，無論是否同時進行，您都必須設計一種方法來自行存儲和/或共享使用歷史記錄。

// LIMITER.PHP 
class Limiter 
{ 
    private $queue = array(); 
    private $size; 
    private $next; 

    private $perSecond; 
    private $perMinute; 
    private $perHour; 

    // Set any constructor parameter to non-zero to allow adherence to the 
    // limit represented. The largest value present will be the size of a 
    // circular queue used to track usage. 
    // ------------------------------------------------------------------- 
    function __construct($perSecond=0,$perMinute=0,$perHour=0) 
    { 
    $this->size = max($perSecond,$perMinute,$perHour); 
    $this->next = 0; 

    $this->perSecond = $perSecond; 
    $this->perMinute = $perMinute; 
    $this->perHour = $perHour; 

    for($i=0; $i < $this->size; $i++) 
     $this->queue[$i] = 0; 
    } 

    // See if a use would violate any of the limits specified. We return true 
    // if a limit has been hit. 
    // ---------------------------------------------------------------------- 
    public function limitHit($verbose=0) 
    {  
    $inSecond = 0; 
    $inMinute = 0; 
    $inHour = 0; 

    $doneSecond = 0; 
    $doneMinute = 0; 
    $doneHour = 0; 

    $now = microtime(true); 

    if ($verbose) 
     echo "Checking if limitHit at $now<br>\n"; 

    for ($offset=1; $offset <= $this->size; $offset++) 
    { 
     $spot = $this->next - $offset; 
     if ($spot < 0) 
     $spot = $this->size - $offset + $this->next; 

     if ($verbose) 
     echo "... next $this->next size $this->size offset $offset spot $spot utime " . $this->queue[$spot] . "<br>\n"; 

     // Count and track within second 
     // ----------------------------- 
     if ($this->perSecond && !$doneSecond && $this->queue[$spot] >= microtime(true) - 1.0) 
     $inSecond++; 
     else 
     $doneSecond = 1; 

     // Count and track within minute 
     // ----------------------------- 
     if ($this->perMinute && !$doneMinute && $this->queue[$spot] >= microtime(true) - 60.0) 
     $inMinute++; 
     else 
     $doneMinute = 1; 

     // Count and track within hour 
     // --------------------------- 
     if ($this->perHour && !$doneHour && $this->queue[$spot] >= microtime(true) - 3600.0) 
     $inHour++; 
     else 
     $doneHour = 1; 

     if ($doneSecond && $doneMinute && $doneHour) 
     break; 
    } 

    if ($verbose) 
     echo "... inSecond $inSecond inMinute $inMinute inHour $inHour<br>\n"; 

    if ($inSecond && $inSecond >= $this->perSecond) 
    { 
     if ($verbose) 
     echo "... limit perSecond hit<br>\n"; 
     return TRUE; 
    } 
    if ($inMinute && $inMinute >= $this->perMinute) 
    { 
     if ($verbose) 
     echo "... limit perMinute hit<br>\n"; 
     return TRUE; 
    } 
    if ($inHour && $inHour >= $this->perHour ) 
    { 
     if ($verbose) 
     echo "... limit perHour hit<br>\n"; 
     return TRUE; 
    } 

    return FALSE; 
    } 

    // When an API is called the using program should voluntarily track usage 
    // via the use function. 
    // ---------------------------------------------------------------------- 
    public function usage() 
    { 
    $this->queue[$this->next++] = microtime(true); 
    if ($this->next >= $this->size) 
     $this->next = 0; 
    } 
} 

// ############################## 
// ### Test the limiter class ### 
// ############################## 

$psec = 2; 
$pmin = 4; 
$phr = 0; 

echo "Creating limiter with limits of $psec/sec and $pmin/min and $phr/hr<br><br>\n"; 
$monitorA = new Limiter($psec,$pmin,$phr); 

for ($i=0; $i<15; $i++) 
{ 
    if (!$monitorA->limitHit(1)) 
    { 
    echo "<br>\n"; 
    echo "API call A here (utime " . microtime(true) . ")<br>\n"; 
    echo "Voluntarily registering usage<br>\n"; 
    $monitorA->usage(); 
    usleep(250000); 
    } 
    else 
    { 
    echo "<br>\n"; 
    usleep(500000); 
    } 
} 

爲了證明它在行動，我已經把在極限檢查功能有些「詳細模式」的語句。這裏是一些示例輸出。

Creating limiter with limits of 2/sec and 4/min and 0/hr 

Checking if limitHit at 1436267440.9957 
... next 0 size 4 offset 1 spot 3 utime 0 
... inSecond 0 inMinute 0 inHour 0 

API call A here (utime 1436267440.9957) 
Voluntarily registering usage 
Checking if limitHit at 1436267441.2497 
... next 1 size 4 offset 1 spot 0 utime 1436267440.9957 
... next 1 size 4 offset 2 spot 3 utime 0 
... inSecond 1 inMinute 1 inHour 0 

API call A here (utime 1436267441.2497) 
Voluntarily registering usage 
Checking if limitHit at 1436267441.5007 
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497 
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957 
... next 2 size 4 offset 3 spot 3 utime 0 
... inSecond 2 inMinute 2 inHour 0 
... limit perSecond hit 

Checking if limitHit at 1436267442.0007 
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497 
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957 
... next 2 size 4 offset 3 spot 3 utime 0 
... inSecond 1 inMinute 2 inHour 0 

API call A here (utime 1436267442.0007) 
Voluntarily registering usage 
Checking if limitHit at 1436267442.2507 
... next 3 size 4 offset 1 spot 2 utime 1436267442.0007 
... next 3 size 4 offset 2 spot 1 utime 1436267441.2497 
... next 3 size 4 offset 3 spot 0 utime 1436267440.9957 
... next 3 size 4 offset 4 spot 3 utime 0 
... inSecond 1 inMinute 3 inHour 0 

API call A here (utime 1436267442.2507) 
Voluntarily registering usage 
Checking if limitHit at 1436267442.5007 
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507 
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007 
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497 
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957 
... inSecond 2 inMinute 4 inHour 0 
... limit perSecond hit 

Checking if limitHit at 1436267443.0007 
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507 
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007 
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497 
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957 
... inSecond 2 inMinute 4 inHour 0 
... limit perSecond hit 

Checking if limitHit at 1436267443.5027 
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507 
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007 
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497 
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957 
... inSecond 0 inMinute 4 inHour 0 
... limit perMinute hit 

Checking if limitHit at 1436267444.0027 
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507 
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007 
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497 
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957 
... inSecond 0 inMinute 4 inHour 0 
... limit perMinute hit 

Answer 2

嗯，首先要做的是 - 只有在實際需要時才應調用任何外部API - 供應商將非常感謝您。

有兩種方法我通常會對自己的API使用量施加限制 - 如果可能的話，將結果緩存N個時間量，通常比API本身的硬性限制要少很多。但是，這隻適用於非常特殊的情況。

第二個是持久性/半持久性計數器，其中您將計數器存儲在某種內存後端以及限制期開始時的時間。每次在調用API之前檢查存儲並查看當前時間減去間隔是否開始以及您已經做出的請求數量是否小於API施加的請求數量。如果是，可以提出請求 - 如果時間間隔較大，可以重新設置限制，並且如果下一個請求將超過限制，並且您仍處於上一個時間間隔內，則可能會顯示出錯誤。在每個外部請求中，如果超過了間隔時間，則更新間隔時間並增加計數器。

Answer 3

我想我們不能用幾句話來回答你的問題。它需要真實反映與您的應用程序相關的架構。對於我來說，重複使用API​​速率限制的方法是使用存儲我的API的值和利用率的高速緩存。我必須找到沒有準備好的代碼。