1
我有cURL的問題。我已經使用curl_multi
啓用流水線(服務器支持它)。但是我有超過10000個單一網址,我需要下載。我沒有那麼多免費插座。如何處理這個?cURL - 多 - 太多開放式套接字
我看了一下curl_multi_socket_action
,但我不能找到一些簡單的例子,或如何修改我的代碼
CURLM * curlm = curl_multi_init();
curl_multi_setopt(curlm, CURLMOPT_MAXCONNECTS, 3);
curl_multi_setopt(curlm, CURLMOPT_PIPELINING, 1);
std::vector<CURL_WRITE *> ptrArray;
std::map<MyStringAnsi, MyStringAnsi>::const_iterator it;
for (it = data[0].begin(); it != data[0].end(); it++)
{
CURL_WRITE * curlData = new CURL_WRITE;
curlData->dataIndex = &(it->first);
curlData->threadID = 0;
CURL * curl = InitSingleCURL(it->first, curlData);
curl_multi_add_handle(curlm, curl);
ptrArray.push_back(curlData);
curls.push_back(curl);
}
int still_running = 0;
/* lets start the fetch */
do
{
while(curl_multi_perform(curlm, &still_running) == CURLM_CALL_MULTI_PERFORM);
#ifdef _WIN32
Sleep(10);
#else
usleep(10);
#endif
} while (still_running);