PHP cURL 多執行緒並發下載：curl_multi 批次抓取遠端檔案完整實作

應用場景

需同時下載多個遠端圖片/檔案（如商品圖批次抓取、站群內容同步），使用 curl_multi 非同步並發可大幅縮短總耗時。

核心函式 remote()

<?php
/**
 * 多執行緒並發下載遠端檔案
 *
 * @param array  $urls  遠端檔案 URL 陣列
 * @param string $name  儲存檔名基礎名稱（不含副檔名）
 * @param string $path  回傳路徑前綴（如 /uploads/）
 * @param string $dir   本地儲存目錄
 * @return array|false  成功儲存的相對路徑陣列，失敗回傳 false
 */
function remote($urls, $name = '', $path = '', $dir = './images/') {
    if (!is_array($urls) || count($urls) === 0) return false;
    
    // 確保目錄存在
    if (!is_dir($dir)) {
        mkdir($dir, 0755, true);
    }
    
    $curl = $nurl = [];
    $handle = curl_multi_init();
    
    // 初始化各 cURL handle
    foreach ($urls as $k => $v) {
        if (empty($v) || !preg_match('~^https?~i', $v)) continue;
        
        $nurl[$k] = trim(str_replace(' ', '%20', $v));
        $curl[$k] = curl_init($nurl[$k]);
        
        curl_setopt_array($curl[$k], [
            CURLOPT_IPRESOLVE      => CURL_IPRESOLVE_V4,
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_HEADER         => false,
            CURLOPT_CONNECTTIMEOUT => 20,
            CURLOPT_TIMEOUT        => 60,
            CURLOPT_FOLLOWLOCATION => true,
            CURLOPT_MAXREDIRS      => 5,
            CURLOPT_USERAGENT      => 'Mozilla/5.0 (compatible; PHP Multi Downloader)',
            CURLOPT_SSL_VERIFYPEER => false, // 僅測試
        ]);
        
        curl_multi_add_handle($handle, $curl[$k]);
    }
    
    // 並發執行主迴圈
    $active = null;
    do {
        $mrc = curl_multi_exec($handle, $active);
    } while ($mrc === CURLM_CALL_MULTI_PERFORM);
    
    while ($active && $mrc === CURLM_OK) {
        if (curl_multi_select($handle) !== -1) {
            do {
                $mrc = curl_multi_exec($handle, $active);
            } while ($mrc === CURLM_CALL_MULTI_PERFORM);
        }
    }
    
    // 收集結果並儲存
    $filepath = [];
    foreach ($curl as $k => $v) {
        if (curl_error($curl[$k]) === "") {
            // 產生檔名：第一個用 base name，其餘加索引
            $ext = strtolower(pathinfo($urls[$k], PATHINFO_EXTENSION));
            $ext = $ext ? ".$ext" : ".bin"; // 無副檔名預設 .bin
            
            if ($k === 0 || $k === array_key_first($curl)) {
                $fname = strtolower($name . $ext);
            } else {
                $fname = strtolower($name . '_' . $k . $ext);
            }
            
            $content = curl_multi_getcontent($curl[$k]);
            $filedir = $dir . $fname;
            
            if (file_put_contents($filedir, $content) !== false) {
                $filepath[$k] = $path . $fname;
            }
        } else {
            error_log("cURL Error [{$k}]: " . curl_error($curl[$k]) . " - URL: " . $nurl[$k]);
        }
        
        curl_multi_remove_handle($handle, $curl[$k]);
        curl_close($curl[$k]);
    }
    
    curl_multi_close($handle);
    return $filepath;
}
?>

使用範例

<?php
$imageUrls = [
    'https://example.com/img/photo1.jpg',
    'https://example.com/img/photo2.png',
    'https://example.com/img/photo3.gif',
    'https://example.com/img/photo4.webp',
];

$result = remote($imageUrls, 'product_123', '/uploads/products/', './storage/products/');

if ($result) {
    echo "下載完成：\n";
    print_r($result);
    /* 輸出範例：
    Array
    (
        [0] => /uploads/products/product_123.jpg
        [1] => /uploads/products/product_123_1.png
        [2] => /uploads/products/product_123_2.gif
        [3] => /uploads/products/product_123_3.webp
    )
    */
} else {
    echo "下載失敗或無有效 URL";
}
?>

效能優化建議

大檔案串流寫入版（不佔記憶體）

<?php
function remoteStream($urls, $dir = './downloads/') {
    if (!is_dir($dir)) mkdir($dir, 0755, true);
    $handle = curl_multi_init();
    $fp = $curl = [];
    
    foreach ($urls as $k => $url) {
        $fname = $dir . basename($url);
        $fp[$k] = fopen($fname, 'wb');
        
        $curl[$k] = curl_init($url);
        curl_setopt_array($curl[$k], [
            CURLOPT_FILE           => $fp[$k],        // 直接寫入檔案指標
            CURLOPT_HEADER         => false,
            CURLOPT_FOLLOWLOCATION => true,
            CURLOPT_CONNECTTIMEOUT => 30,
            CURLOPT_TIMEOUT        => 300,
        ]);
        curl_multi_add_handle($handle, $curl[$k]);
    }
    
    // ... 同前執行迴圈 ...
    
    foreach ($curl as $k => $v) {
        curl_multi_remove_handle($handle, $curl[$k]);
        curl_close($curl[$k]);
        fclose($fp[$k]);
    }
    curl_multi_close($handle);
}
?>

錯誤處理與重試機制

生產環境建議加入：

• 失敗 URL 記錄並重試（指數退避）
• HTTP 狀態碼檢查（curl_getinfo($ch, CURLINFO_HTTP_CODE)）
• Content-Type 驗證確保下載的是預期檔案類型
• 檔案完整性校驗（MD5/SHA256）