PHP 抓取並儲存網站圖片完整實作

實作一個 PHP 爬蟲腳本，自動抓取指定網頁的所有圖片並儲存到本地。涵蓋網頁內容獲取、正則解析圖片連結、相對路徑修正、去重、檔案下載儲存等完整流程。

核心程式碼

<?php
/**
 * 取得網頁所有圖片 URL
 */
function get_img_url($site_name) {
    $site_fd = fopen($site_name, "r");
    $site_content = "";
    while (!feof($site_fd)) {
        $site_content .= fread($site_fd, 1024);
    }
    fclose($site_fd);

    // 正則匹配 img 標籤的 src 屬性
    $reg_tag = '//i';
    preg_match_all($reg_tag, $site_content, $match_result);
    return $match_result[1];
}

/**
 * 修正相對路徑為絕對 URL
 */
function revise_site($site_list, $base_site) {
    $return_list = [];
    foreach ($site_list as $site_item) {
        if (preg_match('/^https?:\/\//i', $site_item)) {
            $return_list[] = $site_item;
        } else {
            // 簡易處理：移除 base_site 的檔名部分
            $base_dir = dirname($base_site);
            $return_list[] = $base_dir . "/" . ltrim($site_item, '/');
        }
    }
    return $return_list;
}

/**
 * 下載並儲存圖片
 */
function get_pic_file($pic_url_array, $pos = './') {
    $reg_tag = '/.*\/(.+?)$/';
    $count = 0;

    foreach ($pic_url_array as $pic_item) {
        preg_match_all($reg_tag, $pic_item, $t_pic_name);
        $pic_name = $pos . $t_pic_name[1][0];
        $pic_url = $pic_item;

        echo "Downloading {$pic_url} ... ";

        $img_read_fd = fopen($pic_url, "r");
        $img_write_fd = fopen($pic_name, "w");

        $img_content = "";
        while (!feof($img_read_fd)) {
            $img_content .= fread($img_read_fd, 1024);
        }

        fwrite($img_write_fd, $img_content);
        fclose($img_read_fd);
        fclose($img_write_fd);

        echo "[OK]\n";
        $count++;
    }
    return $count;
}

/**
 * 主程式
 */
function main() {
    $site_name = "https://www.example.com/gallery.html";

    // 1. 取得圖片 URL
    $img_url = get_img_url($site_name);

    // 2. 修正相對路徑
    $img_url_revised = revise_site($img_url, $site_name);

    // 3. 去重
    $img_url_unique = array_unique($img_url_revised);

    // 4. 下載儲存
    get_pic_file($img_url_unique, "./images/");
}

main();
?>

已知問題與改進方向

改進版：使用 cURL + DOMDocument

<?php
function fetch_images_advanced($url, $save_dir = './images/') {
    if (!is_dir($save_dir)) mkdir($save_dir, 0755, true);

    $ch = curl_init($url);
    curl_setopt_array($ch, [
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_FOLLOWLOCATION => true,
        CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        CURLOPT_TIMEOUT => 30,
        CURLOPT_SSL_VERIFYPEER => false,
    ]);
    $html = curl_exec($ch);
    curl_close($ch);

    if (!$html) return ['error' => '無法取得網頁內容'];

    $dom = new DOMDocument();
    @$dom->loadHTML($html);
    $xpath = new DOMXPath($dom);
    $imgs = $xpath->query('//img/@src');

    $downloaded = 0;
    foreach ($imgs as $img) {
        $src = $img->nodeValue;
        $abs_url = resolve_url($src, $url);
        $ext = strtolower(pathinfo(parse_url($abs_url, PHP_URL_PATH), PATHINFO_EXTENSION));
        if (!in_array($ext, ['jpg','jpeg','png','gif','webp','bmp'])) continue;

        $filename = $save_dir . uniqid('img_') . ".{$ext}";
        if (download_file($abs_url, $filename)) {
            $downloaded++;
        }
        usleep(200000); // 0.2s 延遲
    }
    return ['downloaded' => $downloaded];
}

function resolve_url($src, $base) {
    if (preg_match('/^https?:\/\//i', $src)) return $src;
    $base_parts = parse_url($base);
    $base_dir = dirname($base_parts['path']);
    return $base_parts['scheme'] . '://' . $base_parts['host'] . $base_dir . '/' . ltrim($src, '/');
}

function download_file($url, $path) {
    $ch = curl_init($url);
    $fp = fopen($path, 'w');
    curl_setopt_array($ch, [
        CURLOPT_FILE => $fp,
        CURLOPT_FOLLOWLOCATION => true,
        CURLOPT_TIMEOUT => 60,
        CURLOPT_USERAGENT => 'Mozilla/5.0',
    ]);
    $result = curl_exec($ch);
    $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);
    fclose($fp);
    return $result && $http_code === 200;
}
?>

總結：基本版適合簡單靜態頁面快速抓取；生產環境建議使用改進版，並加入佇列系統（如 Laravel Queue）、資料庫記錄、增量爬取等機制。