Java 爬蟲實戰：基於正則表達式批量抓取網頁圖片完整代碼實現

核心思路：兩步正則提取

1. 匹配 <img> 標籤：<img.*src=(.*?)[^>]*?>
2. 從標籤提取 src 路徑：http:\\\"?(.*?)(\"|>|\\s+)

完整實現類 CatchImage

public class CatchImage {
    private static final String URL = "https://www.csdn.net";
    private static final String ENCODING = "UTF-8";
    private static final String IMG_TAG_REGEX = "<img.*src=(.*?)[^>]*?>";
    private static final String IMG_SRC_REGEX = "http:\\\\\"?(.*?)(\"|>|\\\\s+)";

    public static void main(String[] args) throws Exception {
        CatchImage cm = new CatchImage();
        String html = cm.getHTML(URL);
        List<String> imgTags = cm.getImageUrl(html);
        List<String> imgSrcs = cm.getImageSrc(imgTags);
        cm.download(imgSrcs);
    }

    private String getHTML(String url) throws Exception {
        URLConnection conn = new URL(url).openConnection();
        try (InputStream in = conn.getInputStream()) {
            byte[] buf = new byte[1024];
            StringBuilder sb = new StringBuilder();
            int len;
            while ((len = in.read(buf)) > 0) {
                sb.append(new String(buf, 0, len, ENCODING));
            }
            return sb.toString();
        }
    }

    private List<String> getImageUrl(String html) {
        List<String> list = new ArrayList<>();
        Matcher m = Pattern.compile(IMG_TAG_REGEX).matcher(html);
        while (m.find()) list.add(m.group());
        return list;
    }

    private List<String> getImageSrc(List<String> imgTags) {
        List<String> list = new ArrayList<>();
        Matcher m = Pattern.compile(IMG_SRC_REGEX);
        for (String tag : imgTags) {
            m.reset(tag);
            while (m.find()) {
                String src = m.group(1);  // 直接取捕獲組，避免 substring 易錯
                list.add(src);
            }
        }
        return list;
    }

    private void download(List<String> srcs) {
        for (String url : srcs) {
            String fileName = url.substring(url.lastIndexOf('/') + 1);
            try (InputStream in = new URL(url).openStream();
                 FileOutputStream out = new FileOutputStream(fileName)) {
                byte[] buf = new byte[1024];
                int len;
                System.out.println("下載中：" + url);
                while ((len = in.read(buf)) != -1) out.write(buf, 0, len);
                System.out.println(fileName + " 完成");
            } catch (Exception e) {
                System.err.println("失敗：" + url);
            }
        }
    }
}

關鍵優化點

• 使用 Matcher.reset(CharSequence) 復用 Matcher，避免重複編譯 Pattern
• 直接提取捕獲組 group(1) 而非 substring，更健壯
• try-with-resources 確保流關閉，防止句柄泄漏

局限性與進階建議

1. 正則解析 HTML 脆弱：頁面結構變動即失效，生產建議用 Jsoup / HttpClient + XPath
2. 動態渲染頁面無效：JS 生成的圖片需 Selenium / Puppeteer 等無頭瀏覽器
3. 反爬蟲應對：加請求頭、代理 IP、限流、驗證碼識別
4. 合規性：遵守 robots.txt、版權法、網站服務條款