C# 高效驗證代理 IP 可用性：HttpWebRequest + 異步並發批量檢測

場景：代理池維護、爬蟲前置檢測、隱私瀏覽器插件

手動在 IE 設定代理、刷新 whatismyip.com 驗證極其低效。程式化驗證核心邏輯：

1. 建立 WebProxy 物件；
2. HttpWebRequest.Proxy = proxy；
3. 請求驗證網站（如 https://httpbin.org/ip、https://api.ipify.org）；
4. 解析回應 JSON 中的 origin 或 ip 欄位，若與代理 IP 一致則判定可用；
5. 記錄延遲、匿名度（透過檢測 Via、X-Forwarded-For 標頭）。

基礎同步驗證（原文改良版）

public class ProxyValidator {
    private static readonly HttpClient _httpClient = new HttpClient { Timeout = TimeSpan.FromSeconds(10) };

    public static async Task CheckAsync(string ip, int port, string testUrl = "https://httpbin.org/ip") {
        var proxy = new WebProxy(ip, port);
        using var handler = new HttpClientHandler { Proxy = proxy, UseProxy = true };
        using var client = new HttpClient(handler) { Timeout = TimeSpan.FromSeconds(10) };

        var sw = Stopwatch.StartNew();
        try {
            string json = await client.GetStringAsync(testUrl);
            sw.Stop();
            // 解析 JSON: {"origin": "x.x.x.x"}
            using var doc = JsonDocument.Parse(json);
            string origin = doc.RootElement.GetProperty("origin").GetString();
            bool success = origin?.Contains(ip) == true;
            return new ProxyCheckResult {
                Ip = ip, Port = port, Success = success, LatencyMs = (int)sw.ElapsedMilliseconds,
                ResponseIp = origin, CheckTime = DateTime.Now
            };
        }
        catch (Exception ex) {
            sw.Stop();
            return new ProxyCheckResult {
                Ip = ip, Port = port, Success = false, LatencyMs = (int)sw.ElapsedMilliseconds,
                Error = ex.Message, CheckTime = DateTime.Now
            };
        }
    }
}

public class ProxyCheckResult {
    public string Ip { get; set; }
    public int Port { get; set; }
    public bool Success { get; set; }
    public int LatencyMs { get; set; }
    public string ResponseIp { get; set; }
    public string Error { get; set; }
    public DateTime CheckTime { get; set; }
}

批量異步並發驗證（生產級）

public class BulkProxyValidator {
    private readonly SemaphoreSlim _semaphore;
    private readonly HttpClient _client;

    public BulkProxyValidator(int maxConcurrency = 50) {
        _semaphore = new SemaphoreSlim(maxConcurrency);
        _client = new HttpClient(new HttpClientHandler { UseProxy = true }) { Timeout = TimeSpan.FromSeconds(8) };
    }

    public async Task> ValidateAsync(IEnumerable<(string ip, int port)> proxies, string testUrl = "https://httpbin.org/ip") {
        var tasks = proxies.Select(async p => {
            await _semaphore.WaitAsync();
            try {
                return await CheckSingleAsync(p.ip, p.port, testUrl);
            }
            finally { _semaphore.Release(); }
        });
        return (await Task.WhenAll(tasks)).Where(r => r.Success).OrderBy(r => r.LatencyMs).ToList();
    }

    private async Task CheckSingleAsync(string ip, int port, string testUrl) {
        var proxy = new WebProxy(ip, port);
        var handler = new HttpClientHandler { Proxy = proxy, UseProxy = true };
        using var client = new HttpClient(handler) { Timeout = TimeSpan.FromSeconds(8) };
        var sw = Stopwatch.StartNew();
        try {
            string json = await client.GetStringAsync(testUrl);
            sw.Stop();
            using var doc = JsonDocument.Parse(json);
            string origin = doc.RootElement.GetProperty("origin").GetString();
            return new ProxyCheckResult {
                Ip = ip, Port = port, Success = origin?.Contains(ip) == true,
                LatencyMs = (int)sw.ElapsedMilliseconds, ResponseIp = origin
            };
        }
        catch (Exception ex) {
            sw.Stop();
            return new ProxyCheckResult { Ip = ip, Port = port, Success = false, LatencyMs = (int)sw.ElapsedMilliseconds, Error = ex.Message };
        }
    }
}

進階驗證指標

原文代碼改進點總結

• 棄用 HttpWebRequest：改用 HttpClient + HttpClientHandler，支援異步、連接池、自動解壓、現代 TLS。
• 異步並發：SemaphoreSlim 限制併發數，避免大量併發導致本機端口耗盡或觸發目標站防火牆。
• 標準化驗證端點：推薦 httpbin.org/ip、api.ipify.org?format=json、ipinfo.io/json，回應結構穩定易解析。
• 結果物件化：封裝 ProxyCheckResult，便於持久化、排序、UI 綁定。
• 超時與異常處理：統一捕獲 TaskCanceledException、HttpRequestException、SocketException，記錄錯誤碼便於分析。

部署建議

1. 將驗證邏輯封裝為 BackgroundService (Generic Host) 定時掃描代理池；
2. 結果寫入 Redis Sorted Set (ZADD proxy:valid {score} {ip:port})，Score 為延遲，業務端 ZRANGE proxy:valid 0 9 取最快 10 個；
3. 失敗代理移入 proxy:invalid，每日重試一次；
4. 監控指標：可用率、平均延遲、新增/失效速率，接入 Grafana 告警。

總結

代理驗證看似簡單，生產環境關鍵在於 異步並發控制、標準化驗證端點、多維度指標（匿名度/協議/目標站可達性）、結果持久化與服務化。本文提供從單驗證到批量服務的完整演進代碼，可直接集成至爬蟲框架、代理池服務或隱私工具中。