Java 多執行緒下載器實作：HTTP/1.1 Range 斷點續傳核心技術解析

前言

多執行緒下載器（如 IDM、FDM、Xunlei）核心原理是利用 HTTP/1.1 Range 標頭實現分段並行下載。本文從 HTTP 協定基礎、Socket 程式設計、多執行緒協調、隨機存取檔案四個維度，完整解析 Java 實作關鍵技術。

一、HTTP/1.1 Range 請求核心機制

標準 GET 請求格式

GET /path/to/file.zip HTTP/1.1
Host: example.com
Accept: */*
User-Agent: JavaDownloader/1.0
Connection: keep-alive

分段下載 Range 請求

GET /path/to/file.zip HTTP/1.1
Host: example.com
Range: bytes=0-1023999        # 第 1 執行緒：前 1MB
Range: bytes=1024000-2047999  # 第 2 執行緒：第 2 MB
Range: bytes=2048000-         # 最後執行緒：剩餘所有

伺服器回應

# 成功：206 Partial Content
HTTP/1.1 206 Partial Content
Content-Range: bytes 0-1023999/10485760
Content-Length: 1024000
Content-Type: application/octet-stream

[二進制資料...]

# 不支援 Range：200 OK (全檔回傳)
HTTP/1.1 200 OK
Content-Length: 10485760
...

二、核心類別設計

1. 下載任務配置類

public class DownloadTask {
    private final String url;
    private final File destination;
    private final int threadCount;
    private final int bufferSize;      // 緩衝區大小，建議 8KB-64KB
    private final int connectTimeout;  // 連線超時 (ms)
    private final int readTimeout;     // 讀取超時 (ms)
    
    // Builder 模式建構...
}

2. 檔案分段計算

public class RangeCalculator {
    public static List calculateRanges(long fileSize, int threadCount) {
        List ranges = new ArrayList<>();
        long blockSize = fileSize / threadCount;
        
        for (int i = 0; i < threadCount; i++) {
            long start = i * blockSize;
            long end = (i == threadCount - 1) ? fileSize - 1 : (start + blockSize - 1);
            ranges.add(new Range(start, end));
        }
        return ranges;
    }
    
    public record Range(long start, long end) {
        public String toHeaderValue() {
            return "bytes=" + start + "-" + end;
        }
    }
}

3. 下載執行緒實作

public class DownloadThread extends Thread {
    private final DownloadTask task;
    private final Range range;
    private final RandomAccessFile raf;
    private final AtomicLong downloadedBytes;
    private volatile boolean paused = false;
    private volatile boolean cancelled = false;
    
    public DownloadThread(DownloadTask task, Range range, RandomAccessFile raf, AtomicLong counter) {
        this.task = task;
        this.range = range;
        this.raf = raf;
        this.downloadedBytes = counter;
    }
    
    @Override
    public void run() {
        HttpURLConnection conn = null;
        InputStream in = null;
        try {
            URL url = new URL(task.getUrl());
            conn = (HttpURLConnection) url.openConnection();
            conn.setRequestMethod("GET");
            conn.setRequestProperty("Range", range.toHeaderValue());
            conn.setConnectTimeout(task.getConnectTimeout());
            conn.setReadTimeout(task.getReadTimeout());
            
            int responseCode = conn.getResponseCode();
            if (responseCode != HttpURLConnection.HTTP_PARTIAL && responseCode != HttpURLConnection.HTTP_OK) {
                throw new IOException("伺服器回應異常: " + responseCode);
            }
            
            in = conn.getInputStream();
            byte[] buffer = new byte[task.getBufferSize()];
            int len;
            
            // 定位寫入位置
            synchronized (raf) {
                raf.seek(range.start());
            }
            
            while ((len = in.read(buffer)) != -1 && !cancelled) {
                while (paused) {
                    Thread.sleep(100); // 暫停等待
                }
                
                synchronized (raf) {
                    raf.write(buffer, 0, len);
                }
                downloadedBytes.addAndGet(len);
            }
        } catch (IOException | InterruptedException e) {
            // 記錄錯誤，標記失敗
        } finally {
            IOUtils.closeQuietly(in);
            if (conn != null) conn.disconnect();
        }
    }
    
    public void pause() { paused = true; }
    public void resume() { paused = false; }
    public void cancel() { cancelled = true; interrupt(); }
}

4. 主下載器協調類

public class MultiThreadDownloader {
    private final DownloadTask task;
    private final List threads = new ArrayList<>();
    private final AtomicLong totalDownloaded = new AtomicLong(0);
    private volatile long fileSize = -1;
    
    public void start() throws IOException {
        // 1. 獲取檔案大小 (HEAD 請求)
        fileSize = fetchFileSize(task.getUrl());
        if (fileSize <= 0) throw new IOException("無法獲取檔案大小或不支援 Range");
        
        // 2. 建立隨機存取檔案 (預分配空間)
        try (RandomAccessFile raf = new RandomAccessFile(task.getDestination(), "rw")) {
            raf.setLength(fileSize); // 預分配，避免碎片
            
            // 3. 計算分段
            List ranges = RangeCalculator.calculateRanges(fileSize, task.getThreadCount());
            
            // 4. 啟動執行緒
            for (Range range : ranges) {
                DownloadThread thread = new DownloadThread(task, range, raf, totalDownloaded);
                threads.add(thread);
                thread.start();
            }
            
            // 5. 進度監控
            monitorProgress();
            
            // 6. 等待完成
            for (DownloadThread t : threads) {
                t.join();
            }
            
            // 7. 驗證完整性
            verifyIntegrity();
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            cancelAll();
            throw new IOException("下載被中斷", e);
        }
    }
    
    private long fetchFileSize(String urlString) throws IOException {
        HttpURLConnection conn = (HttpURLConnection) new URL(urlString).openConnection();
        conn.setRequestMethod("HEAD");
        conn.connect();
        return conn.getContentLengthLong(); // Java 7+ 支援長度 > 2GB
    }
    
    private void monitorProgress() {
        new Thread(() -> {
            while (threads.stream().anyMatch(Thread::isAlive)) {
                long downloaded = totalDownloaded.get();
                double percent = 100.0 * downloaded / fileSize;
                System.out.printf("\r進度: %.2f%% (%d/%d bytes)", percent, downloaded, fileSize);
                try { Thread.sleep(500); } catch (InterruptedException ignored) {}
            }
            System.out.println();
        }).start();
    }
    
    private void verifyIntegrity() throws IOException {
        // 可選：MD5/SHA256 校驗
        try (FileInputStream fis = new FileInputStream(task.getDestination())) {
            MessageDigest md5 = MessageDigest.getInstance("MD5");
            byte[] buffer = new byte[8192];
            int len;
            while ((len = fis.read(buffer)) != -1) {
                md5.update(buffer, 0, len);
            }
            String actualMd5 = Hex.encodeHexString(md5.digest());
            // 比對預期 MD5...
        }
    }
    
    public void pauseAll() { threads.forEach(DownloadThread::pause); }
    public void resumeAll() { threads.forEach(DownloadThread::resume); }
    public void cancelAll() { threads.forEach(DownloadThread::cancel); }
}

三、關鍵技術難點與解決方案

四、斷點續傳持久化設計

狀態儲存格式

public class DownloadState implements Serializable {
    private String url;
    private String destination;
    private long fileSize;
    private List ranges; // 每段起始、已下載、結束
    private String expectedHash; // MD5/SHA256
    private long lastUpdated;
    
    public record RangeProgress(long start, long end, long downloaded) {}
}

啟動時恢復邏輯

public void resumeDownload(File stateFile) throws IOException, ClassNotFoundException {
    try (ObjectInputStream ois = new ObjectInputStream(new FileInputStream(stateFile))) {
        DownloadState state = (DownloadState) ois.readObject();
        
        // 驗證檔案未變更
        if (!verifyFileUnchanged(state)) {
            throw new IllegalStateException("伺服器檔案已變更，需重新下載");
        }
        
        // 計算各段剩餘範圍
        List remainingRanges = new ArrayList<>();
        for (RangeProgress rp : state.ranges()) {
            if (rp.downloaded() < rp.end() - rp.start() + 1) {
                long newStart = rp.start() + rp.downloaded();
                remainingRanges.add(new Range(newStart, rp.end()));
            }
        }
        
        // 以剩餘範圍繼續下載...
    }
}

五、現代化改良建議

1. 使用 HttpClient (Java 11+)：支援 HTTP/2、非阻塞 IO、CompletableFuture 異步模型
2. 零拷貝傳輸：FileChannel.transferFrom() 直接從 SocketChannel 寫入檔案
3. 反應式流：Project Reactor / RxJava 背壓控制、聲明式重試
4. 原生映像：GraalVM 編譯為原生執行檔，啟動快、記憶體少
5. 跨平台 GUI：JavaFX / Compose for Desktop 替代 Swing

六、完整範例：簡易命令列下載器

public class SimpleDownloader {
    public static void main(String[] args) {
        if (args.length < 2) {
            System.err.println("用法: java SimpleDownloader  <輸出檔案> [執行緒數=4]");
            return;
        }
        
        String url = args[0];
        File dest = new File(args[1]);
        int threads = args.length > 2 ? Integer.parseInt(args[2]) : 4;
        
        DownloadTask task = DownloadTask.builder()
            .url(url)
            .destination(dest)
            .threadCount(threads)
            .bufferSize(32 * 1024)
            .build();
        
        MultiThreadDownloader downloader = new MultiThreadDownloader(task);
        try {
            downloader.start();
            System.out.println("\n下載完成: " + dest.getAbsolutePath());
        } catch (IOException e) {
            System.err.println("下載失敗: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

總結

Java 多執行緒下載器核心在於：HTTP Range 分段、RandomAccessFile 隨機寫入、執行緒同步協調、狀態持久化斷點續傳。理解這些底層機制，不僅能實作下載工具，更能深入掌握 HTTP 協定、NIO、併發程式設計等核心技術，為構建高效能網路應用打下堅實基礎。