Java DFA 算法實現敏感詞過濾：構建高性能敏感詞引擎全流程

方案選型對比

DFA 核心原理

將敏感詞集合構建為前綴樹，每個節點為字符，標記 isEnd 標識詞尾。檢測時單次遍歷文本，狀態機跳轉，命中即輸出。

數據結構定義

// 敏感詞庫：Map<Character, Node> 遞歸嵌套
// Node 結構：{ children: Map<Char, Node>, isEnd: boolean }
Map<Object, Object> sensitiveWordMap = new HashMap<>();

完整實現代碼

1. 敏感詞庫初始化

public class SensitiveWordInit {
    public Map<Object, Object> initKeyWord(List<String> words) {
        Map<Object, Object> root = new HashMap<>();
        for (String word : words) {
            Map<Object, Object> node = root;
            for (int i = 0; i < word.length(); i++) {
                char c = word.charAt(i);
                @SuppressWarnings("unchecked")
                Map<Object, Object> next = (Map<Object, Object>) node.get(c);
                if (next == null) {
                    next = new HashMap<>();
                    next.put("isEnd", false);
                    node.put(c, next);
                }
                node = next;
                if (i == word.length() - 1) node.put("isEnd", true);
            }
        }
        return root;
    }
}

2. 檢測引擎核心

public class SensitiveWordEngine {
    private static Map<Object, Object> WORD_MAP;
    public static final int MIN_MATCH = 1; // 最短匹配
    public static final int MAX_MATCH = 2; // 最長匹配

    public static void setWordMap(Map<Object, Object> map) { WORD_MAP = map; }

    /** 檢測文本中是否包含敏感詞 */
    public static boolean contains(String text) {
        for (int i = 0; i < text.length(); i++) {
            if (check(text, i, MIN_MATCH) > 0) return true;
        }
        return false;
    }

    /** 獲取所有敏感詞（去重） */
    public static Set<String> getWords(String text, int matchType) {
        Set<String> result = new HashSet<>();
        for (int i = 0; i < text.length(); i++) {
            int len = check(text, i, matchType);
            if (len > 0) {
                result.add(text.substring(i, i + len));
                i += len - 1; // 跳過已匹配部分
            }
        }
        return result;
    }

    /** 替換敏感詞為指定字符 */
    public static String replace(String text, int matchType, String replacement) {
        StringBuilder sb = new StringBuilder(text);
        Set<String> words = getWords(text, matchType);
        for (String w : words) {
            String repl = replacement.repeat(w.length());
            int idx = -1;
            while ((idx = sb.indexOf(w, idx + 1)) != -1) {
                sb.replace(idx, idx + w.length(), repl);
            }
        }
        return sb.toString();
    }

    /** 核心檢測邏輯：從 text[begin] 開始在字典樹中行進 */
    private static int check(String text, int begin, int matchType) {
        Map<Object, Object> node = WORD_MAP;
        int matched = 0;
        boolean isEnd = false;
        
        for (int i = begin; i < text.length(); i++) {
            char c = text.charAt(i);
            @SuppressWarnings("unchecked")
            Map<Object, Object> next = (Map<Object, Object>) node.get(c);
            if (next == null) break;
            
            matched++;
            node = next;
            if (Boolean.TRUE.equals(node.get("isEnd"))) {
                isEnd = true;
                if (matchType == MIN_MATCH) break; // 最短匹配即返回
            }
        }
        return isEnd ? matched : 0;
    }
}

3. Service 層整合

@Service
public class SensitiveWordService {
    @Resource private SensitiveWordDao dao;
    
    public Set<String> filter(String text) {
        // 1. 加載詞庫（建議啟動時緩存，定時刷新）
        List<String> words = dao.loadAllWords();
        Map<Object, Object> map = new SensitiveWordInit().initKeyWord(words);
        SensitiveWordEngine.setWordMap(map);
        
        // 2. 檢測
        return SensitiveWordEngine.getWords(text, SensitiveWordEngine.MAX_MATCH);
    }
}

4. Controller 暴露

@RestController
@RequestMapping("/api/sensitive")
public class SensitiveWordController {
    @Resource private SensitiveWordService service;
    
    @GetMapping("/check")
    public R check(@RequestParam String text) {
        Set<String> words = service.filter(text);
        return R.ok().put("words", words).put("hasSensitive", !words.isEmpty());
    }
}

生產優化建議

1. 詞庫熱加載：使用 ReadWriteLock 保護 WORD_MAP，定時任務異步重建新 Map 原子替換
2. 分級過濾：詞庫帶等級（高/中/低），引擎返回帶等級結果，業務按策略處理
3. 分詞預處理：中文結合分詞器（IK/Jieba）切詞後再過濾，減少誤傷
4. 併發壓測：單機 QPS 可達 10w+，瓶頸在 GC，建議對象池複用 StringBuilder
5. 持久化結構：詞庫序列化為二進制文件，啟動直接 mmap 加載，毫秒級初始化

測試用例

@Test
public void testEngine() {
    List<String> words = Arrays.asList("敏感詞", "測試", "關鍵字");
    Map<Object, Object> map = new SensitiveWordInit().initKeyWord(words);
    SensitiveWordEngine.setWordMap(map);
    
    String text = "這是一個測試文本，包含敏感詞和關鍵字。";
    assertTrue(SensitiveWordEngine.contains(text));
    assertEquals(Set.of("測試", "敏感詞", "關鍵字"), 
        SensitiveWordEngine.getWords(text, SensitiveWordEngine.MAX_MATCH));
    assertEquals("這是一個**文本，包含****和****。",
        SensitiveWordEngine.replace(text, SensitiveWordEngine.MAX_MATCH, "*"));
}