Started by GitHub push by ti-chi-bot[bot]Started by GitHub push by ti-chi-bot[bot]00368883688846617466171.00821182111055f9dacd257266ea0b88227be9429b2d833b5b1b70f9dacd257266ea0b88227be9429b2d833b5b1b70origin/mainf9dacd257266ea0b88227be9429b2d833b5b1b70f9dacd257266ea0b88227be9429b2d833b5b1b70origin/mainf9dacd257266ea0b88227be9429b2d833b5b1b70f9dacd257266ea0b88227be9429b2d833b5b1b70origin/mainhttps://github.com/PingCAP-QE/ci.gitfalse#10554661786474seed #10551055falsefalse10552137544SUCCESS1760067427609https://do.pingcap.net/jenkins/view/all/job/seed/1055/default-l1z17.gitignoretools/error-log-review/README.mdtools/error-log-review/config.yaml.exampletools/error-log-review/config.goconfigs/error-log-review/config.yamltools/error-log-review/matcher_test.gotools/error-log-review/matcher.gof9dacd257266ea0b88227be9429b2d833b5b1b701760067378000https://do.pingcap.net/jenkins/user/noreplynoreplynoreply@github.comfeat: Add exclusion patterns support to error-log-review CI tool (#3787) ## 🎯 Overview This PR adds support for exclusion patterns in the error-log-review CI tool, allowing test code and other non-production paths to be ignored by specific rules or globally per repository. ## 🐛 Problem The error-log-review CI was flagging log statements in test files, creating noise and false positives. For example, in [this PR run](https://prow.tidb.net/view/gs/prow-tidb-logs/pr-logs/pull/pingcap_ticdc/2337/pull-error-log-review/1976460951846653952): ``` tests/integration_tests/ddl_wait/test.go [string_literals]: log.Fatalf("insert value failed:, host:%s, port:%s, k:%d, i:%d, val:%d, num:%d, err: %+v", host, port, k, i, val, num, err) tests/integration_tests/ddl_wait/test.go [variables_and_functions]: log.Fatalf("insert value failed:, host:%s, port:%s, k:%d, i:%d, val:%d, num:%d, err: %+v", host, port, k, i, val, num, err) ``` Test code often uses different logging patterns and doesn't require the same scrutiny as production code. ## ✨ Solution Implemented a two-level exclusion system using glob patterns: ### 1. Pattern-Specific Excludes Apply exclusions to individual patterns only: ```yaml patterns: - name: "string_literals" regex: '...' excludes: - "tests/**" # Exclude all files under tests/ - "*_test.go" # Exclude all Go test files - "examples/**" # Exclude examples directory ``` ### 2. Repository-Level Excludes Apply exclusions to all patterns in a repository: ```yaml repositories: - name: "owner/repo" excludes: - "vendor/**" - "third_party/**" patterns: - name: "..." ``` ### Glob Pattern Support - `*` - matches any sequence of characters in a filename (e.g., `*_test.go`) - `**` - matches directories recursively (e.g., `tests/**` matches all files under tests/ and subdirectories) - `?` - matches any single character ## 📝 Changes ### Code Changes - **`config.go`**: Added `Excludes []string` field to both `Pattern` and `Repository` structs - **`config.go`**: Implemented `matchesAnyPattern()` function for glob pattern matching - **`matcher.go`**: Updated `CheckPRDiff()` to check exclusions before applying pattern matching ### Configuration - **`configs/error-log-review/config.yaml`**: Added `excludes: ["tests/**"]` to `pingcap/ticdc` patterns to fix the reported issue - **`config.yaml.example`**: Added comprehensive documentation and examples ### Documentation - **`README.md`**: Added "Exclusion Patterns" section with detailed examples and glob syntax reference ### Testing - **`matcher_test.go`**: Created comprehensive test suite with: - `TestMatchesAnyPattern` - Tests glob pattern matching (8 test cases) - `TestCheckPRDiffWithExclusions` - Tests end-to-end exclusion logic - `TestTiCDCExclusionScenario` - Tests the specific issue scenario - All tests passing ✅ ## 🔍 Example Usage ```yaml repositories: - name: "pingcap/ticdc" patterns: - name: "string_literals" description: "Pattern for string literals" regex: '...' excludes: - "tests/**" # Now test files won't trigger this rule approvers: - "flowbehappy" ``` ## ✅ Verification - All tests pass (3 test suites, 10 test cases) - Build succeeds - Code formatted with `go fmt` - No `go vet` issues - Configuration validated with YAML parser ## 🎉 Impact - **Immediate**: Fixes the false positives for `pingcap/ticdc` test files - **Future**: Any repository can now configure exclusions without code changes - **Flexible**: Supports both global and pattern-specific exclusions - **Backward Compatible**: Existing configurations continue to work unchanged Fixes #[issue_number]  <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>Support exclusion patterns in pull-error-log-review CI (exclude test code)</issue_title> > <issue_description>## 🪧 Summary > Add support for exclusion patterns in the error-log review CI job (pull-error-log-review) so test code and other non-production paths can be ignored by specific rules. > > ## 🙋 Problem > Currently, the job flags issues in test files, which creates noise and false positives. Example from a recent run: > - tests/integration_tests/ddl_wait/test.go [string_literals]: log.Fatalf("insert value failed:, host:%s, port:%s, k:%d, i:%d, val:%d, num:%d, err: %+v", host, port, k, i, val, num, err) > - tests/integration_tests/ddl_wait/test.go [variables_and_functions]: log.Fatalf("insert value failed:, host:%s, port:%s, k:%d, i:%d, val:%d, num:%d, err: %+v", host, port, k, i, val, num, err) > > ## Example CI run > https://prow.tidb.net/view/gs/prow-tidb-logs/pr-logs/pull/pingcap_ticdc/2337/pull-error-log-review/1976460951846653952 > > ## 📄 Proposal > - Add configurable exclusion patterns (e.g., glob paths) so we can exclude directories like tests/** by default, or on a per-repo basis. > - Consider supporting both: > - Global excludes (apply to all rules), and > - Rule-specific excludes (e.g., exclude tests/** only for string_literals and variables_and_functions). > - Allow repo-level configuration (e.g., a YAML config) to declare excludes without changing CI code for each repo. > > ## 🧐 Acceptance criteria > - The job supports path-based exclude patterns. > - Can define global and rule-specific excludes. > - Document how to configure excludes for a repository. > - Test files such as tests/integration_tests/** are not flagged by the above rules once configured. > </issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > <comment_new><author>@wuhuizuo</author><body> > /assign @lybcodes </body></comment_new> > <comment_new><author>@wuhuizuo</author><body> > The CI tool information: > - code path: tools/error-log-review > - config path: configs/error-log-review/config.yaml</body></comment_new> > </comments> > </details> Fixes PingCAP-QE/ci#3786  --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: wuhuizuo <2574558+wuhuizuo@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> 2025-10-10 03:36:18 +0000f9dacd257266ea0b88227be9429b2d833b5b1b70feat: Add exclusion patterns support to error-log-review CI tool (#3787)edit.gitignoreedittools/error-log-review/README.mdeditconfigs/error-log-review/config.yamledittools/error-log-review/config.yaml.exampleaddtools/error-log-review/matcher_test.goedittools/error-log-review/config.goedittools/error-log-review/matcher.gogithttps://do.pingcap.net/jenkins/user/noreplynoreply