test(phase23): KEEP-VERIFY — remove 44 pass shells + fake INV tests, add shell guard

Phase 23 "可信系统测试重建 (REFACTOR)" re-audit. STATUS claimed "10 E2E
files / ~80+ cases" and "INV-011~020 ✅"; measurement showed the opposite.

D-23-A (Critical): V11InvariantsTest's 14 cases asserted only Constants
strings. INV-016 "Banner target types" actually asserted ARTICLE_STATUS_*;
INV-017 "home 8 entries" declared local arrays then asserted
firstRow.length == 4 (tautology, never called the home service);
INV-018/019/020 used appointment/article/account status constants as
stand-ins for permission and data-scope checks. Rewritten to 12 cases:
4 real service-behavior guards (INV-013 via real CounselingService.getDetail
— Matchmaker sees null userRequestSummary, with an assigned-Counselor
counter-proof so global sanitization cannot fake a pass; INV-016 reads the
production ALLOWED_TARGET_TYPES whitelist and asserts no external/http/url/
web entry; INV-017 calls real getHome and pins all 8 codes + order;
INV-019 asserts content_editor reads null), plus assertBehaviorTestExists
turning "行为验证见 XxxTest" pointers into executable Class.forName contracts.

D-23-B (High): 44 `pass` shells across 6 files in tests/e2e (4 files were
100% shells). Gate forbids skip/pass shells. Rewritten in place as honest
placeholders with the full scenario lists preserved.

D-23-C (High, residual): the pytest E2E suite is not executable at all —
no conftest.py anywhere, ~15 fixtures (api_client/verified_user/
banner_factory/...) do not exist. Only run_v11_tests.py really runs
(16 cases, needs a live stack + E2E_PASSWORD).

D-23-D (High, residual, NOT fixed): CI does not chain E2E as the plan
requires. Needs an infrastructure decision on how CI provisions a live
stack; not implemented rather than written unverified.

D-23-E (Medium): the CI migration job claimed to "验证 migration SQL 语法"
but only checked file non-emptiness, while the real syntax/baseline checks
live in MigrationValidationTest (already run by the backend job). Replaced
with a real sequence check (naming / non-empty / unique versions / no
gaps); verified by extracting the exact inline script and running it.

New gate: scripts/check-no-empty-tests.py (AST scan, blocks pass-only and
unconditionally skipped tests) + a CI test-shell-guard job. Verified both
ways: 0 findings on the real tree, 2 precise findings on injected shells.

Negative validation: 3 injected production defects (external_url in the
banner whitelist, swapped home entry order, isMatchmaker returning the full
record) turned exactly inv016/inv017/inv013 red, 9 others green, zero
unrelated failures; md5sum -c confirmed byte-identical restore.

Full regression: 655 tests / 652 passed / 0 failures / 0 errors / 3 skipped
(baseline 657; -2 because 14 vacuous cases were merged into 12). The 3
skips carry explicit reasons (2x @EnabledIfEnvironmentVariable(CI=true),
1x @Disabled "需要完整数据库连接") — none is a shell.

Phase 23 is PARTIALLY closed: fake coverage is gone, but the UAT gate
requires the end-to-end chain to actually execute, so "失败为零可进入 UAT"
is NOT claimed.
This commit is contained in:
WorkBuddy
2026-09-18 11:49:18 +08:00
parent 050d005c0a
commit ae1bd2b734
12 changed files with 798 additions and 415 deletions
+62 -11
View File
@@ -79,15 +79,66 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Validate Flyway migrations
working-directory: backend
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
# Phase 23 KEEP-VERIFY:本步骤原注释称「验证 migration SQL 语法」,实际只检查文件非空,
# 属名不副实的假校验(SQL 语法与基线表完整性由后端测试 MigrationValidationTest 覆盖,
# 该测试随 backend job 的 `mvn verify` 运行)。现改为该测试未覆盖的**序列**校验。
- name: 迁移序列校验(命名 / 非空 / 版本号唯一 / 无缺号)
run: |
# 验证 migration SQL 语法(通过检查文件是否存在且非空)
find src/main/resources/db/migration -name "V*.sql" -type f | sort | while read f; do
echo "Validating: $f"
if [ ! -s "$f" ]; then
echo "ERROR: Empty migration file: $f"
exit 1
fi
done
echo "All migration files validated."
python3 - <<'PY'
import re, sys, pathlib, collections
d = pathlib.Path("backend/src/main/resources/db/migration")
files = sorted(d.glob("V*.sql"))
if not files:
sys.exit("ERROR: 未找到任何迁移文件")
vers, bad_name, empty = [], [], []
for f in files:
m = re.match(r"V(\d+)__", f.name)
if not m:
bad_name.append(f.name)
continue
if f.stat().st_size == 0:
empty.append(f.name)
vers.append(int(m.group(1)))
problems = []
if bad_name:
problems.append(f"迁移文件命名不合规(应为 V<序号>__<描述>.sql): {bad_name}")
if empty:
problems.append(f"空迁移文件: {empty}")
dups = sorted(v for v, c in collections.Counter(vers).items() if c > 1)
if dups:
problems.append(f"迁移版本号重复: {dups}")
gaps = [v for v in range(min(vers), max(vers) + 1) if v not in set(vers)]
if gaps:
problems.append(f"迁移版本号缺号: {gaps}")
if problems:
for p in problems:
print("ERROR: " + p, file=sys.stderr)
sys.exit(1)
print(f"OK: {len(files)} 个迁移文件,版本 {min(vers)}~{max(vers)} 连续且唯一")
PY
test-shell-guard:
name: Test Shell Guard (Phase 23 Gate)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
# Phase 23 Gate:「不允许 skip/pass 空壳」。
# 机械阻断空壳回归 —— 详见 scripts/check-no-empty-tests.py。
- name: 禁止 skip/pass 空壳测试
run: python3 scripts/check-no-empty-tests.py tests