Skip to content

Summary: Answering Your Two Questions About Persistent Issues

Your Observation Was Correct

The ERL/RENUM issue appears in ALL 8 versions analyzed (v7, v8, v9, v10, v11, v14, v21, v22) - that's 100% persistence! The FileIO issue appears in v21 and v22. These are indeed chronic problems that haven't been resolved despite multiple attempts.

Why Fixes Don't Stick

The core problem is incomplete resolution. When agents attempt to fix these issues, they:

  1. Find only some instances (e.g., 5 out of 8 actual occurrences)
  2. Miss variations (search for "ERL renumbering" but miss "renumber ERL comparison")
  3. Don't verify the fix (never re-run the checker to confirm the issue is gone)
  4. Move on too quickly (batch processing 20+ issues instead of thoroughly fixing one)

What Would Actually Work

I've created three deliverables to address this:

  1. docs/dev/CONSISTENCY_CHECKER_CONVERGENCE_PROPOSAL.md - A detailed proposal for fixing the convergence problem with a systematic "complete resolution" approach

  2. docs/dev/PERSISTENT_ISSUES_ANALYSIS.md - Analysis of 3,424 issues across 8 versions, identifying 334 unique recurring problems ranked by persistence

  3. utils/checker/find_all_issue_instances.py - A tool to exhaustively find ALL instances of an issue using multiple search patterns

The Solution

The key insight from analyzing the failures: Don't move to the next issue until the current one is provably eliminated.

For the ERL/RENUM issue specifically: - Use multiple search patterns (ERL.renum, renum.*ERL, _renum_erl, etc.) - Find ALL instances across code and docs - Fix them all with the same resolution - **Immediately verify* the issue is gone - Only then move to the next issue

Industry Solutions That Apply

While your distributed inconsistency problem is unique, the solution borrows from: - Boeing's approach: Fix by component, not by issue type - Distributed consensus protocols: Each subsystem votes, conflicts get resolved definitively - Test-driven development: Add regression tests to prevent reappearance

The fundamental fix is to treat this as a verification problem, not just a fixing problem. The convergence will happen when fixes are verified to be complete before moving on.

Key Files Created

  • docs/dev/CONSISTENCY_CHECKER_CONVERGENCE_PROPOSAL.md - The main proposal
  • docs/dev/PERSISTENT_ISSUES_ANALYSIS.md - Detailed analysis of all recurring issues
  • docs/dev/PERSISTENT_ISSUES_ANSWER.md - Direct answers to your two questions
  • docs/dev/persistent_issues.json - Machine-readable data of persistent issues
  • utils/checker/analyze_persistent_issues.py - Tool to analyze historical reports
  • utils/checker/find_all_issue_instances.py - Tool to exhaustively find issue instances
  • docs/dev/PERSISTENT_ISSUES_SUMMARY.md - This summary file