typed-holes-refactor

Refactor codebases using Design by Typed Holes methodology - iterative, test-driven refactoring with formal hole resolution, constraint propagation, and continuous validation. Use when refactoring existing code, optimizing architecture, or consolidating technical debt through systematic hole-driven development.

Typed Holes Refactoring

Systematically refactor codebases using the Design by Typed Holes meta-framework: treat architectural unknowns as typed holes, resolve them iteratively with test-driven validation, and propagate constraints through dependency graphs.

Core Workflow

Phase 0: Hole Discovery & Setup

1. Create safe working branch:

git checkout -b refactor/typed-holes-v1
# CRITICAL: Never work in main, never touch .beads/ in main

2. Analyze current state and identify holes:

python scripts/discover_holes.py
# Creates REFACTOR_IR.md with hole catalog

The Refactor IR documents:

3. Write baseline characterization tests:

Create tests/characterization/ to capture exact current behavior:

# tests/characterization/test_current_behavior.py
def test_api_contracts():
    """All public APIs must behave identically post-refactor"""
    for endpoint in discover_public_apis():
        old_result = run_current(endpoint, test_inputs)
        save_baseline(endpoint, old_result)

def test_performance_baselines():
    """Record current performance - don't regress"""
    baselines = measure_all_operations()
    save_json("baselines.json", baselines)

Run tests on main branch - they should all pass. These are your safety net.

Phase 1-N: Iterative Hole Resolution

For each hole (in dependency order):

1. Select next ready hole:

python scripts/next_hole.py
# Shows holes whose dependencies are resolved

2. Write validation tests FIRST (test-driven):

# tests/refactor/test_h{N}_resolution.py
def test_h{N}_resolved():
    """Define what 'resolved correctly' means"""
    # This should FAIL initially
    assert desired_state_achieved()

def test_h{N}_equivalence():
    """Ensure no behavioral regressions"""
    old_behavior = load_baseline()
    new_behavior = run_refactored()
    assert old_behavior == new_behavior

3. Implement resolution:

4. Validate resolution:

python scripts/validate_resolution.py H{N}
# Checks: tests pass, constraints satisfied, main untouched

5. Propagate constraints:

python scripts/propagate.py H{N}
# Updates dependent holes based on resolution

6. Document and commit:

git add .
git commit -m "Resolve H{N}: {description}

- Tests: tests/refactor/test_h{N}_*.py pass
- Constraints: {constraints satisfied}
- Propagates to: {dependent holes}"

Phase Final: Reporting

Generate comprehensive delta report:

python scripts/generate_report.py > REFACTOR_REPORT.md

Report includes:

Key Principles

1. Test-Driven Everything

2. Hole-Driven Progress

3. Continuous Validation

Every commit must validate:

4. Safe by Construction

5. Formal Completeness

Design complete when:

Hole Quality Framework

SMART Criteria for Good Holes

Every hole must be:

Hole Estimation Framework

Size holes using these categories:

| Size | Duration | Characteristics | Examples | |------|----------|-----------------|----------| | Nano | 1-2 hours | Simple, mechanical changes | Rename files, update imports | | Small | 4-8 hours | Single module refactor | Extract class, consolidate functions | | Medium | 1-3 days | Cross-module changes | Define interfaces, reorganize packages | | Large | 4-7 days | Architecture changes | Layer extraction, pattern implementation | | Epic | >7 days | SPLIT THIS HOLE | Too large, break into smaller holes |

Estimation Red Flags:

Hole Splitting Guidelines

Split a hole when:

  1. Estimate exceeds 7 days
  2. More than 5 dependencies
  3. Validation criteria unclear
  4. Multiple distinct concerns mixed

Splitting strategy:

Epic hole: "Refactor entire authentication system"
→ Split into:
  R10_auth_interface: Define new auth interface (Medium)
  R11_token_handling: Implement JWT tokens (Small)
  R12_session_management: Refactor sessions (Medium)
  R13_auth_middleware: Update middleware (Small)
  R14_auth_testing: Comprehensive test suite (Medium)

After splitting:

Common Hole Types

Architecture Holes

"?R1_target_architecture": "What should the ideal structure be?"
"?R2_module_boundaries": "How should modules be organized?"
"?R3_abstraction_layers": "What layers/interfaces are needed?"

Validation: Architecture tests, dependency analysis, layer violation checks

Implementation Holes

"?R4_consolidation_targets": "What code should merge?"
"?R5_extraction_targets": "What code should split out?"
"?R6_elimination_targets": "What code should be removed?"

Validation: Duplication detection, equivalence tests, dead code analysis

Quality Holes

"?R7_test_strategy": "How to validate equivalence?"
"?R8_migration_path": "How to safely transition?"
"?R9_rollback_mechanism": "How to undo if needed?"

Validation: Test coverage metrics, migration dry-runs, rollback tests

See HOLE_TYPES.md for complete catalog.

Constraint Propagation Rules

Rule 1: Interface Resolution → Type Constraints

When: Interface hole resolved with concrete types
Then: Propagate type requirements to all consumers

Example:
  Resolve R6: NodeInterface = BaseNode with async run()
  Propagates to:
    → R4: Parallel execution must handle async
    → R5: Error recovery must handle async exceptions

Rule 2: Implementation → Performance Constraints

When: Implementation resolved with resource usage
Then: Propagate limits to dependent holes

Example:
  Resolve R4: Parallelization with max_concurrent=3
  Propagates to:
    → R8: Rate limit = provider_limit / 3
    → R7: Memory budget = 3 * single_operation_memory

Rule 3: Validation → Test Requirements

When: Validation resolved with test requirements
Then: Propagate data needs upstream

Example:
  Resolve R9: Testing needs 50 examples
  Propagates to:
    → R7: Metrics must support batch evaluation
    → R8: Test data collection strategy needed

See CONSTRAINT_RULES.md for complete propagation rules.

Success Indicators

Weekly Progress

Red Flags (Stop & Reassess)

Validation Gates

| Gate | Criteria | Check | |------|----------|-------| | Gate 1: Discovery Complete | All holes cataloged, dependencies mapped | python scripts/check_discovery.py | | Gate 2: Foundation Holes | Core interfaces resolved, tests pass | python scripts/check_foundation.py | | Gate 3: Implementation | All refactor holes resolved, metrics improved | python scripts/check_implementation.py | | Gate 4: Production Ready | Migration tested, rollback verified | python scripts/check_production.py |

Claude-Assisted Workflow

This skill is designed for effective Claude/LLM collaboration. Here's how to divide work:

Phase 0: Discovery

Claude's Role:

Your Role:

Phase 1-N: Hole Resolution

Claude's Role:

Your Role:

Phase Final: Reporting

Claude's Role:

Your Role:

Effective Prompting Patterns

Starting a session:

"I need to refactor [description]. Use typed-holes-refactor skill.
Start with discovery phase."

Resolving a hole:

"Resolve H3 (target_architecture). Write tests first, then implement.
Use [specific pattern/approach]."

Checking progress:

"Run check_completeness.py and show me the dashboard.
What's ready to work on next?"

Generating visualizations:

"Generate dependency graph showing bottlenecks and critical path.
Use visualize_graph.py with --analyze."

Claude's Limitations

Claude CANNOT:

Claude CAN:

Multi-Session Continuity

At session start:

"Continue typed-holes refactoring. Import beads state and
show current status from REFACTOR_IR.md."

Claude will:

You should:

Beads Integration

Why beads + typed holes?

Setup

# Install beads (once)
go install github.com/steveyegge/beads/cmd/bd@latest

# After running discover_holes.py
python scripts/holes_to_beads.py

# Check what's ready
bd ready --json

Workflow Integration

During hole resolution:

# Start work on a hole
bd update bd-5 --status in_progress --json

# Implement resolution
# ... write tests, implement code ...

# Validate resolution
python scripts/validate_resolution.py H3

# Close bead
bd close bd-5 --reason "Resolved H3: target_architecture" --json

# Export state
bd export -o .beads/issues.jsonl
git add .beads/issues.jsonl REFACTOR_IR.md
git commit -m "Resolve H3: Define target architecture"

Syncing holes ↔ beads:

# After updating REFACTOR_IR.md manually
python scripts/holes_to_beads.py  # Sync changes to beads

# After resolving holes
python scripts/holes_to_beads.py  # Update bead statuses

Cross-session continuity:

# Session start
bd import -i .beads/issues.jsonl
bd ready --json  # Shows ready holes
python scripts/check_completeness.py  # Shows overall progress

# Session end
bd export -o .beads/issues.jsonl
git add .beads/issues.jsonl
git commit -m "Session checkpoint: 3 holes resolved"

Bead advantages:

Scripts Reference

All scripts are in scripts/:

Run any script with --help for detailed usage.

Meta-Consistency

This skill uses its own principles:

| Typed Holes Principle | Application to Refactoring | |-----------------------|----------------------------| | Typed Holes | Architectural unknowns cataloged with types | | Constraint Propagation | Design constraints flow through dependency graph | | Iterative Refinement | Hole-by-hole resolution cycles | | Test-Driven Validation | Tests define correctness | | Formal Completeness | Gates verify design completeness |

We use the system to refactor the system.

Advanced Topics

For complex scenarios, see:

Quick Start Example

# 1. Setup
git checkout -b refactor/typed-holes-v1
python scripts/discover_holes.py

# 2. Write baseline tests
# Create tests/characterization/test_*.py

# 3. Resolve first hole
python scripts/next_hole.py  # Shows H1 is ready
# Write tests/refactor/test_h1_*.py (fails initially)
# Refactor code until tests pass
python scripts/validate_resolution.py H1
python scripts/propagate.py H1
git commit -m "Resolve H1: ..."

# 4. Repeat for each hole
# ...

# 5. Generate report
python scripts/generate_report.py > REFACTOR_REPORT.md

Troubleshooting

Characterization tests fail

Symptom: Tests that captured baseline behavior now fail

Resolution:

  1. Revert changes: git diff to see what changed
  2. Investigate: What behavior changed and why?
  3. Decision tree:

``python # Update baseline with reason save_baseline("v2_api", new_behavior, reason="Switched to async implementation") ``

Prevention: Run characterization tests before AND after each hole resolution.

Hole can't be resolved

Symptom: Stuck on a hole for >3 days, unclear how to proceed

Resolution:

  1. Check dependencies: Are they actually resolved?

``bash python scripts/visualize_graph.py --analyze # Look for unresolved dependencies ``

  1. Review constraints: Are they contradictory?
  1. Split the hole: If hole is too large

``bash # Original: R4_consolidate_all (Epic, 10+ days) # Split into: R4a_consolidate_parsers (Medium, 2 days) R4b_consolidate_validators (Small, 1 day) R4c_consolidate_handlers (Medium, 2 days) ``

  1. Check for circular dependencies:

``bash python scripts/visualize_graph.py # Look for cycles: R4 → R5 → R6 → R4 ``

Escalation: If still stuck after 5 days, consider alternative refactoring approach.

Contradictory Constraints

Symptom: Cannot satisfy all constraints simultaneously

Example:

Resolution Framework:

  1. Identify the conflict:

``markdown C1 requires: Keep synchronous operations C5 requires: Switch to async operations → Contradiction: Can't be both sync and async ``

  1. Negotiate priorities:

| Option | C1 | C5 | Tradeoff | |--------|----|----|----------| | A: Keep sync | ✓ | ✗ | No performance gain | | B: Switch to async | ✗ | ✓ | Breaking change | | C: Add async, deprecate sync | ⚠️ | ✓ | Migration burden |

  1. Choose resolution strategy:
  1. Document decision:

```markdown ## Constraint Resolution: C1 vs C5

Decision: Relax C1 to allow async migration Rationale: Performance critical for user experience Migration: 3-month deprecation period for sync API Approved by: [Stakeholder], [Date] ```

Circular Dependencies

Symptom: visualize_graph.py shows cycles

Example:

R4 (consolidate parsers) → depends on R6 (define interface)
R6 (define interface) → depends on R4 (needs parser examples)

Resolution strategies:

  1. Introduce intermediate hole:

`` H0_parser_analysis: Analyze existing parsers (no dependencies) R6_interface: Define interface using H0 analysis R4_consolidate: Implement using R6 interface ``

  1. Redefine dependencies:
  1. Accept iterative refinement:

`` R6_interface_v1: Initial interface (simple) R4_consolidate: Implement with v1 interface R6_interface_v2: Refine based on R4 learnings ``

Prevention: Define architecture holes before implementation holes.

No Progress for 3+ Days

Symptom: Feeling stuck, no commits, uncertain how to proceed

Resolution checklist:

Reset protocol: If still stuck, revert to last working state and try different approach.

Estimation Failures

Symptom: Hole taking 3x longer than estimated

Analysis:

  1. Why did estimate fail?
  1. Immediate actions:
  1. Future improvements:

Learning: Track actual vs estimated time in REFACTOR_REPORT.md for future reference.

Begin with Phase 0: Discovery. Always work in a branch. Test first, refactor second.