shopdb-flask/.claude/skills/migration-check.md

name, description

name	description
migration-check	Deep validation of plugin migrations for ADR-008 compliance - check linearity, dependencies, foreign keys

Migration Safety Check

Comprehensive validation of database migrations to prevent ADR-008 violations and production failures.

What This Skill Does

Deep migration analysis:

1. Verify each plugin migration chain is linear (no branches)
2. Check for circular dependencies between plugins
3. Validate foreign key references exist
4. Test migrations up AND down
5. Verify PLUGIN_TABLE_OWNERS coordination
6. Check for duplicate version numbers
7. Identify orphaned migrations

Why This Matters

ADR-008: Plugin Migration Ownership

• Each plugin owns its migration chain
• Chains must be linear (no branches)
• Migrations run in dependency order (PLUGIN_TABLE_OWNERS)
• Violations cause production deployment failures

Historical issues:

• Plugin migration branches blocked deployments
• Circular dependencies caused deadlocks
• Missing foreign keys led to data integrity issues

Workflow

1. List All Migrations

Scan all plugin migration directories:

find plugins/*/migrations/versions/ -name "*.py" -type f | sort

For each plugin:

• Count migration files
• Check naming convention (revision_description.py)
• Verify chronological order by timestamp

2. Check Chain Linearity

For each plugin, verify migration chain has no branches:

# Pseudo-code logic:
for plugin in plugins:
    migrations = get_migrations(plugin)
    
    # Build dependency graph
    graph = {}
    for migration in migrations:
        parent = migration.down_revision
        graph[parent] = graph.get(parent, []) + [migration.revision]
    
    # Check for branches (parent with multiple children)
    for parent, children in graph.items():
        if len(children) > 1:
            ERROR: Branch detected at {parent}
            Children: {children}

What to flag:

• Multiple migrations with same down_revision (branch)
• Migrations with no parent (orphaned)
• Migrations with missing parent (broken chain)

Example violation:

ERROR: Plugin 'endpoints' has branched migration chain:
  Parent: a1b2c3d4
  Branch 1: e5f6g7h8 (Add certificate_thumbprint column)
  Branch 2: i9j0k1l2 (Add last_login_at column)
  
  ADR-008 violation: Merge these branches or drop one

3. Check Plugin Dependencies

Verify PLUGIN_TABLE_OWNERS coordination:

grep -A 50 "PLUGIN_TABLE_OWNERS" shopdb/config.py

What to check:

• Plugins listed in dependency order
• If Plugin A references Plugin B's table, B must come first
• No circular dependencies (A depends on B, B depends on A)

Build dependency graph:

# For each plugin migration:
for migration in all_migrations:
    # Check for foreign keys to other plugins
    if references_other_plugin_table(migration):
        add_dependency(current_plugin, other_plugin)

# Check for cycles
if has_circular_dependency(dependencies):
    ERROR: Circular dependency detected

Example violation:

ERROR: Circular plugin dependencies:
  endpoints -> computers (fk: computer_id)
  computers -> endpoints (fk: endpoint_id)
  
  ADR-008 violation: Break circular reference
  Recommendation: Create junction table or denormalize

4. Validate Foreign Key References

For each foreign key in migrations, verify target table exists:

# Check op.create_foreign_key() calls
for migration in all_migrations:
    for fk in get_foreign_keys(migration):
        target_table = fk.target_table
        target_plugin = get_owning_plugin(target_table)
        
        # Verify target plugin comes before current in PLUGIN_TABLE_OWNERS
        if not comes_before(target_plugin, current_plugin):
            ERROR: Invalid foreign key order

What to flag:

• Foreign key to table owned by later plugin
• Foreign key to non-existent table
• Foreign key without index on source column

Example violation:

WARNING: Plugin 'printers' references 'computers' table
  Migration: 2024_03_15_add_computer_id.py
  Foreign key: printer.computer_id -> computer.id
  
  PLUGIN_TABLE_OWNERS order:
    printers (index 5)
    computers (index 8)
  
  ERROR: printers loads before computers (table won't exist)
  Fix: Move printers after computers in PLUGIN_TABLE_OWNERS

5. Test Migrations Up

Apply all migrations in test database:

# Create test database
docker compose exec api flask db upgrade

# Upgrade each plugin
docker compose exec api flask plugin upgrade-all

Monitor for:

• Alembic errors
• Foreign key constraint violations
• Circular dependency deadlocks
• Duplicate table/column errors

If migrations fail:

• Show exact error
• Identify which migration failed
• Suggest fix based on error type

6. Test Migrations Down

Verify migrations can roll back cleanly:

# Downgrade plugins in reverse order
for plugin in reverse(PLUGIN_TABLE_OWNERS):
    docker compose exec api flask plugin downgrade $plugin base
done

# Downgrade core
docker compose exec api flask db downgrade base

What to check:

• All downgrades succeed
• No orphaned tables/columns
• Foreign keys dropped before tables

Common issues:

• Downgrade drops table but doesn't drop foreign keys first
• Downgrade missing (NotImplementedError)
• Downgrade order violation

7. Check for Duplicate Versions

Ensure no two migrations share same revision ID:

# Extract all revision IDs
grep -r "revision = " plugins/*/migrations/versions/*.py | cut -d"'" -f2 | sort | uniq -d

If duplicates found:

ERROR: Duplicate revision IDs detected:
  a1b2c3d4 in plugins/endpoints/migrations/versions/001_initial.py
  a1b2c3d4 in plugins/computers/migrations/versions/003_add_field.py
  
  Fix: Regenerate one migration with new revision ID

8. Identify Orphaned Migrations

Find migrations with parents that don't exist:

# Build set of all revision IDs
all_revisions = set(migration.revision for migration in all_migrations)

# Check each migration's parent exists
for migration in all_migrations:
    if migration.down_revision and migration.down_revision not in all_revisions:
        ERROR: Orphaned migration {migration.revision}
        Parent {migration.down_revision} not found

Causes:

• Parent migration deleted
• Migration from different branch merged incorrectly
• Manual migration editing gone wrong

9. Summary Report

[X] All checks passed:

Migration safety check complete:

Plugin Migration Chains:
[X] endpoints: 12 migrations, linear
[X] computers: 8 migrations, linear
[X] printers: 5 migrations, linear
[X] (... all plugins ...)

Dependency Analysis:
[X] No circular dependencies
[X] All foreign keys valid
[X] PLUGIN_TABLE_OWNERS order correct

Migration Tests:
[X] All migrations applied successfully
[X] All downgrades succeeded
[X] No duplicate revision IDs

Migrations are safe to deploy!

WARNING: Issues detected:

Migration safety check FAILED:

CRITICAL ISSUES (block deployment):
1. Plugin 'endpoints' has branched migration chain
   Parent a1b2c3d4 has 2 children
   Action: Merge branches or drop conflicting migration

2. Circular dependency: endpoints -> computers
   Action: Break circular reference with junction table

3. Foreign key order violation: printers -> computers
   printers loads before computers in PLUGIN_TABLE_OWNERS
   Action: Reorder plugins or remove foreign key

HIGH PRIORITY (should fix before deploy):
4. Migration 'endpoints/003_add_field' has no downgrade
   Action: Implement down() method

5. Orphaned migration: computers/005_update
   Parent revision not found
   Action: Fix down_revision or remove migration

WARNINGS (review but not blocking):
6. Large migration: endpoints/012_backfill (may be slow)
   Action: Consider data migration strategy

DO NOT DEPLOY until critical issues resolved!

Detailed logs: /tmp/migration-check-{timestamp}.log

Troubleshooting

Branch detected:

• Option 1: Squash branch into single migration
• Option 2: Drop conflicting branch
• Option 3: Reorder to make linear

Circular dependency:

• Create junction table (owned by one plugin)
• Denormalize (copy data instead of FK)
• Move table ownership to core

Foreign key order violation:

• Reorder PLUGIN_TABLE_OWNERS
• Remove foreign key (use app-level relationship)
• Move both tables to same plugin

Migration fails to apply:

• Check for missing dependencies
• Verify table/column doesn't already exist
• Check for data that violates constraints

Related

• /pre-commit - Includes basic migration linearity check
• /production-health - Post-deployment validation
• ADR-008: Plugin migration ownership
• PLUGIN_TABLE_OWNERS in shopdb/config.py
• Alembic documentation: https://alembic.sqlalchemy.org/