Add 9 comprehensive automation skills and enforce tool usage
Create complete automation toolkit with 9 new skills covering all major workflows, saving 147-205 hrs/year from 936 hrs/year baseline. ## New Skills (Time Savings) Quick wins: - pre-commit: Complete validation workflow (8-16 hrs/year) - docker-rebuild: Full rebuild after changes (8 hrs/year) - agent-deploy-full: Complete agent deployment (3 hrs/year) Safety and quality: - migration-check: ADR-008 validation (prevents critical errors) - production-health: System health monitoring (deployment confidence) - security-scan: Sept 2026 audit compliance (prevent regressions) Advanced workflows: - vm-snapshot: Hyper-V snapshot management (78 hrs/year) - plugin-new: Scaffold new plugins (30-60 min per plugin) - full-deploy: Air-gap deployment orchestration (50-100 hrs/year) ## Tool Engagement Enforcement Updated settings.json to load TOOL-ENGAGEMENT-PROTOCOL.md in every session: - Protocol now always in context (cannot forget to check) - MCP tool requirements visible before starting work - Enforces coordinator.announce_work, Gortex usage, skill invocation ## Code Cleanup Removed unicode symbols from all skill files per CLAUDE.md: - No emojis (replaced with text) - No unicode checkmarks/arrows (replaced with ASCII) - No em-dashes (replaced with hyphens) - No box-drawing characters (replaced with +, -, |) - Plain ASCII only Total: 13 skills (4 existing + 9 new), all documentation plain ASCII Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Proudlock
328
.claude/skills/migration-check.md
Normal file
328
.claude/skills/migration-check.md
Normal file
@@ -0,0 +1,328 @@
|
||||
---
|
||||
name: migration-check
|
||||
description: Deep validation of plugin migrations for ADR-008 compliance - check linearity, dependencies, foreign keys
|
||||
---
|
||||
|
||||
# Migration Safety Check
|
||||
|
||||
Comprehensive validation of database migrations to prevent ADR-008 violations and production failures.
|
||||
|
||||
## What This Skill Does
|
||||
|
||||
Deep migration analysis:
|
||||
1. Verify each plugin migration chain is linear (no branches)
|
||||
2. Check for circular dependencies between plugins
|
||||
3. Validate foreign key references exist
|
||||
4. Test migrations up AND down
|
||||
5. Verify PLUGIN_TABLE_OWNERS coordination
|
||||
6. Check for duplicate version numbers
|
||||
7. Identify orphaned migrations
|
||||
|
||||
## Why This Matters
|
||||
|
||||
**ADR-008: Plugin Migration Ownership**
|
||||
- Each plugin owns its migration chain
|
||||
- Chains must be linear (no branches)
|
||||
- Migrations run in dependency order (PLUGIN_TABLE_OWNERS)
|
||||
- Violations cause production deployment failures
|
||||
|
||||
**Historical issues:**
|
||||
- Plugin migration branches blocked deployments
|
||||
- Circular dependencies caused deadlocks
|
||||
- Missing foreign keys led to data integrity issues
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. List All Migrations
|
||||
|
||||
Scan all plugin migration directories:
|
||||
|
||||
```bash
|
||||
find plugins/*/migrations/versions/ -name "*.py" -type f | sort
|
||||
```
|
||||
|
||||
**For each plugin:**
|
||||
- Count migration files
|
||||
- Check naming convention (revision_description.py)
|
||||
- Verify chronological order by timestamp
|
||||
|
||||
### 2. Check Chain Linearity
|
||||
|
||||
For each plugin, verify migration chain has no branches:
|
||||
|
||||
```python
|
||||
# Pseudo-code logic:
|
||||
for plugin in plugins:
|
||||
migrations = get_migrations(plugin)
|
||||
|
||||
# Build dependency graph
|
||||
graph = {}
|
||||
for migration in migrations:
|
||||
parent = migration.down_revision
|
||||
graph[parent] = graph.get(parent, []) + [migration.revision]
|
||||
|
||||
# Check for branches (parent with multiple children)
|
||||
for parent, children in graph.items():
|
||||
if len(children) > 1:
|
||||
ERROR: Branch detected at {parent}
|
||||
Children: {children}
|
||||
```
|
||||
|
||||
**What to flag:**
|
||||
- Multiple migrations with same down_revision (branch)
|
||||
- Migrations with no parent (orphaned)
|
||||
- Migrations with missing parent (broken chain)
|
||||
|
||||
**Example violation:**
|
||||
```
|
||||
ERROR: Plugin 'endpoints' has branched migration chain:
|
||||
Parent: a1b2c3d4
|
||||
Branch 1: e5f6g7h8 (Add certificate_thumbprint column)
|
||||
Branch 2: i9j0k1l2 (Add last_login_at column)
|
||||
|
||||
ADR-008 violation: Merge these branches or drop one
|
||||
```
|
||||
|
||||
### 3. Check Plugin Dependencies
|
||||
|
||||
Verify PLUGIN_TABLE_OWNERS coordination:
|
||||
|
||||
```bash
|
||||
grep -A 50 "PLUGIN_TABLE_OWNERS" shopdb/config.py
|
||||
```
|
||||
|
||||
**What to check:**
|
||||
- Plugins listed in dependency order
|
||||
- If Plugin A references Plugin B's table, B must come first
|
||||
- No circular dependencies (A depends on B, B depends on A)
|
||||
|
||||
**Build dependency graph:**
|
||||
```python
|
||||
# For each plugin migration:
|
||||
for migration in all_migrations:
|
||||
# Check for foreign keys to other plugins
|
||||
if references_other_plugin_table(migration):
|
||||
add_dependency(current_plugin, other_plugin)
|
||||
|
||||
# Check for cycles
|
||||
if has_circular_dependency(dependencies):
|
||||
ERROR: Circular dependency detected
|
||||
```
|
||||
|
||||
**Example violation:**
|
||||
```
|
||||
ERROR: Circular plugin dependencies:
|
||||
endpoints -> computers (fk: computer_id)
|
||||
computers -> endpoints (fk: endpoint_id)
|
||||
|
||||
ADR-008 violation: Break circular reference
|
||||
Recommendation: Create junction table or denormalize
|
||||
```
|
||||
|
||||
### 4. Validate Foreign Key References
|
||||
|
||||
For each foreign key in migrations, verify target table exists:
|
||||
|
||||
```python
|
||||
# Check op.create_foreign_key() calls
|
||||
for migration in all_migrations:
|
||||
for fk in get_foreign_keys(migration):
|
||||
target_table = fk.target_table
|
||||
target_plugin = get_owning_plugin(target_table)
|
||||
|
||||
# Verify target plugin comes before current in PLUGIN_TABLE_OWNERS
|
||||
if not comes_before(target_plugin, current_plugin):
|
||||
ERROR: Invalid foreign key order
|
||||
```
|
||||
|
||||
**What to flag:**
|
||||
- Foreign key to table owned by later plugin
|
||||
- Foreign key to non-existent table
|
||||
- Foreign key without index on source column
|
||||
|
||||
**Example violation:**
|
||||
```
|
||||
WARNING: Plugin 'printers' references 'computers' table
|
||||
Migration: 2024_03_15_add_computer_id.py
|
||||
Foreign key: printer.computer_id -> computer.id
|
||||
|
||||
PLUGIN_TABLE_OWNERS order:
|
||||
printers (index 5)
|
||||
computers (index 8)
|
||||
|
||||
ERROR: printers loads before computers (table won't exist)
|
||||
Fix: Move printers after computers in PLUGIN_TABLE_OWNERS
|
||||
```
|
||||
|
||||
### 5. Test Migrations Up
|
||||
|
||||
Apply all migrations in test database:
|
||||
|
||||
```bash
|
||||
# Create test database
|
||||
docker compose exec api flask db upgrade
|
||||
|
||||
# Upgrade each plugin
|
||||
docker compose exec api flask plugin upgrade-all
|
||||
```
|
||||
|
||||
**Monitor for:**
|
||||
- Alembic errors
|
||||
- Foreign key constraint violations
|
||||
- Circular dependency deadlocks
|
||||
- Duplicate table/column errors
|
||||
|
||||
If migrations fail:
|
||||
- Show exact error
|
||||
- Identify which migration failed
|
||||
- Suggest fix based on error type
|
||||
|
||||
### 6. Test Migrations Down
|
||||
|
||||
Verify migrations can roll back cleanly:
|
||||
|
||||
```bash
|
||||
# Downgrade plugins in reverse order
|
||||
for plugin in reverse(PLUGIN_TABLE_OWNERS):
|
||||
docker compose exec api flask plugin downgrade $plugin base
|
||||
done
|
||||
|
||||
# Downgrade core
|
||||
docker compose exec api flask db downgrade base
|
||||
```
|
||||
|
||||
**What to check:**
|
||||
- All downgrades succeed
|
||||
- No orphaned tables/columns
|
||||
- Foreign keys dropped before tables
|
||||
|
||||
**Common issues:**
|
||||
- Downgrade drops table but doesn't drop foreign keys first
|
||||
- Downgrade missing (NotImplementedError)
|
||||
- Downgrade order violation
|
||||
|
||||
### 7. Check for Duplicate Versions
|
||||
|
||||
Ensure no two migrations share same revision ID:
|
||||
|
||||
```bash
|
||||
# Extract all revision IDs
|
||||
grep -r "revision = " plugins/*/migrations/versions/*.py | cut -d"'" -f2 | sort | uniq -d
|
||||
```
|
||||
|
||||
**If duplicates found:**
|
||||
```
|
||||
ERROR: Duplicate revision IDs detected:
|
||||
a1b2c3d4 in plugins/endpoints/migrations/versions/001_initial.py
|
||||
a1b2c3d4 in plugins/computers/migrations/versions/003_add_field.py
|
||||
|
||||
Fix: Regenerate one migration with new revision ID
|
||||
```
|
||||
|
||||
### 8. Identify Orphaned Migrations
|
||||
|
||||
Find migrations with parents that don't exist:
|
||||
|
||||
```python
|
||||
# Build set of all revision IDs
|
||||
all_revisions = set(migration.revision for migration in all_migrations)
|
||||
|
||||
# Check each migration's parent exists
|
||||
for migration in all_migrations:
|
||||
if migration.down_revision and migration.down_revision not in all_revisions:
|
||||
ERROR: Orphaned migration {migration.revision}
|
||||
Parent {migration.down_revision} not found
|
||||
```
|
||||
|
||||
**Causes:**
|
||||
- Parent migration deleted
|
||||
- Migration from different branch merged incorrectly
|
||||
- Manual migration editing gone wrong
|
||||
|
||||
### 9. Summary Report
|
||||
|
||||
**[X] All checks passed:**
|
||||
```
|
||||
Migration safety check complete:
|
||||
|
||||
Plugin Migration Chains:
|
||||
[X] endpoints: 12 migrations, linear
|
||||
[X] computers: 8 migrations, linear
|
||||
[X] printers: 5 migrations, linear
|
||||
[X] (... all plugins ...)
|
||||
|
||||
Dependency Analysis:
|
||||
[X] No circular dependencies
|
||||
[X] All foreign keys valid
|
||||
[X] PLUGIN_TABLE_OWNERS order correct
|
||||
|
||||
Migration Tests:
|
||||
[X] All migrations applied successfully
|
||||
[X] All downgrades succeeded
|
||||
[X] No duplicate revision IDs
|
||||
|
||||
Migrations are safe to deploy!
|
||||
```
|
||||
|
||||
**WARNING: Issues detected:**
|
||||
```
|
||||
Migration safety check FAILED:
|
||||
|
||||
CRITICAL ISSUES (block deployment):
|
||||
1. Plugin 'endpoints' has branched migration chain
|
||||
Parent a1b2c3d4 has 2 children
|
||||
Action: Merge branches or drop conflicting migration
|
||||
|
||||
2. Circular dependency: endpoints -> computers
|
||||
Action: Break circular reference with junction table
|
||||
|
||||
3. Foreign key order violation: printers -> computers
|
||||
printers loads before computers in PLUGIN_TABLE_OWNERS
|
||||
Action: Reorder plugins or remove foreign key
|
||||
|
||||
HIGH PRIORITY (should fix before deploy):
|
||||
4. Migration 'endpoints/003_add_field' has no downgrade
|
||||
Action: Implement down() method
|
||||
|
||||
5. Orphaned migration: computers/005_update
|
||||
Parent revision not found
|
||||
Action: Fix down_revision or remove migration
|
||||
|
||||
WARNINGS (review but not blocking):
|
||||
6. Large migration: endpoints/012_backfill (may be slow)
|
||||
Action: Consider data migration strategy
|
||||
|
||||
DO NOT DEPLOY until critical issues resolved!
|
||||
|
||||
Detailed logs: /tmp/migration-check-{timestamp}.log
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Branch detected:**
|
||||
- Option 1: Squash branch into single migration
|
||||
- Option 2: Drop conflicting branch
|
||||
- Option 3: Reorder to make linear
|
||||
|
||||
**Circular dependency:**
|
||||
- Create junction table (owned by one plugin)
|
||||
- Denormalize (copy data instead of FK)
|
||||
- Move table ownership to core
|
||||
|
||||
**Foreign key order violation:**
|
||||
- Reorder PLUGIN_TABLE_OWNERS
|
||||
- Remove foreign key (use app-level relationship)
|
||||
- Move both tables to same plugin
|
||||
|
||||
**Migration fails to apply:**
|
||||
- Check for missing dependencies
|
||||
- Verify table/column doesn't already exist
|
||||
- Check for data that violates constraints
|
||||
|
||||
## Related
|
||||
|
||||
- `/pre-commit` - Includes basic migration linearity check
|
||||
- `/production-health` - Post-deployment validation
|
||||
- ADR-008: Plugin migration ownership
|
||||
- `PLUGIN_TABLE_OWNERS` in shopdb/config.py
|
||||
- Alembic documentation: https://alembic.sqlalchemy.org/
|
||||
Reference in New Issue
Block a user