Clean up sql directory after production sync
- Remove 27 completed one-off migration scripts - Remove old backup file (dev-backup-20251120) - Remove completed shell scripts for prod import/export - Archive migration_phase1-4 directories and documentation - Keep only view_consolidation.sql as active script All migrations have been applied to production. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
cproudlock
499
sql/archive/migration_phase2/README.md
Normal file
499
sql/archive/migration_phase2/README.md
Normal file
@@ -0,0 +1,499 @@
|
||||
# Phase 2: PC Data Migration
|
||||
|
||||
## Overview
|
||||
|
||||
**Phase 2** migrates all PC data from the legacy `pc` table and related tables into the new unified schema created in Phase 1. This phase handles the actual data migration of **277 active PCs** and their associated records.
|
||||
|
||||
**Status**: Ready for testing on DEV environment
|
||||
**Estimated Time**: 10-15 minutes
|
||||
**Dependencies**: Phase 1 must be completed first
|
||||
|
||||
## What This Phase Does
|
||||
|
||||
Phase 2 migrates data from these legacy tables:
|
||||
- `pc` (277 records) → `machines` table
|
||||
- `pc_network_interfaces` (705 records) → `communications` table
|
||||
- `pc_comm_config` → `communications` table
|
||||
- `pc` warranty fields → `warranties` table
|
||||
- `pc_dualpath_assignments` → `machinerelationships` table
|
||||
|
||||
## Migration Scripts
|
||||
|
||||
### Script Execution Order
|
||||
|
||||
Run scripts in this exact order:
|
||||
|
||||
| Script | Purpose | Records | Time |
|
||||
|--------|---------|---------|------|
|
||||
| **01_migrate_pcs_to_machines.sql** | Migrate PC records to machines table | ~277 PCs | 5-10 min |
|
||||
| **02_migrate_network_interfaces_to_communications.sql** | Migrate network interfaces | ~705 interfaces | 3-5 min |
|
||||
| **03_migrate_comm_config_to_communications.sql** | Migrate serial/IP comm configs | Variable | 2-3 min |
|
||||
| **04_migrate_warranties.sql** | Migrate PC warranty data | Variable | 1-2 min |
|
||||
| **05_migrate_dualpath_assignments.sql** | Migrate dualpath machine relationships | Variable | 1-2 min |
|
||||
| **VERIFY_PHASE2_MIGRATION.sql** | Comprehensive verification | N/A | 1-2 min |
|
||||
|
||||
### Script Details
|
||||
|
||||
#### 01_migrate_pcs_to_machines.sql
|
||||
**Critical Script - All other scripts depend on this**
|
||||
|
||||
Migrates all active PC records from `pc` table to `machines` table:
|
||||
|
||||
- Creates `pc_to_machine_id_mapping` table for tracking old→new IDs
|
||||
- Maps PC types to new machine types:
|
||||
- Standard → PC - Standard
|
||||
- Shopfloor → PC - Shopfloor
|
||||
- Engineer → PC - Engineer
|
||||
- Server/VM → PC - Server
|
||||
- Laptop → PC - Laptop
|
||||
- Sets `machinenumber` = `hostname` for PCs
|
||||
- Creates backup: `pc_backup_phase2`
|
||||
|
||||
**Key Fields Migrated**:
|
||||
- hostname, loggedinuser, serialnumber
|
||||
- modelnumberid, osid, machinestatusid, pctypeid
|
||||
- requires_manual_machine_config
|
||||
- businessunitid, printerid (if exists)
|
||||
- mapleft, maptop (if exists)
|
||||
|
||||
#### 02_migrate_network_interfaces_to_communications.sql
|
||||
|
||||
Migrates PC network interfaces to the unified `communications` table:
|
||||
|
||||
- Uses `comstypeid` for 'Network_Interface'
|
||||
- Maps `ipaddress` → `address` field
|
||||
- Preserves: macaddress, subnetmask, defaultgateway
|
||||
- Preserves: interfacename, isdhcp, ismachinenetwork
|
||||
- Creates backup: `pc_network_interfaces_backup_phase2`
|
||||
|
||||
#### 03_migrate_comm_config_to_communications.sql
|
||||
|
||||
Migrates serial and IP communication configs:
|
||||
|
||||
- Maps Serial configs: `portid` → `address`, preserves baud/databits/stopbits/parity
|
||||
- Maps IP configs: `ipaddress` → `address`, `socketnumber` → `port`
|
||||
- Combines `crlf` + `additionalsettings` into `settings` field
|
||||
- Creates backup: `pc_comm_config_backup_phase2`
|
||||
|
||||
#### 04_migrate_warranties.sql
|
||||
|
||||
Migrates warranty data from PC table to normalized `warranties` table:
|
||||
|
||||
- Only migrates PCs with `warrantyenddate` NOT NULL
|
||||
- Maps fields:
|
||||
- `warrantyenddate` → `enddate`
|
||||
- `warrantyservicelevel` → `servicelevel`
|
||||
- `warrantylastchecked` → `lastcheckeddate`
|
||||
- `warrantyname` set to NULL (not in old schema)
|
||||
|
||||
#### 05_migrate_dualpath_assignments.sql
|
||||
|
||||
Migrates dualpath machine relationships:
|
||||
|
||||
- Creates **bidirectional** relationships:
|
||||
- primary_machine → secondary_machine
|
||||
- secondary_machine → primary_machine
|
||||
- Uses `relationshiptype` = 'Dualpath'
|
||||
- Joins on `machinenumber` (hostname) to find machine IDs
|
||||
- Creates backup: `pc_dualpath_assignments_backup_phase2`
|
||||
|
||||
#### VERIFY_PHASE2_MIGRATION.sql
|
||||
|
||||
Comprehensive verification script with 9 sections:
|
||||
|
||||
1. PC to machines migration verification
|
||||
2. Network interfaces migration verification
|
||||
3. Comm config migration verification
|
||||
4. Warranties migration verification
|
||||
5. Dualpath assignments verification
|
||||
6. Overall communications summary
|
||||
7. Relationship summary
|
||||
8. Backup table verification
|
||||
9. Final summary and critical issues check
|
||||
|
||||
All checks should show status with 0 critical issues.
|
||||
|
||||
## How to Run
|
||||
|
||||
### Option A: Run Individual Scripts (Recommended for Testing)
|
||||
|
||||
```bash
|
||||
# Connect to MySQL
|
||||
mysql -u root -p shopdb
|
||||
|
||||
# Run scripts in order
|
||||
source 01_migrate_pcs_to_machines.sql
|
||||
source 02_migrate_network_interfaces_to_communications.sql
|
||||
source 03_migrate_comm_config_to_communications.sql
|
||||
source 04_migrate_warranties.sql
|
||||
source 05_migrate_dualpath_assignments.sql
|
||||
|
||||
# Verify migration
|
||||
source VERIFY_PHASE2_MIGRATION.sql
|
||||
```
|
||||
|
||||
### Option B: Run All Scripts at Once
|
||||
|
||||
```bash
|
||||
# Run all scripts and capture output
|
||||
mysql -u root -p shopdb < 01_migrate_pcs_to_machines.sql > /tmp/phase2_01.log 2>&1
|
||||
mysql -u root -p shopdb < 02_migrate_network_interfaces_to_communications.sql > /tmp/phase2_02.log 2>&1
|
||||
mysql -u root -p shopdb < 03_migrate_comm_config_to_communications.sql > /tmp/phase2_03.log 2>&1
|
||||
mysql -u root -p shopdb < 04_migrate_warranties.sql > /tmp/phase2_04.log 2>&1
|
||||
mysql -u root -p shopdb < 05_migrate_dualpath_assignments.sql > /tmp/phase2_05.log 2>&1
|
||||
mysql -u root -p shopdb < VERIFY_PHASE2_MIGRATION.sql > /tmp/phase2_verify.log 2>&1
|
||||
|
||||
# Review logs
|
||||
cat /tmp/phase2_*.log
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Phase 1 Must Be Complete
|
||||
|
||||
Before running Phase 2, verify Phase 1 is complete:
|
||||
|
||||
```sql
|
||||
-- Should return 8 tables
|
||||
SELECT COUNT(*) FROM information_schema.tables
|
||||
WHERE table_schema = 'shopdb'
|
||||
AND table_name IN (
|
||||
'comstypes', 'communications', 'warranties', 'compliance',
|
||||
'relationshiptypes', 'machinerelationships', 'machinestatus'
|
||||
);
|
||||
|
||||
-- Should return rows
|
||||
SELECT COUNT(*) FROM comstypes;
|
||||
SELECT COUNT(*) FROM relationshiptypes;
|
||||
|
||||
-- machines table should have new columns
|
||||
DESCRIBE machines; -- Check for hostname, osid, pctypeid, etc.
|
||||
```
|
||||
|
||||
### Database Backup
|
||||
|
||||
**CRITICAL**: Create a full database backup before running Phase 2:
|
||||
|
||||
```bash
|
||||
# Create backup
|
||||
mysqldump -u root -p shopdb > /tmp/shopdb-before-phase2-$(date +%Y%m%d-%H%M%S).sql
|
||||
|
||||
# Verify backup
|
||||
ls -lh /tmp/shopdb-before-phase2-*.sql
|
||||
```
|
||||
|
||||
## What Gets Created
|
||||
|
||||
### New Records
|
||||
|
||||
- **277 new machine records** (PCs migrated to machines table)
|
||||
- **~705 communication records** (network interfaces)
|
||||
- **Variable communication records** (serial/IP configs)
|
||||
- **Variable warranty records** (PCs with warranty dates)
|
||||
- **Variable relationship records** (dualpath assignments × 2)
|
||||
|
||||
### Mapping Tables
|
||||
|
||||
- **pc_to_machine_id_mapping**: Maps old `pcid` to new `machineid`
|
||||
- Used by scripts 02-05
|
||||
- Used by Phase 3 view updates
|
||||
- Used by Phase 4 application updates
|
||||
- **DO NOT DELETE** until migration fully complete
|
||||
|
||||
### Backup Tables
|
||||
|
||||
Phase 2 creates these backup tables:
|
||||
|
||||
- `pc_backup_phase2` (277 records)
|
||||
- `pc_network_interfaces_backup_phase2` (~705 records)
|
||||
- `pc_comm_config_backup_phase2`
|
||||
- `pc_dualpath_assignments_backup_phase2`
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After running all scripts, verify:
|
||||
|
||||
### 1. Record Counts Match
|
||||
|
||||
```sql
|
||||
-- PCs migrated
|
||||
SELECT COUNT(*) FROM pc WHERE isactive = 1; -- Original count
|
||||
SELECT COUNT(*) FROM machines WHERE pctypeid IS NOT NULL; -- Should match
|
||||
|
||||
-- Network interfaces migrated
|
||||
SELECT COUNT(*) FROM pc_network_interfaces WHERE isactive = 1; -- Original
|
||||
SELECT COUNT(*) FROM communications c
|
||||
JOIN comstypes ct ON c.comstypeid = ct.comstypeid
|
||||
WHERE ct.typename = 'Network_Interface'; -- Should match
|
||||
|
||||
-- Mapping table populated
|
||||
SELECT COUNT(*) FROM pc_to_machine_id_mapping; -- Should equal PC count
|
||||
```
|
||||
|
||||
### 2. No NULL Values in Critical Fields
|
||||
|
||||
```sql
|
||||
-- Check for data quality issues
|
||||
SELECT COUNT(*) FROM machines
|
||||
WHERE pctypeid IS NOT NULL
|
||||
AND (hostname IS NULL OR machinetypeid IS NULL); -- Should be 0
|
||||
|
||||
SELECT COUNT(*) FROM communications
|
||||
WHERE machineid IS NULL OR comstypeid IS NULL; -- Should be 0
|
||||
```
|
||||
|
||||
### 3. Dualpath Relationships Bidirectional
|
||||
|
||||
```sql
|
||||
-- Check for one-way relationships (should be 0)
|
||||
SELECT COUNT(*) FROM machinerelationships mr1
|
||||
JOIN relationshiptypes rt ON mr1.relationshiptypeid = rt.relationshiptypeid
|
||||
WHERE rt.relationshiptype = 'Dualpath'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM machinerelationships mr2
|
||||
WHERE mr2.machineid = mr1.related_machineid
|
||||
AND mr2.related_machineid = mr1.machineid
|
||||
AND mr2.relationshiptypeid = mr1.relationshiptypeid
|
||||
);
|
||||
```
|
||||
|
||||
### 4. All Backup Tables Created
|
||||
|
||||
```sql
|
||||
-- Should return 4 rows
|
||||
SELECT table_name, table_rows
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema = 'shopdb'
|
||||
AND table_name LIKE '%_backup_phase2';
|
||||
```
|
||||
|
||||
### 5. Run Full Verification
|
||||
|
||||
```bash
|
||||
mysql -u root -p shopdb < VERIFY_PHASE2_MIGRATION.sql
|
||||
```
|
||||
|
||||
Review output for:
|
||||
- All checks showing status
|
||||
- Critical issues count = 0
|
||||
- Record counts matching expected values
|
||||
|
||||
## Rollback Procedures
|
||||
|
||||
### Full Rollback
|
||||
|
||||
If Phase 2 needs to be completely rolled back:
|
||||
|
||||
```sql
|
||||
-- 1. Delete migrated PC records from machines
|
||||
DELETE FROM machines WHERE pctypeid IS NOT NULL;
|
||||
|
||||
-- 2. Delete communications from PCs
|
||||
DELETE FROM communications
|
||||
WHERE machineid IN (SELECT new_machineid FROM pc_to_machine_id_mapping);
|
||||
|
||||
-- 3. Delete warranties for PCs
|
||||
DELETE FROM warranties
|
||||
WHERE machineid IN (SELECT new_machineid FROM pc_to_machine_id_mapping);
|
||||
|
||||
-- 4. Delete dualpath relationships
|
||||
DELETE FROM machinerelationships
|
||||
WHERE relationshiptypeid = (
|
||||
SELECT relationshiptypeid FROM relationshiptypes
|
||||
WHERE relationshiptype = 'Dualpath'
|
||||
);
|
||||
|
||||
-- 5. Drop mapping table
|
||||
DROP TABLE IF EXISTS pc_to_machine_id_mapping;
|
||||
|
||||
-- 6. Verify original data intact
|
||||
SELECT COUNT(*) FROM pc WHERE isactive = 1;
|
||||
SELECT COUNT(*) FROM pc_network_interfaces WHERE isactive = 1;
|
||||
SELECT COUNT(*) FROM pc_comm_config;
|
||||
```
|
||||
|
||||
### Restore from Database Backup
|
||||
|
||||
If full rollback needed:
|
||||
|
||||
```bash
|
||||
# Stop applications using database
|
||||
# ...
|
||||
|
||||
# Restore from backup
|
||||
mysql -u root -p shopdb < /tmp/shopdb-before-phase2-YYYYMMDD-HHMMSS.sql
|
||||
|
||||
# Verify restoration
|
||||
mysql -u root -p shopdb -e "SELECT COUNT(*) FROM pc; SELECT COUNT(*) FROM machines;"
|
||||
```
|
||||
|
||||
## Known Issues and Considerations
|
||||
|
||||
### 1. Hostname as Machine Number
|
||||
|
||||
PCs use `hostname` as `machinenumber` in the machines table. This differs from other equipment that uses numeric machine numbers (e.g., "M-1234").
|
||||
|
||||
**Rationale**: PCs are identified by hostname in the organization. Using hostname preserves this convention.
|
||||
|
||||
### 2. NULL Warranty Names
|
||||
|
||||
The old `pc` table didn't have a `warrantyname` field, so migrated PC warranties will have `warrantyname = NULL`. This is expected and correct.
|
||||
|
||||
### 3. Business Unit Assignment
|
||||
|
||||
Script 01 attempts to preserve existing `businessunitid` if the PC already exists in machines table. Otherwise defaults to businessunitid = 1.
|
||||
|
||||
**Post-migration**: Review and update business unit assignments as needed.
|
||||
|
||||
### 4. Communication Type Mapping
|
||||
|
||||
The old schema used string-based `configtype` ('Serial', 'IP'). The new schema uses foreign keys to `comstypes` table. Mapping is:
|
||||
|
||||
- `configtype = 'Serial'` → comstypeid for 'Serial'
|
||||
- `configtype = 'IP'` → comstypeid for 'IP'
|
||||
- Network interfaces → comstypeid for 'Network_Interface'
|
||||
|
||||
### 5. MySQL 5.6 Compatibility
|
||||
|
||||
All scripts are tested and compatible with MySQL 5.6. Phase 1 script 03 was updated to use `FROM DUAL` syntax for MySQL 5.6 compatibility.
|
||||
|
||||
## Testing Recommendations
|
||||
|
||||
### DEV Environment Testing
|
||||
|
||||
1. **Create backup** before testing
|
||||
2. **Run Phase 2 scripts** individually
|
||||
3. **Review each script output** for errors
|
||||
4. **Run verification script**
|
||||
5. **Test application** (check PC pages still work)
|
||||
6. **Test rollback** procedure
|
||||
7. **Restore from backup** to test restoration
|
||||
8. **Document any issues** found
|
||||
|
||||
### Application Testing
|
||||
|
||||
After Phase 2, test these application areas:
|
||||
|
||||
- PC list pages
|
||||
- PC detail pages
|
||||
- Network interface displays
|
||||
- Warranty reports
|
||||
- Dualpath machine displays
|
||||
- Any reports using PC data
|
||||
|
||||
**Note**: Application code still uses old `pc` table. Phase 3 and 4 will update views and application code.
|
||||
|
||||
## Timeline
|
||||
|
||||
### Phase 2 Execution Timeline
|
||||
|
||||
- **Script execution**: 10-15 minutes
|
||||
- **Verification**: 5-10 minutes
|
||||
- **Application testing**: 1-2 hours
|
||||
- **Total**: ~2-3 hours
|
||||
|
||||
### Overall Migration Timeline
|
||||
|
||||
- **Phase 1**: Schema changes (COMPLETE)
|
||||
- **Phase 2**: Data migration (CURRENT - Ready for testing)
|
||||
- **Phase 3**: View updates (Pending)
|
||||
- **Phase 4**: Application code updates (Pending)
|
||||
|
||||
**Estimated total**: 20-30 days from design to production deployment
|
||||
|
||||
## Support and Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Issue: Script 01 fails with mapping error
|
||||
|
||||
**Symptom**: Error joining pc to machines on machinenumber
|
||||
**Cause**: Duplicate hostnames in pc or machines tables
|
||||
**Fix**: Investigate duplicates:
|
||||
|
||||
```sql
|
||||
-- Find duplicate hostnames in pc
|
||||
SELECT hostname, COUNT(*)
|
||||
FROM pc WHERE isactive = 1
|
||||
GROUP BY hostname HAVING COUNT(*) > 1;
|
||||
|
||||
-- Find duplicate hostnames in machines
|
||||
SELECT hostname, COUNT(*)
|
||||
FROM machines WHERE isactive = 1
|
||||
GROUP BY hostname HAVING COUNT(*) > 1;
|
||||
```
|
||||
|
||||
#### Issue: Communications migration shows wrong counts
|
||||
|
||||
**Symptom**: Verification shows fewer communications than expected
|
||||
**Cause**: Missing `pc_to_machine_id_mapping` entries
|
||||
**Fix**: Check mapping table:
|
||||
|
||||
```sql
|
||||
-- Find unmapped PCs
|
||||
SELECT p.pcid, p.hostname
|
||||
FROM pc p
|
||||
WHERE p.isactive = 1
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM pc_to_machine_id_mapping m WHERE m.pcid = p.pcid
|
||||
);
|
||||
```
|
||||
|
||||
#### Issue: Dualpath relationships not bidirectional
|
||||
|
||||
**Symptom**: Verification shows one-way relationships
|
||||
**Cause**: Script 05 only ran first INSERT, not second
|
||||
**Fix**: Manually run second INSERT block from script 05
|
||||
|
||||
### Getting Help
|
||||
|
||||
- Review VERIFY_PHASE2_MIGRATION.sql output
|
||||
- Check backup tables for data comparison
|
||||
- Review script comments and NOTES sections
|
||||
- Consult PC_MACHINES_CONSOLIDATION_PLAN.md for design rationale
|
||||
|
||||
## Next Steps
|
||||
|
||||
After Phase 2 is complete and verified:
|
||||
|
||||
1. **Phase 3**: Update 19 views that reference `pc` table
|
||||
- Views need to query `machines` table instead
|
||||
- Some views may need UNION with both tables during transition
|
||||
|
||||
2. **Phase 4**: Update ASP application files (50+ files estimated)
|
||||
- Update queries to use `machines` table
|
||||
- Update stored procedures
|
||||
- Update reports
|
||||
|
||||
3. **Production Deployment**
|
||||
- Schedule maintenance window
|
||||
- Execute on production database
|
||||
- Monitor application performance
|
||||
|
||||
## File Manifest
|
||||
|
||||
```
|
||||
migration_phase2/
|
||||
├── README.md (this file)
|
||||
├── 01_migrate_pcs_to_machines.sql (354 lines)
|
||||
├── 02_migrate_network_interfaces_to_communications.sql (201 lines)
|
||||
├── 03_migrate_comm_config_to_communications.sql (213 lines)
|
||||
├── 04_migrate_warranties.sql (165 lines)
|
||||
├── 05_migrate_dualpath_assignments.sql (178 lines)
|
||||
└── VERIFY_PHASE2_MIGRATION.sql (562 lines)
|
||||
|
||||
Total: 7 files, 1,673 lines
|
||||
```
|
||||
|
||||
## Change Log
|
||||
|
||||
| Date | Version | Changes |
|
||||
|------|---------|---------|
|
||||
| 2025-11-06 | 1.0 | Initial Phase 2 scripts created |
|
||||
|
||||
---
|
||||
|
||||
**Author**: Database Migration Team
|
||||
**Date**: 2025-11-06
|
||||
**Status**: Ready for DEV testing
|
||||
**Next Review**: After DEV testing complete
|
||||
Reference in New Issue
Block a user