Reorganize repository structure and clean up documentation

Clean up and organize the flask-shopdb repository for better maintainability and discoverability: ## Changes - Update .gitignore for build artifacts (publish/, data/, 122M total) - Reorganize .claude/ into 6 subdirectories with README navigation - Consolidate overlapping MCP setup guides into MCP-QUICK-START.md - Move deployment docs from root to docs/deployment/ - Archive dated status reports to .claude/archived/ - Delete temporary files (logs, backups, empty reports, ~50K) ## New Structure .claude/ organized into: - setup/ - MCP, VM, Hyper-V, GitHub setup guides + scripts (17 files) - plans/ - Feature plans and protocols (5 files) - maintenance/ - Deployment checklists and reports (4 files) - archived/ - Historical documentation (9 files) - skills/ - Claude Code automation (5 files) - workflows/ - Security audit workflow (1 file) docs/deployment/ created with 6 deployment guides: - LOCAL-DEV-SETUP.md, DOCKER-SETUP.md, DOCKER-PREBUILT-APPROACH.md - PRODUCTION-READINESS.md, DEPLOY-TO-VM.md, README.md Root directory cleaned to essential docs only: - CLAUDE.md, README.md, CONTRIBUTING.md, CHANGELOG.md Benefits: - Clear navigation via README.md in each subdirectory - No duplicate/overlapping documentation - Large artifacts properly gitignored (won't bloat repository) - Historical context preserved in archived/ See .claude/CLEANUP-SUMMARY.md for complete details. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-09-11 15:45:05 -04:00
parent 0b252db7d2
commit bd48814021
57 changed files with 11978 additions and 29 deletions
--- a/docs/deployment/PRODUCTION-READINESS.md
+++ b/docs/deployment/PRODUCTION-READINESS.md
@@ -0,0 +1,345 @@
+# ShopDB Production Readiness Assessment
+
+**Last Updated:** 2026-09-07  
+**Branch:** wj (post-wilm merge)
+
+## Executive Summary
+
+[CRITICAL] **NOT PRODUCTION READY** in current state without significant operational investment.
+
+The Docker deployment requires extensive manual setup, multiple services, and assumes expertise with Docker Compose, restic backup systems, and Flask plugin architecture. This assessment documents current gaps and provides recommendations.
+
+## Current State: Deployment Complexity
+
+### What Works
+
+[OK] **Flask application code** - Stable, tested, 17 plugins functional  
+[OK] **Database schema** - Migrations work, data model is sound  
+[OK] **Frontend build** - Vue 3 SPA compiles and serves correctly  
+[OK] **Docker images** - Build successfully (with caveats below)  
+[OK] **Plugin architecture** - Clean, documented, extensible  
+
+### Critical Gaps
+
+[FAIL] **One-command deployment** - Requires 7-10 manual steps  
+[FAIL] **Corporate proxy handling** - Zscaler cert extraction needed for npm  
+[FAIL] **Backup system complexity** - 3 containers + restic + scheduler  
+[FAIL] **TLS certificate generation** - Manual OpenSSL commands required  
+[FAIL] **Plugin initialization** - Must run `apply-profile` after first start  
+[FAIL] **Password reset** - No default admin password, requires manual SQL  
+[FAIL] **Build time** - 15+ minutes with corporate proxy  
+[FAIL] **Documentation scattered** - 3 different setup guides (Docker, Local, Backup)
+
+## Why Is This So Complex?
+
+### 1. Corporate Network Reality
+
+**Problem:** Behind Zscaler proxy, npm can't trust npm registry  
+**Impact:** Docker build fails without extracted corporate root CA  
+**Workaround:** PowerShell script to extract cert before building  
+**Production Fix:** Pre-build images in CI/CD with cert, publish to registry
+
+### 2. Plugin Architecture Design
+
+**Problem:** Plugin state in `instance/plugins.json`, not database  
+**Impact:** Importing prod database doesn't register plugins  
+**Workaround:** Manual `flask plugin apply-profile` after first start  
+**Production Fix:** Auto-detect installed plugins from schema on boot
+
+### 3. Backup System Architecture
+
+**Problem:** Designed for restic with 3-container orchestration  
+**Impact:** Requires backup-store + backup-watcher + backup containers  
+**Workaround:** All documented, but complex to deploy  
+**Production Fix:** Simpler mysqldump-only mode with direct file storage
+
+### 4. Multi-Stage Docker Build
+
+**Problem:** Frontend build needs Node 20, backend needs Python 3.14  
+**Impact:** Multi-stage build is slow, complex, hard to debug  
+**Workaround:** Build works, but takes 15 minutes  
+**Production Fix:** Pre-build frontend on CI, copy dist/ into Python-only image
+
+## Deployment Models (Ordered by Simplicity)
+
+### Option 1: IIS + Windows Services (SIMPLEST)
+
+**Best for:** Single Windows server deployment  
+**Complexity:** Low  
+**Maintenance:** Medium  
+
+**Setup:**
+1. Install Python 3.13+
+2. Install MySQL (or use existing instance)
+3. Run installer (creates Windows service for Flask)
+4. Configure IIS reverse proxy
+5. Install plugins via UI
+
+**Pros:**
+- Native Windows deployment
+- Familiar IIS management
+- No Docker complexity
+- Fast startup
+
+**Cons:**
+- Windows-only
+- Manual updates
+- No containerization benefits
+
+**Production Ready:** [WARN] **YES** (with installer improvements)
+
+### Option 2: Docker Pre-Built Images (RECOMMENDED)
+
+**Best for:** Sites with Docker but no build capability  
+**Complexity:** Low-Medium  
+**Maintenance:** Low  
+
+**Setup:**
+1. Pull pre-built image from registry
+2. Run docker-compose with .env file
+3. Access UI, enable plugins
+
+**Pros:**
+- No build step
+- Fast deployment (2 minutes)
+- Consistent environment
+- Easy updates (pull new image)
+
+**Cons:**
+- Requires Docker
+- Need image registry
+- Initial registry setup
+
+**Production Ready:** [OK] **YES** (requires CI/CD setup)
+
+### Option 3: Docker Source Build (CURRENT)
+
+**Best for:** Development, customization  
+**Complexity:** High  
+**Maintenance:** High  
+
+**Setup:**
+1. Extract Zscaler cert
+2. Generate TLS certs
+3. Build Docker images (15+ min)
+4. Configure .env with 15+ variables
+5. Start docker-compose
+6. Run migrations
+7. Install plugins
+8. Reset admin password
+9. Configure backup system
+10. Test everything
+
+**Pros:**
+- Full source control
+- Customizable
+- No external dependencies
+
+**Cons:**
+- Extremely complex
+- Long build times
+- Many failure points
+- Requires Docker expertise
+
+**Production Ready:** [CRITICAL] **NO** (too complex for operations)
+
+### Option 4: Kubernetes (FUTURE)
+
+**Best for:** Multi-site deployment, high availability  
+**Complexity:** Very High  
+**Maintenance:** Medium (with proper GitOps)  
+
+**Status:** Not implemented  
+**Effort:** 2-3 weeks of work  
+
+## What Would Make This Production Ready?
+
+### Critical Path (Must Have)
+
+1. **Pre-Built Images in Registry**
+   - Build images in CI/CD
+   - Push to container registry
+   - Sites just `docker pull`
+   - **Effort:** 1 week
+
+2. **One-Command Setup**
+   - `./deploy.sh` does everything
+   - No manual cert extraction
+   - No manual plugin installation
+   - **Effort:** 3 days
+
+3. **Automated Plugin Detection**
+   - Scan database schema on boot
+   - Auto-register plugins in JSON
+   - No manual `apply-profile`
+   - **Effort:** 2 days
+
+4. **Simplified Backup (Option)**
+   - Simple mysqldump + tar mode
+   - Single container, no restic
+   - Good enough for 80% of sites
+   - **Effort:** 1 week
+
+5. **Windows Installer Improvements**
+   - One-click install
+   - Includes Python runtime
+   - Creates service automatically
+   - **Effort:** 1 week
+
+### Nice to Have (Future)
+
+- Helm chart for Kubernetes
+- Monitoring/metrics built-in
+- Health check endpoints
+- Automated testing in CI
+- Rolling update strategy
+
+## Honest Recommendations
+
+### For West Jefferson (Your Site)
+
+**Recommended:** Option 1 (IIS + Windows Services)
+
+**Why:**
+- You already have Windows infrastructure
+- Simpler to operate
+- Faster to deploy
+- Your team knows IIS
+
+**Steps:**
+1. Build Windows installer
+2. Deploy to Windows Server
+3. Configure IIS reverse proxy
+4. Import prod database
+5. Done
+
+**Timeline:** 2-3 days including testing
+
+### For New Sites
+
+**Recommended:** Option 2 (Docker Pre-Built Images)
+
+**Prerequisites:**
+- Set up CI/CD to build images
+- Set up container registry (Docker Hub, Harbor, or AWS ECR)
+- Create `deploy.sh` script
+
+**Then sites can:**
+```bash
+# 1. Copy .env.example to .env and edit
+cp .env.example .env
+
+# 2. Run deployment
+./deploy.sh
+
+# 3. Access UI
+https://your-server:5001
+```
+
+**Timeline for CI/CD setup:** 1-2 weeks
+
+### For Current Docker Source Build
+
+**Status:** Development/customization only, not for production
+
+**Use when:**
+- Testing new features
+- Developing plugins
+- Debugging issues
+- Learning the system
+
+**Don't use when:**
+- Deploying to operations
+- Training new ops staff
+- Time-sensitive deployments
+
+## Current Documentation Issues
+
+### Too Many Guides
+
+- `LOCAL-DEV-SETUP.md` - Windows development (port 3307, run_dev.py)
+- `DOCKER-SETUP.md` - Docker source build (complex, 15+ steps)
+- `PRODUCTION-READINESS.md` - This file (meta/assessment)
+- `scripts/backup/README.md` - Backup system (3 containers, restic)
+
+**Problem:** Which one do I follow?
+
+**Solution:** Create decision tree:
+- Developer on Windows -> LOCAL-DEV-SETUP.md
+- Ops deploying to prod -> (DOESN'T EXIST YET - need simplified guide)
+- Custom development -> DOCKER-SETUP.md
+- Backup questions -> scripts/backup/README.md
+
+### Missing Guides
+
+- **PRODUCTION-DEPLOY.md** - Step-by-step for operations (IIS or Docker pull)
+- **QUICK-START.md** - 5-minute version for evaluation
+- **TROUBLESHOOTING.md** - Common issues, not scattered across files
+- **UPGRADE-GUIDE.md** - How to update between versions
+
+## What We Did Today (Honest Assessment)
+
+[OK] **Merged wilm->wj** - 98 commits, wilm features now in wj  
+[OK] **Fixed local dev** - `run_dev.py` works for Windows development  
+[OK] **Documented Docker** - Comprehensive but too complex  
+[OK] **Added backup docs** - System works but needs expertise  
+[WARN] **Docker "production ready"** - Technically works, operationally complex  
+[FAIL] **Simple deployment** - Still requires too many manual steps  
+
+## Next Steps (Priority Order)
+
+### Immediate (This Week)
+
+1. **Create QUICK-START.md**
+   - 5-minute eval version
+   - Uses pre-built image (if available) OR local dev
+   - Gets people to working system fast
+
+2. **Test Windows Installer**
+   - Build installer for wj branch
+   - Deploy to test Windows Server
+   - Document any issues
+
+3. **Create deployment decision tree**
+   - Flowchart: what's my situation -> which method to use
+   - Don't make users read 4 docs to figure out where to start
+
+### Short Term (Next Sprint)
+
+4. **Set up CI/CD for image builds**
+   - GitHub Actions or Jenkins
+   - Build on every push to wj
+   - Push to internal registry
+
+5. **Create `deploy.sh` script**
+   - One command: `./deploy.sh --env prod`
+   - Handles certs, plugins, migrations automatically
+
+6. **Simplify backup system**
+   - Add "simple mode" - just mysqldump + tar
+   - Keep restic for advanced sites
+   - Document tradeoffs
+
+### Long Term (Next Month)
+
+7. **Auto-plugin detection**
+   - Boot-time schema scan
+   - No manual `apply-profile`
+
+8. **Consolidate documentation**
+   - One entry point
+   - Clear paths for each audience
+   - Tested by someone who hasn't seen the system
+
+## Conclusion
+
+**Is ShopDB production-ready?**
+
+- **The code:** Yes, it's solid
+- **Docker source build deployment:** No, too complex
+- **Windows installer deployment:** Yes, with improvements
+- **Docker pre-built images deployment:** Yes, but needs CI/CD setup
+
+**Bottom line:** The application is mature. The deployment story needs work. Focus should shift from "making Docker work" to "making deployment simple."
+
+**Recommendation:** Invest 2-3 weeks in deployment automation (CI/CD, pre-built images, deploy.sh) rather than continuing to document complex manual processes.