Reorganize repository structure and clean up documentation
Clean up and organize the flask-shopdb repository for better maintainability and discoverability: ## Changes - Update .gitignore for build artifacts (publish/, data/, 122M total) - Reorganize .claude/ into 6 subdirectories with README navigation - Consolidate overlapping MCP setup guides into MCP-QUICK-START.md - Move deployment docs from root to docs/deployment/ - Archive dated status reports to .claude/archived/ - Delete temporary files (logs, backups, empty reports, ~50K) ## New Structure .claude/ organized into: - setup/ - MCP, VM, Hyper-V, GitHub setup guides + scripts (17 files) - plans/ - Feature plans and protocols (5 files) - maintenance/ - Deployment checklists and reports (4 files) - archived/ - Historical documentation (9 files) - skills/ - Claude Code automation (5 files) - workflows/ - Security audit workflow (1 file) docs/deployment/ created with 6 deployment guides: - LOCAL-DEV-SETUP.md, DOCKER-SETUP.md, DOCKER-PREBUILT-APPROACH.md - PRODUCTION-READINESS.md, DEPLOY-TO-VM.md, README.md Root directory cleaned to essential docs only: - CLAUDE.md, README.md, CONTRIBUTING.md, CHANGELOG.md Benefits: - Clear navigation via README.md in each subdirectory - No duplicate/overlapping documentation - Large artifacts properly gitignored (won't bloat repository) - Historical context preserved in archived/ See .claude/CLEANUP-SUMMARY.md for complete details. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Proudlock
177
docs/deployment/DEPLOY-TO-VM.md
Normal file
177
docs/deployment/DEPLOY-TO-VM.md
Normal file
@@ -0,0 +1,177 @@
|
||||
# Deploy ShopDB to Hyper-V VM
|
||||
|
||||
Quick guide to deploy the Docker stack to a Windows 11 Hyper-V VM.
|
||||
|
||||
## Prerequisites on VM
|
||||
|
||||
1. **Docker Desktop** installed and running
|
||||
2. **Git Bash** or **WSL** installed (for bash commands)
|
||||
3. **PowerShell** (already on Windows 11)
|
||||
|
||||
## Step 1: Copy Files to VM
|
||||
|
||||
**Method A: Shared Folder**
|
||||
```powershell
|
||||
# On host PC, share the repo
|
||||
# Then on VM, copy from network share
|
||||
Copy-Item -Recurse \\host-pc\shopdb-flask C:\shopdb
|
||||
```
|
||||
|
||||
**Method B: Git Clone**
|
||||
```bash
|
||||
# On VM
|
||||
cd C:\
|
||||
git clone git@github.com:ge-aero/shopdb-flask.git
|
||||
cd shopdb-flask
|
||||
git checkout wj
|
||||
```
|
||||
|
||||
**Method C: Export/Import**
|
||||
```powershell
|
||||
# On host PC
|
||||
Compress-Archive -Path C:\Users\270015376\Documents\github\flask-shopdb -DestinationPath C:\shopdb-deploy.zip
|
||||
|
||||
# Copy shopdb-deploy.zip to VM, then on VM:
|
||||
Expand-Archive -Path C:\shopdb-deploy.zip -DestinationPath C:\shopdb
|
||||
```
|
||||
|
||||
## Step 2: Run Deployment Script
|
||||
|
||||
```powershell
|
||||
# On VM, open PowerShell as Administrator
|
||||
cd C:\shopdb
|
||||
|
||||
# Option 1: Quick deploy with defaults
|
||||
.\deploy-to-vm.ps1
|
||||
|
||||
# Option 2: Specify site name and password
|
||||
.\deploy-to-vm.ps1 -SiteName "west-jefferson" -AdminPassword "YourSecurePassword123!"
|
||||
|
||||
# Option 3: See all options
|
||||
.\deploy-to-vm.ps1 -Help
|
||||
```
|
||||
|
||||
## What the Script Does
|
||||
|
||||
1. [OK] Checks Docker Desktop is running
|
||||
2. [OK] Creates .env from template
|
||||
3. [OK] Builds frontend (if needed)
|
||||
4. [OK] Builds Docker images (2-3 minutes, offline packages)
|
||||
5. [OK] Starts all services (db + api + backup)
|
||||
6. [OK] Runs database migrations
|
||||
7. [OK] Sets admin password
|
||||
|
||||
## Manual Deployment (Alternative)
|
||||
|
||||
If you prefer to run commands manually:
|
||||
|
||||
```bash
|
||||
# 1. Create environment
|
||||
cp .env.docker-dev .env
|
||||
|
||||
# 2. Build frontend (if not already built)
|
||||
cd frontend
|
||||
npm ci
|
||||
npm run build
|
||||
cd ..
|
||||
|
||||
# 3. Build and start Docker stack
|
||||
docker compose -f docker-compose.prebuilt.yml -f docker-compose.backup.yml --profile backup build
|
||||
docker compose -f docker-compose.prebuilt.yml -f docker-compose.backup.yml --profile backup up -d
|
||||
|
||||
# 4. Initialize database
|
||||
docker exec flask-shopdb-api-1 flask db upgrade
|
||||
|
||||
# 5. Reset admin password (optional)
|
||||
docker exec flask-shopdb-db-1 mysql -uroot -proot-dev-password-not-for-production shopdb_flask -e \
|
||||
"UPDATE users SET passwordhash='scrypt:32768:8:1\$B4LhpPrM6qxijInK\$b99b47a0bb5fd45761a1823f401f70a3ceccf71a5d38dc433378aa07426ef8eaf7e679ff3dd7b0ce9772600f83368ac642e9419b9beaa49f87483318bae34547', failedlogins=0, lockeduntil=NULL WHERE username='admin';"
|
||||
# (Password is: admin)
|
||||
```
|
||||
|
||||
## Access
|
||||
|
||||
**URL:** https://localhost:5001 (or https://vm-ip-address:5001 from host)
|
||||
|
||||
**Login:**
|
||||
- Username: `admin`
|
||||
- Password: Whatever you set in deploy script (default: `ChangeMe123!`)
|
||||
|
||||
## Verify Deployment
|
||||
|
||||
```powershell
|
||||
# Check all services are running
|
||||
docker ps
|
||||
|
||||
# Should see:
|
||||
# - flask-shopdb-db-1
|
||||
# - flask-shopdb-api-1
|
||||
# - flask-shopdb-backup-store-1
|
||||
# - flask-shopdb-backup-watcher-1
|
||||
|
||||
# View logs
|
||||
docker logs -f flask-shopdb-api-1
|
||||
|
||||
# Test backup
|
||||
# Go to https://localhost:5001/settings/shopdbbackups
|
||||
# Click "Back Up Now"
|
||||
# Wait 30 seconds, should show as "Checked"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Docker not running:**
|
||||
```powershell
|
||||
# Start Docker Desktop manually
|
||||
& "C:\Program Files\Docker\Docker\Docker Desktop.exe"
|
||||
Start-Sleep -Seconds 30
|
||||
```
|
||||
|
||||
**Port 5001 already in use:**
|
||||
```powershell
|
||||
# Edit .env and change API_PORT
|
||||
# API_PORT=5002
|
||||
```
|
||||
|
||||
**Can't access from host PC:**
|
||||
```powershell
|
||||
# On VM, check Windows Firewall
|
||||
New-NetFirewallRule -DisplayName "ShopDB" -Direction Inbound -LocalPort 5001 -Protocol TCP -Action Allow
|
||||
```
|
||||
|
||||
**SSL certificate warnings:**
|
||||
- Normal for self-signed certs
|
||||
- Click "Advanced" -> "Proceed to localhost (unsafe)"
|
||||
|
||||
## Stop/Start
|
||||
|
||||
```bash
|
||||
# Stop
|
||||
docker compose -f docker-compose.prebuilt.yml -f docker-compose.backup.yml down
|
||||
|
||||
# Start (after first deployment)
|
||||
docker compose -f docker-compose.prebuilt.yml -f docker-compose.backup.yml --profile backup up -d
|
||||
```
|
||||
|
||||
## Update After Code Changes
|
||||
|
||||
```bash
|
||||
# 1. Pull latest code
|
||||
git pull origin wj
|
||||
|
||||
# 2. Rebuild frontend (if changed)
|
||||
cd frontend && npm run build && cd ..
|
||||
|
||||
# 3. Rebuild and restart
|
||||
docker compose -f docker-compose.prebuilt.yml -f docker-compose.backup.yml build
|
||||
docker compose -f docker-compose.prebuilt.yml -f docker-compose.backup.yml up -d
|
||||
```
|
||||
|
||||
## Offline Deployment Notes
|
||||
|
||||
**This deployment works completely offline** because:
|
||||
- [OK] Frontend pre-built and checked into repo
|
||||
- [OK] Restic binary pre-downloaded (8.6MB in repo)
|
||||
- [OK] No npm/pip downloads during Docker build
|
||||
- [OK] All dependencies baked into images
|
||||
|
||||
Perfect for air-gapped or restricted network environments!
|
||||
96
docs/deployment/DOCKER-PREBUILT-APPROACH.md
Normal file
96
docs/deployment/DOCKER-PREBUILT-APPROACH.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Docker Pre-Built Frontend Approach
|
||||
|
||||
**Problem:** Zscaler corporate proxy causes npm to fail inside Docker builds with "Exit handler never called" error.
|
||||
|
||||
**Solution:** Build frontend on host (where npm works), copy pre-built dist/ into Docker image.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Build frontend on host
|
||||
cd frontend
|
||||
npm ci
|
||||
npm run build
|
||||
|
||||
# 2. Build Docker image (uses pre-built dist/)
|
||||
cd ..
|
||||
docker compose -f docker-compose.prebuilt.yml build
|
||||
|
||||
# 3. Start services
|
||||
docker compose -f docker-compose.prebuilt.yml up -d
|
||||
|
||||
# 4. Initialize (first time only)
|
||||
docker compose -f docker-compose.prebuilt.yml exec api flask db upgrade
|
||||
docker compose -f docker-compose.prebuilt.yml exec api bash /app/scripts/bootstrap-plugins.sh
|
||||
|
||||
# 5. Access
|
||||
https://localhost:5001
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
- `Dockerfile.prebuilt` - Simplified Dockerfile without npm build
|
||||
- `docker-compose.prebuilt.yml` - Uses Dockerfile.prebuilt
|
||||
- `.dockerignore.prebuilt` - Allows frontend/dist to be copied
|
||||
|
||||
## Why This Works
|
||||
|
||||
**Normal Dockerfile:**
|
||||
- Multi-stage build
|
||||
- Stage 1: Node 20 runs `npm ci` and `npm run build`
|
||||
- Stage 2: Copies built frontend from stage 1
|
||||
- **PROBLEM:** npm fails inside Docker with Zscaler proxy
|
||||
|
||||
**Pre-Built Approach:**
|
||||
- Build frontend on Windows host (npm works here)
|
||||
- Single-stage Dockerfile copies pre-built `frontend/dist/`
|
||||
- Only installs Python dependencies
|
||||
- **FASTER:** 2-3 minutes vs 15+ minutes
|
||||
|
||||
## Trade-offs
|
||||
|
||||
**Pros:**
|
||||
- Avoids corporate proxy issues
|
||||
- Much faster builds (no npm in container)
|
||||
- Frontend build errors visible immediately
|
||||
- Can iterate on frontend without rebuilding container
|
||||
|
||||
**Cons:**
|
||||
- Requires npm on host machine
|
||||
- Two-step build process
|
||||
- frontend/dist must be in version control OR built before deployment
|
||||
|
||||
## For Production (CI/CD)
|
||||
|
||||
```yaml
|
||||
# GitHub Actions / Jenkins example
|
||||
- name: Build frontend
|
||||
run: |
|
||||
cd frontend
|
||||
npm ci
|
||||
npm run build
|
||||
|
||||
- name: Build Docker image
|
||||
run: |
|
||||
docker build -f Dockerfile.prebuilt -t shopdb:${{ github.sha }} .
|
||||
|
||||
- name: Push to registry
|
||||
run: |
|
||||
docker push shopdb:${{ github.sha }}
|
||||
```
|
||||
|
||||
## Comparison
|
||||
|
||||
| Approach | Build Time | Corporate Proxy | Complexity | Production Ready |
|
||||
|----------|-----------|----------------|------------|------------------|
|
||||
| Multi-stage Dockerfile | 15+ min | [FAIL] Fails | High | [FAIL] No |
|
||||
| Pre-built frontend | 2-3 min | [OK] Works | Medium | [WARN] Yes (with CI/CD) |
|
||||
| Pre-built images from registry | 30 sec | [OK] Works | Low | [OK] Yes |
|
||||
|
||||
## Next Steps
|
||||
|
||||
For true production deployment:
|
||||
1. Set up CI/CD to build frontend + Docker image
|
||||
2. Push to container registry
|
||||
3. Sites pull pre-built images
|
||||
4. No build step at deployment site
|
||||
398
docs/deployment/DOCKER-SETUP.md
Normal file
398
docs/deployment/DOCKER-SETUP.md
Normal file
@@ -0,0 +1,398 @@
|
||||
# ShopDB Docker Setup
|
||||
|
||||
Complete Docker deployment for development and testing.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker Desktop installed and running
|
||||
- At least 8GB RAM allocated to Docker
|
||||
- Zscaler corporate certificate extracted (if behind corporate proxy)
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Clone repository
|
||||
cd C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb
|
||||
|
||||
# 2. Extract Zscaler certificate (if needed)
|
||||
powershell.exe -File C:/temp/extract-ca.ps1
|
||||
# This creates corp-root-ca.pem in repo root
|
||||
|
||||
# 3. Build images
|
||||
docker compose build
|
||||
|
||||
# 4. Start stack
|
||||
docker compose up -d
|
||||
|
||||
# 5. Initialize database (first time only)
|
||||
docker compose exec api flask db upgrade
|
||||
|
||||
# 6. Install plugins (first time only)
|
||||
docker compose exec api flask plugin apply-profile deploy/site-profile-universal.json
|
||||
|
||||
# 7. Access ShopDB
|
||||
https://localhost:5001
|
||||
```
|
||||
|
||||
Login: `admin` / (reset password - see below)
|
||||
|
||||
## Architecture
|
||||
|
||||
The Docker stack includes:
|
||||
|
||||
1. **MySQL Database** (`db` service)
|
||||
- Port: 3306 (internal), mapped to host based on `.env`
|
||||
- Volume: `db_data` for persistence
|
||||
- Health checks enabled
|
||||
|
||||
2. **Flask API** (`api` service)
|
||||
- Port: 5001 (HTTPS with self-signed cert)
|
||||
- Built frontend served from `/app/frontend/dist`
|
||||
- Gunicorn with gevent workers (4 workers, 2000 connections each)
|
||||
- Volumes:
|
||||
- `instance_data` - Plugin state, uploads, branding
|
||||
- `./plugins` - Plugin code (read-only)
|
||||
- `./scripts` - Deployment scripts (read-only)
|
||||
- `./certs` - TLS certificates (read-only)
|
||||
|
||||
3. **Backup Service** (optional, via `docker-compose.backup.yml`)
|
||||
- One-shot container for backups
|
||||
- Triggered manually or via scheduler
|
||||
|
||||
## First-Time Setup
|
||||
|
||||
### 1. Extract Zscaler Certificate (Corporate Proxy)
|
||||
|
||||
Save this as `C:/temp/extract-ca.ps1`:
|
||||
|
||||
```powershell
|
||||
$request = [System.Net.HttpWebRequest]::Create('https://registry.npmjs.org/')
|
||||
$request.Method = 'HEAD'
|
||||
$request.Timeout = 10000
|
||||
$response = $request.GetResponse()
|
||||
$root = [System.Security.Cryptography.X509Certificates.X509Certificate2]::new($request.ServicePoint.Certificate)
|
||||
$builder = [System.Text.StringBuilder]::new()
|
||||
[void]$builder.AppendLine('-----BEGIN CERTIFICATE-----')
|
||||
[void]$builder.AppendLine([System.Convert]::ToBase64String($root.RawData, 'InsertLineBreaks'))
|
||||
[void]$builder.AppendLine('-----END CERTIFICATE-----')
|
||||
$certPath = 'C:\Users\270015376\Documents\github\flask-shopdb\corp-root-ca.pem'
|
||||
Set-Content -Path $certPath -Value $builder.ToString() -Encoding ASCII
|
||||
Write-Host "Staged corp-root-ca.pem"
|
||||
```
|
||||
|
||||
Run: `powershell.exe -ExecutionPolicy Bypass -File C:/temp/extract-ca.ps1`
|
||||
|
||||
### 2. Generate TLS Certificates
|
||||
|
||||
```bash
|
||||
cd certs
|
||||
openssl req -x509 -newkey rsa:2048 -nodes \
|
||||
-keyout server.key -out server.crt -days 365 \
|
||||
-subj "//CN=localhost"
|
||||
```
|
||||
|
||||
Or use the provided script:
|
||||
```bash
|
||||
./scripts/generate-dev-certs.sh
|
||||
```
|
||||
|
||||
### 3. Configure Environment
|
||||
|
||||
Copy and edit `.env`:
|
||||
```bash
|
||||
cp .env.docker-dev .env
|
||||
# Edit if needed - defaults work for local dev
|
||||
```
|
||||
|
||||
### 4. Build Images
|
||||
|
||||
```bash
|
||||
docker compose build
|
||||
```
|
||||
|
||||
**Note:** First build takes 10-15 minutes due to:
|
||||
- npm install (~5 min with Zscaler proxy)
|
||||
- Frontend Vite build (~2 min)
|
||||
- Python dependencies (~3 min)
|
||||
|
||||
Subsequent builds use Docker layer cache and are much faster.
|
||||
|
||||
### 5. Start Services
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Check status:
|
||||
```bash
|
||||
docker compose ps
|
||||
docker compose logs -f api
|
||||
```
|
||||
|
||||
### 6. Initialize Database
|
||||
|
||||
```bash
|
||||
# Run migrations
|
||||
docker compose exec api flask db upgrade
|
||||
|
||||
# Install and enable all plugins
|
||||
docker compose exec api flask plugin apply-profile deploy/site-profile-universal.json
|
||||
```
|
||||
|
||||
The `apply-profile` command:
|
||||
- Installs all 17 plugins
|
||||
- Registers them in `instance/plugins.json`
|
||||
- Leaves them disabled by default
|
||||
- Enables them via Settings UI as needed
|
||||
|
||||
### 7. Reset Admin Password
|
||||
|
||||
```bash
|
||||
# Generate hash for "admin" password
|
||||
docker compose exec api python -c "from werkzeug.security import generate_password_hash; print(generate_password_hash('admin'))"
|
||||
|
||||
# Update database
|
||||
docker compose exec db mysql -u root -p shopdb_flask -e "
|
||||
UPDATE users
|
||||
SET passwordhash = 'PASTE_HASH_HERE',
|
||||
failedlogins = 0,
|
||||
lockeduntil = NULL
|
||||
WHERE username = 'admin';"
|
||||
```
|
||||
|
||||
Or use quick method (sets password to `admin`):
|
||||
```bash
|
||||
docker compose exec db mysql -u root -p shopdb_flask -e "
|
||||
UPDATE users SET
|
||||
passwordhash = 'scrypt:32768:8:1\$B4LhpPrM6qxijInK\$b99b47a0bb5fd45761a1823f401f70a3ceccf71a5d38dc433378aa07426ef8eaf7e679ff3dd7b0ce9772600f83368ac642e9419b9beaa49f87483318bae34547',
|
||||
failedlogins = 0,
|
||||
lockeduntil = NULL
|
||||
WHERE username = 'admin';"
|
||||
```
|
||||
|
||||
## Import Production Database
|
||||
|
||||
```bash
|
||||
# Copy SQL dump to container
|
||||
docker cp latest.sql shopdb-api-1:/tmp/
|
||||
|
||||
# Import
|
||||
docker compose exec db mysql -u root -p shopdb_flask < /tmp/latest.sql
|
||||
|
||||
# Run migrations to bring schema up to date
|
||||
docker compose exec api flask db upgrade
|
||||
docker compose exec api flask plugin upgrade-all
|
||||
```
|
||||
|
||||
## Common Commands
|
||||
|
||||
### View Logs
|
||||
```bash
|
||||
docker compose logs -f api
|
||||
docker compose logs -f db
|
||||
```
|
||||
|
||||
### Restart Services
|
||||
```bash
|
||||
docker compose restart api
|
||||
docker compose down && docker compose up -d
|
||||
```
|
||||
|
||||
### Rebuild After Code Changes
|
||||
```bash
|
||||
docker compose build api
|
||||
docker compose up -d api
|
||||
```
|
||||
|
||||
### Access Container Shell
|
||||
```bash
|
||||
docker compose exec api bash
|
||||
docker compose exec db bash
|
||||
```
|
||||
|
||||
### Run Flask Commands
|
||||
```bash
|
||||
docker compose exec api flask plugin list
|
||||
docker compose exec api flask db current
|
||||
docker compose exec api flask routes
|
||||
```
|
||||
|
||||
### Backup & Restore
|
||||
```bash
|
||||
# Trigger backup
|
||||
docker compose -f docker-compose.backup.yml up backup
|
||||
|
||||
# View backups
|
||||
docker compose exec api flask plugin list
|
||||
# (Accessible via Settings > Backups in UI)
|
||||
|
||||
# Restore (via UI at Settings > Backups > Select backup > Restore)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "npm ci failed" during build
|
||||
|
||||
**Cause:** Zscaler proxy blocking npm registry.
|
||||
|
||||
**Fix:**
|
||||
1. Extract Zscaler certificate (see above)
|
||||
2. Ensure `corp-root-ca.pem` exists in repo root
|
||||
3. Rebuild: `docker compose build --no-cache`
|
||||
|
||||
### "database is unhealthy"
|
||||
|
||||
**Cause:** MySQL container not ready or port conflict.
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
# Check logs
|
||||
docker compose logs db
|
||||
|
||||
# Check if port 3306 is already in use
|
||||
netstat -an | grep 3306
|
||||
|
||||
# If conflict, edit .env and set MYSQL_PORT=3307
|
||||
```
|
||||
|
||||
### "Access denied for user 'shopdb'"
|
||||
|
||||
**Cause:** Database credentials mismatch.
|
||||
|
||||
**Fix:** Check `.env` file - `MYSQL_PASSWORD` must match `DATABASE_URL`.
|
||||
|
||||
### "Could not connect to database"
|
||||
|
||||
**Cause:** Database service not started or wrong hostname.
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
# Check db is running
|
||||
docker compose ps
|
||||
|
||||
# API must use hostname 'db' not 'localhost'
|
||||
# In docker-compose.yml: DATABASE_URL=...@db:3306/...
|
||||
```
|
||||
|
||||
### "ERR_CERT_AUTHORITY_INVALID" in browser
|
||||
|
||||
**Expected:** Self-signed certificate warning.
|
||||
|
||||
**Fix:** Click "Advanced" > "Proceed to localhost (unsafe)" - this is normal for dev.
|
||||
|
||||
Or generate proper cert:
|
||||
```bash
|
||||
# Add your machine's hostname to cert
|
||||
openssl req -x509 -newkey rsa:2048 -nodes \
|
||||
-keyout certs/server.key -out certs/server.crt -days 365 \
|
||||
-subj "//CN=$(hostname)" \
|
||||
-addext "subjectAltName=DNS:localhost,DNS:$(hostname),IP:127.0.0.1"
|
||||
```
|
||||
|
||||
### "Plugin X is already installed" but not visible
|
||||
|
||||
**Cause:** Plugin installed but not enabled.
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
docker compose exec api flask plugin enable X
|
||||
docker compose restart api
|
||||
```
|
||||
|
||||
### Build is very slow
|
||||
|
||||
**Expected:** First build with Zscaler proxy takes 10-15 minutes.
|
||||
|
||||
**Speed up:**
|
||||
- Subsequent builds use cache (much faster)
|
||||
- If rebuilding often, comment out `--no-cache` flag
|
||||
- Build frontend separately on host, copy dist/ into image
|
||||
|
||||
## Production Deployment Notes
|
||||
|
||||
For production:
|
||||
|
||||
1. **Change all secrets** in `.env`:
|
||||
- `SECRET_KEY`
|
||||
- `JWT_SECRET_KEY`
|
||||
- `MYSQL_ROOT_PASSWORD`
|
||||
- `MYSQL_PASSWORD`
|
||||
|
||||
2. **Use proper TLS certificates** (not self-signed):
|
||||
- Place in `./certs/server.crt` and `./certs/server.key`
|
||||
- Or use reverse proxy (nginx/IIS) for TLS termination
|
||||
|
||||
3. **Set `FLASK_DEBUG=0`** in production
|
||||
|
||||
4. **Configure backups**:
|
||||
- Set up cron job: `0 2 * * * cd /path/to/shopdb && docker compose -f docker-compose.backup.yml up backup`
|
||||
- Configure SMB or S3 destination in Settings > Backups
|
||||
|
||||
5. **Set `SHOPDB_ENV_LABEL`** to site name (e.g., "WEST-JEFFERSON")
|
||||
|
||||
6. **Review `docker-compose.yml`**:
|
||||
- Adjust worker count based on load
|
||||
- Configure proper `restart` policy
|
||||
- Set resource limits if needed
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
### Why HTTPS in Docker?
|
||||
|
||||
Secure context APIs (clipboard, geolocation) require HTTPS. Running over HTTP disables these browser features.
|
||||
|
||||
### Why Gevent Workers?
|
||||
|
||||
Endpoint agents long-poll `/api/endpoints/commands/next`. With threading, 200 agents = 200 threads = resource exhaustion. Gevent uses greenlets (cheap coroutines) so 2000+ connections per worker is feasible.
|
||||
|
||||
### Plugin Bootstrap
|
||||
|
||||
The entrypoint (`scripts/docker-entrypoint.sh`) automatically:
|
||||
1. Runs database migrations
|
||||
2. Checks if `instance/plugins.json` exists
|
||||
3. If not, installs all available plugins (disabled)
|
||||
4. Starts gunicorn
|
||||
|
||||
This ensures imported databases with plugin tables get matching plugin registry.
|
||||
|
||||
### Instance Data Persistence
|
||||
|
||||
The `instance_data` volume persists:
|
||||
- `plugins.json` - Which plugins are enabled
|
||||
- Uploaded files (floor plans, branding, photos, warranties, slides)
|
||||
- Dell OAuth tokens
|
||||
- Any plugin-specific state
|
||||
|
||||
**Important:** `docker compose down -v` deletes volumes - use plain `docker compose down` to preserve data.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
flask-shopdb/
|
||||
|-- docker-compose.yml # Main stack definition
|
||||
|-- docker-compose.backup.yml # Backup service
|
||||
|-- Dockerfile # Multi-stage build
|
||||
|-- .env # Environment config (gitignored)
|
||||
|-- corp-root-ca.pem # Zscaler cert (gitignored)
|
||||
|-- certs/ # TLS certificates (gitignored)
|
||||
| |-- server.crt
|
||||
| +-- server.key
|
||||
|-- scripts/
|
||||
| |-- docker-entrypoint.sh # Container startup
|
||||
| +-- bootstrap-plugins.sh # Plugin auto-install
|
||||
+-- deploy/
|
||||
+-- site-profile-universal.json # Full plugin list
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
After setup:
|
||||
|
||||
1. Browse to https://localhost:5001
|
||||
2. Login as `admin`
|
||||
3. Go to Settings > Plugins
|
||||
4. Enable plugins your site needs
|
||||
5. Configure settings (CORS, backup destination, etc.)
|
||||
6. Import data or start fresh
|
||||
|
||||
For local development with hot-reload, see `LOCAL-DEV-SETUP.md`.
|
||||
362
docs/deployment/LOCAL-DEV-SETUP.md
Normal file
362
docs/deployment/LOCAL-DEV-SETUP.md
Normal file
@@ -0,0 +1,362 @@
|
||||
# ShopDB Local Development Setup (Windows)
|
||||
|
||||
Complete guide for setting up ShopDB Flask development environment on Windows.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.13+
|
||||
- Docker Desktop (for MySQL and testing)
|
||||
- Git
|
||||
- Node.js 20+ (for frontend development)
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Clone and enter repo
|
||||
cd C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb
|
||||
|
||||
# 2. Create Python virtual environment
|
||||
python -m venv venv
|
||||
./venv/Scripts/activate
|
||||
pip install -r requirements.txt
|
||||
|
||||
# 3. Start MySQL in Docker
|
||||
docker compose up -d db
|
||||
|
||||
# 4. Configure environment
|
||||
cp .env.docker .env
|
||||
# Edit .env and set DATABASE_URL (see below)
|
||||
|
||||
# 5. Install and enable plugins
|
||||
export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev"
|
||||
./venv/Scripts/flask.exe plugin install backups computers employees endpoints geenforce guides knowledgebase machines measuringtools network notifications printedparts printers slides tools usb warranty
|
||||
./venv/Scripts/flask.exe plugin enable <each-plugin-name>
|
||||
|
||||
# 6. Start Flask
|
||||
./venv/Scripts/python.exe run_dev.py
|
||||
```
|
||||
|
||||
Visit http://localhost:5001 and login with `admin` / `admin`.
|
||||
|
||||
## Detailed Setup
|
||||
|
||||
### 1. Database Setup
|
||||
|
||||
MySQL runs in Docker on **port 3307** (not 3306) to avoid conflicts with any local MySQL instance.
|
||||
|
||||
```bash
|
||||
# Start MySQL container
|
||||
docker compose up -d db
|
||||
|
||||
# Verify it's running
|
||||
docker ps --filter name=shopdb-dev-db
|
||||
# Should show: 127.0.0.1:3307->3306/tcp
|
||||
|
||||
# Test connection
|
||||
docker exec shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' -e "SELECT 1;"
|
||||
```
|
||||
|
||||
### 2. Environment Configuration
|
||||
|
||||
Create `.env` file with:
|
||||
|
||||
```bash
|
||||
# Database (port 3307 because Docker maps it there)
|
||||
DATABASE_URL=mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev
|
||||
|
||||
# Flask
|
||||
FLASK_ENV=development
|
||||
FLASK_DEBUG=1
|
||||
|
||||
# Secrets (dev only)
|
||||
SECRET_KEY=docker-dev-secret-key-change-for-production
|
||||
JWT_SECRET_KEY=docker-dev-jwt-secret-change-for-production
|
||||
|
||||
# CORS
|
||||
CORS_ORIGINS=http://localhost:5173,http://localhost:5001
|
||||
|
||||
# API Port
|
||||
API_PORT=5001
|
||||
|
||||
# ShopDB Label
|
||||
SHOPDB_ENV_LABEL=LOCAL-DEV
|
||||
|
||||
# Log Level
|
||||
LOG_LEVEL=DEBUG
|
||||
|
||||
# Backup Storage (for local dev)
|
||||
SHOPDB_BACKUPS_ROOT=C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb/backups
|
||||
```
|
||||
|
||||
**Important:** Change `YOUR_USERNAME` to your actual Windows username.
|
||||
|
||||
### 3. Import Database (Optional)
|
||||
|
||||
If you have a production database dump:
|
||||
|
||||
```bash
|
||||
# Import latest.sql
|
||||
docker exec -i shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' shopdb_flask_dev < latest.sql
|
||||
|
||||
# Run migrations
|
||||
export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev"
|
||||
./venv/Scripts/flask.exe db upgrade
|
||||
```
|
||||
|
||||
### 4. Plugin Installation
|
||||
|
||||
All 17 plugins must be installed before they can be enabled:
|
||||
|
||||
```bash
|
||||
# Set DATABASE_URL for CLI commands
|
||||
export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev"
|
||||
|
||||
# Install all plugins
|
||||
for plugin in backups computers employees endpoints geenforce guides knowledgebase machines measuringtools network notifications printedparts printers slides tools usb warranty; do
|
||||
./venv/Scripts/flask.exe plugin install $plugin
|
||||
done
|
||||
|
||||
# Enable all plugins
|
||||
for plugin in backups computers employees endpoints geenforce guides knowledgebase machines measuringtools network notifications printedparts printers slides tools usb warranty; do
|
||||
./venv/Scripts/flask.exe plugin enable $plugin
|
||||
done
|
||||
```
|
||||
|
||||
**Why this is necessary:**
|
||||
- Plugin state is tracked in `instance/plugins.json`, not the database
|
||||
- When you import a prod database, it has plugin tables but no plugin registry
|
||||
- Installing registers plugins in the JSON file
|
||||
- Enabling makes them active (but requires app restart)
|
||||
|
||||
### 5. Reset Admin Password
|
||||
|
||||
If you imported a prod database and don't know the password:
|
||||
|
||||
```bash
|
||||
# Generate new password hash
|
||||
./venv/Scripts/python.exe -c "from werkzeug.security import generate_password_hash; print(generate_password_hash('admin'))"
|
||||
|
||||
# Reset via MySQL
|
||||
docker exec shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' shopdb_flask_dev -e "
|
||||
UPDATE users
|
||||
SET passwordhash = 'PASTE_HASH_HERE',
|
||||
failedlogins = 0,
|
||||
lockeduntil = NULL
|
||||
WHERE username = 'admin';
|
||||
"
|
||||
```
|
||||
|
||||
Or use the simpler method:
|
||||
```bash
|
||||
docker exec shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' shopdb_flask_dev -e "
|
||||
UPDATE users
|
||||
SET passwordhash = 'scrypt:32768:8:1\$B4LhpPrM6qxijInK\$b99b47a0bb5fd45761a1823f401f70a3ceccf71a5d38dc433378aa07426ef8eaf7e679ff3dd7b0ce9772600f83368ac642e9419b9beaa49f87483318bae34547',
|
||||
failedlogins = 0,
|
||||
lockeduntil = NULL
|
||||
WHERE username = 'admin';
|
||||
"
|
||||
```
|
||||
|
||||
Password is now: `admin` / `admin`
|
||||
|
||||
### 6. Starting Flask
|
||||
|
||||
**Always use `run_dev.py`** (not `flask run` directly):
|
||||
|
||||
```bash
|
||||
./venv/Scripts/python.exe run_dev.py
|
||||
```
|
||||
|
||||
This script:
|
||||
- Loads `.env` file with DATABASE_URL
|
||||
- Sets all environment variables
|
||||
- Starts Flask without auto-reloader (prevents env var loss)
|
||||
- Binds to 0.0.0.0:5001
|
||||
|
||||
**Do NOT use:**
|
||||
- `flask run` - Won't load `.env` properly
|
||||
- `flask run --reload` - Loses environment variables
|
||||
|
||||
### 7. Frontend Development (Optional)
|
||||
|
||||
To develop the Vue frontend:
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm ci
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Frontend dev server runs on http://localhost:5173 and proxies API requests to Flask on port 5001.
|
||||
|
||||
For production build (served by Flask):
|
||||
```bash
|
||||
cd frontend
|
||||
npm run build
|
||||
# Outputs to frontend/dist/
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Can't connect to MySQL on 127.0.0.1:3306"
|
||||
|
||||
**Cause:** Flask is trying port 3306 instead of 3307.
|
||||
|
||||
**Fix:**
|
||||
1. Make sure `.env` has `DATABASE_URL=...@localhost:3307/...`
|
||||
2. Use `run_dev.py` to start Flask (NOT `flask run`)
|
||||
3. If problem persists, restart Flask completely
|
||||
|
||||
### "Cannot import name '_backups_configured'"
|
||||
|
||||
**Cause:** Merge conflict between wj and wilm backup implementations.
|
||||
|
||||
**Fix:** This was fixed in commit `8a1fe0ada8` - pull latest wj branch.
|
||||
|
||||
### "Plugin X is not installed"
|
||||
|
||||
**Cause:** Plugin registry (`instance/plugins.json`) doesn't exist or is incomplete.
|
||||
|
||||
**Fix:** Run plugin installation commands (section 4 above).
|
||||
|
||||
### "Access is denied: '/backups'"
|
||||
|
||||
**Cause:** Backup system trying to use Linux path on Windows.
|
||||
|
||||
**Fix:**
|
||||
1. Add to `.env`: `SHOPDB_BACKUPS_ROOT=C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb/backups`
|
||||
2. Create directory: `mkdir backups`
|
||||
3. Restart Flask with `run_dev.py`
|
||||
|
||||
### Docker npm install fails (Zscaler proxy)
|
||||
|
||||
**Cause:** Corporate proxy intercepts HTTPS, npm doesn't trust certificate.
|
||||
|
||||
**Fix:** Extract Zscaler root CA certificate:
|
||||
|
||||
```powershell
|
||||
# Run this PowerShell script
|
||||
$request = [System.Net.HttpWebRequest]::Create('https://registry.npmjs.org/')
|
||||
$request.Method = 'HEAD'
|
||||
$request.Timeout = 10000
|
||||
$response = $request.GetResponse()
|
||||
$root = [System.Security.Cryptography.X509Certificates.X509Certificate2]::new($request.ServicePoint.Certificate)
|
||||
$builder = [System.Text.StringBuilder]::new()
|
||||
[void]$builder.AppendLine('-----BEGIN CERTIFICATE-----')
|
||||
[void]$builder.AppendLine([System.Convert]::ToBase64String($root.RawData, 'InsertLineBreaks'))
|
||||
[void]$builder.AppendLine('-----END CERTIFICATE-----')
|
||||
Set-Content -Path 'C:\Users\270015376\Documents\github\flask-shopdb\corp-root-ca.pem' -Value $builder.ToString() -Encoding ASCII
|
||||
```
|
||||
|
||||
Then rebuild Docker: `docker compose build --no-cache`
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Run Tests
|
||||
```bash
|
||||
pytest
|
||||
pytest tests/test_specific.py
|
||||
pytest -v -s # Verbose with print output
|
||||
```
|
||||
|
||||
### Run Migrations
|
||||
```bash
|
||||
export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev"
|
||||
./venv/Scripts/flask.exe db upgrade
|
||||
```
|
||||
|
||||
### Create Migration
|
||||
```bash
|
||||
./venv/Scripts/flask.exe db migrate -m "Description of changes"
|
||||
```
|
||||
|
||||
### Check Plugin Status
|
||||
```bash
|
||||
./venv/Scripts/flask.exe plugin list
|
||||
```
|
||||
|
||||
### Enable/Disable Plugin
|
||||
```bash
|
||||
./venv/Scripts/flask.exe plugin enable slides
|
||||
./venv/Scripts/flask.exe plugin disable slides
|
||||
```
|
||||
|
||||
### Trigger Backup
|
||||
```bash
|
||||
# Via API (requires admin JWT token)
|
||||
curl -X POST http://localhost:5001/api/shopdbbackups \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
|
||||
# Or via Docker
|
||||
docker compose -f docker-compose.backup.yml up backup
|
||||
```
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
### Why Port 3307?
|
||||
Docker MySQL maps internal port 3306 to host port 3307 to avoid conflicts with any local MySQL installation. This is configured in `docker-compose.yml`:
|
||||
```yaml
|
||||
ports:
|
||||
- "127.0.0.1:3307:3306"
|
||||
```
|
||||
|
||||
### Why run_dev.py?
|
||||
Flask's `flask run` command uses an auto-reloader that spawns a subprocess. This subprocess loses environment variables loaded from `.env`. The `run_dev.py` wrapper:
|
||||
1. Loads `.env` via python-dotenv BEFORE importing Flask
|
||||
2. Uses `--no-reload` flag to prevent subprocess spawning
|
||||
3. Ensures DATABASE_URL and other vars persist
|
||||
|
||||
### Plugin Architecture
|
||||
- Plugins are Python packages in `plugins/` directory
|
||||
- Plugin state tracked in `instance/plugins.json` (not database)
|
||||
- Plugin tables tracked via Alembic migration chains
|
||||
- Each plugin has own migrations in `plugins/<name>/migrations/`
|
||||
|
||||
### Backup System
|
||||
- Designed for Docker compose deployment
|
||||
- API queues backups via sentinel file (`.backup-requested`)
|
||||
- Actual backup execution requires Docker sidecar container
|
||||
- On Windows dev, API can queue but won't execute without Docker
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
flask-shopdb/
|
||||
|-- .env # Local environment config (gitignored)
|
||||
|-- run_dev.py # Local development launcher
|
||||
|-- shopdb/ # Flask application
|
||||
| |-- __init__.py # App factory
|
||||
| |-- config.py # Configuration classes
|
||||
| |-- core/ # Core models and APIs
|
||||
| +-- plugins/ # Plugin manager
|
||||
|-- plugins/ # Plugin packages
|
||||
| |-- backups/
|
||||
| |-- computers/
|
||||
| +-- ...
|
||||
|-- frontend/ # Vue 3 SPA
|
||||
| |-- src/
|
||||
| +-- dist/ # Production build (gitignored)
|
||||
|-- instance/ # Runtime state (gitignored)
|
||||
| |-- plugins.json # Plugin registry
|
||||
| +-- ...
|
||||
|-- migrations/ # Core Alembic migrations
|
||||
|-- scripts/ # Deployment and backup scripts
|
||||
|-- backups/ # Local backup storage (gitignored)
|
||||
+-- tests/ # Test suite
|
||||
```
|
||||
|
||||
## Git Workflow
|
||||
|
||||
- `main` branch - Published/public version (GitHub)
|
||||
- `wj` branch - Production deployment (West Jefferson site)
|
||||
- `wilm` branch - Development branch (upstream from benjamin.baker)
|
||||
- `dev` branch - Ops deployment branch
|
||||
|
||||
**Never commit to `main` directly** - it's published to public wiki.
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Backup Setup](scripts/backup/README.md)
|
||||
- [Plugin Development](docs/PLUGIN-HOOKS.md)
|
||||
- [ADRs](wiki/ADRs/)
|
||||
- [API Docs](http://localhost:5001/api/docs) (when Flask is running)
|
||||
345
docs/deployment/PRODUCTION-READINESS.md
Normal file
345
docs/deployment/PRODUCTION-READINESS.md
Normal file
@@ -0,0 +1,345 @@
|
||||
# ShopDB Production Readiness Assessment
|
||||
|
||||
**Last Updated:** 2026-09-07
|
||||
**Branch:** wj (post-wilm merge)
|
||||
|
||||
## Executive Summary
|
||||
|
||||
[CRITICAL] **NOT PRODUCTION READY** in current state without significant operational investment.
|
||||
|
||||
The Docker deployment requires extensive manual setup, multiple services, and assumes expertise with Docker Compose, restic backup systems, and Flask plugin architecture. This assessment documents current gaps and provides recommendations.
|
||||
|
||||
## Current State: Deployment Complexity
|
||||
|
||||
### What Works
|
||||
|
||||
[OK] **Flask application code** - Stable, tested, 17 plugins functional
|
||||
[OK] **Database schema** - Migrations work, data model is sound
|
||||
[OK] **Frontend build** - Vue 3 SPA compiles and serves correctly
|
||||
[OK] **Docker images** - Build successfully (with caveats below)
|
||||
[OK] **Plugin architecture** - Clean, documented, extensible
|
||||
|
||||
### Critical Gaps
|
||||
|
||||
[FAIL] **One-command deployment** - Requires 7-10 manual steps
|
||||
[FAIL] **Corporate proxy handling** - Zscaler cert extraction needed for npm
|
||||
[FAIL] **Backup system complexity** - 3 containers + restic + scheduler
|
||||
[FAIL] **TLS certificate generation** - Manual OpenSSL commands required
|
||||
[FAIL] **Plugin initialization** - Must run `apply-profile` after first start
|
||||
[FAIL] **Password reset** - No default admin password, requires manual SQL
|
||||
[FAIL] **Build time** - 15+ minutes with corporate proxy
|
||||
[FAIL] **Documentation scattered** - 3 different setup guides (Docker, Local, Backup)
|
||||
|
||||
## Why Is This So Complex?
|
||||
|
||||
### 1. Corporate Network Reality
|
||||
|
||||
**Problem:** Behind Zscaler proxy, npm can't trust npm registry
|
||||
**Impact:** Docker build fails without extracted corporate root CA
|
||||
**Workaround:** PowerShell script to extract cert before building
|
||||
**Production Fix:** Pre-build images in CI/CD with cert, publish to registry
|
||||
|
||||
### 2. Plugin Architecture Design
|
||||
|
||||
**Problem:** Plugin state in `instance/plugins.json`, not database
|
||||
**Impact:** Importing prod database doesn't register plugins
|
||||
**Workaround:** Manual `flask plugin apply-profile` after first start
|
||||
**Production Fix:** Auto-detect installed plugins from schema on boot
|
||||
|
||||
### 3. Backup System Architecture
|
||||
|
||||
**Problem:** Designed for restic with 3-container orchestration
|
||||
**Impact:** Requires backup-store + backup-watcher + backup containers
|
||||
**Workaround:** All documented, but complex to deploy
|
||||
**Production Fix:** Simpler mysqldump-only mode with direct file storage
|
||||
|
||||
### 4. Multi-Stage Docker Build
|
||||
|
||||
**Problem:** Frontend build needs Node 20, backend needs Python 3.14
|
||||
**Impact:** Multi-stage build is slow, complex, hard to debug
|
||||
**Workaround:** Build works, but takes 15 minutes
|
||||
**Production Fix:** Pre-build frontend on CI, copy dist/ into Python-only image
|
||||
|
||||
## Deployment Models (Ordered by Simplicity)
|
||||
|
||||
### Option 1: IIS + Windows Services (SIMPLEST)
|
||||
|
||||
**Best for:** Single Windows server deployment
|
||||
**Complexity:** Low
|
||||
**Maintenance:** Medium
|
||||
|
||||
**Setup:**
|
||||
1. Install Python 3.13+
|
||||
2. Install MySQL (or use existing instance)
|
||||
3. Run installer (creates Windows service for Flask)
|
||||
4. Configure IIS reverse proxy
|
||||
5. Install plugins via UI
|
||||
|
||||
**Pros:**
|
||||
- Native Windows deployment
|
||||
- Familiar IIS management
|
||||
- No Docker complexity
|
||||
- Fast startup
|
||||
|
||||
**Cons:**
|
||||
- Windows-only
|
||||
- Manual updates
|
||||
- No containerization benefits
|
||||
|
||||
**Production Ready:** [WARN] **YES** (with installer improvements)
|
||||
|
||||
### Option 2: Docker Pre-Built Images (RECOMMENDED)
|
||||
|
||||
**Best for:** Sites with Docker but no build capability
|
||||
**Complexity:** Low-Medium
|
||||
**Maintenance:** Low
|
||||
|
||||
**Setup:**
|
||||
1. Pull pre-built image from registry
|
||||
2. Run docker-compose with .env file
|
||||
3. Access UI, enable plugins
|
||||
|
||||
**Pros:**
|
||||
- No build step
|
||||
- Fast deployment (2 minutes)
|
||||
- Consistent environment
|
||||
- Easy updates (pull new image)
|
||||
|
||||
**Cons:**
|
||||
- Requires Docker
|
||||
- Need image registry
|
||||
- Initial registry setup
|
||||
|
||||
**Production Ready:** [OK] **YES** (requires CI/CD setup)
|
||||
|
||||
### Option 3: Docker Source Build (CURRENT)
|
||||
|
||||
**Best for:** Development, customization
|
||||
**Complexity:** High
|
||||
**Maintenance:** High
|
||||
|
||||
**Setup:**
|
||||
1. Extract Zscaler cert
|
||||
2. Generate TLS certs
|
||||
3. Build Docker images (15+ min)
|
||||
4. Configure .env with 15+ variables
|
||||
5. Start docker-compose
|
||||
6. Run migrations
|
||||
7. Install plugins
|
||||
8. Reset admin password
|
||||
9. Configure backup system
|
||||
10. Test everything
|
||||
|
||||
**Pros:**
|
||||
- Full source control
|
||||
- Customizable
|
||||
- No external dependencies
|
||||
|
||||
**Cons:**
|
||||
- Extremely complex
|
||||
- Long build times
|
||||
- Many failure points
|
||||
- Requires Docker expertise
|
||||
|
||||
**Production Ready:** [CRITICAL] **NO** (too complex for operations)
|
||||
|
||||
### Option 4: Kubernetes (FUTURE)
|
||||
|
||||
**Best for:** Multi-site deployment, high availability
|
||||
**Complexity:** Very High
|
||||
**Maintenance:** Medium (with proper GitOps)
|
||||
|
||||
**Status:** Not implemented
|
||||
**Effort:** 2-3 weeks of work
|
||||
|
||||
## What Would Make This Production Ready?
|
||||
|
||||
### Critical Path (Must Have)
|
||||
|
||||
1. **Pre-Built Images in Registry**
|
||||
- Build images in CI/CD
|
||||
- Push to container registry
|
||||
- Sites just `docker pull`
|
||||
- **Effort:** 1 week
|
||||
|
||||
2. **One-Command Setup**
|
||||
- `./deploy.sh` does everything
|
||||
- No manual cert extraction
|
||||
- No manual plugin installation
|
||||
- **Effort:** 3 days
|
||||
|
||||
3. **Automated Plugin Detection**
|
||||
- Scan database schema on boot
|
||||
- Auto-register plugins in JSON
|
||||
- No manual `apply-profile`
|
||||
- **Effort:** 2 days
|
||||
|
||||
4. **Simplified Backup (Option)**
|
||||
- Simple mysqldump + tar mode
|
||||
- Single container, no restic
|
||||
- Good enough for 80% of sites
|
||||
- **Effort:** 1 week
|
||||
|
||||
5. **Windows Installer Improvements**
|
||||
- One-click install
|
||||
- Includes Python runtime
|
||||
- Creates service automatically
|
||||
- **Effort:** 1 week
|
||||
|
||||
### Nice to Have (Future)
|
||||
|
||||
- Helm chart for Kubernetes
|
||||
- Monitoring/metrics built-in
|
||||
- Health check endpoints
|
||||
- Automated testing in CI
|
||||
- Rolling update strategy
|
||||
|
||||
## Honest Recommendations
|
||||
|
||||
### For West Jefferson (Your Site)
|
||||
|
||||
**Recommended:** Option 1 (IIS + Windows Services)
|
||||
|
||||
**Why:**
|
||||
- You already have Windows infrastructure
|
||||
- Simpler to operate
|
||||
- Faster to deploy
|
||||
- Your team knows IIS
|
||||
|
||||
**Steps:**
|
||||
1. Build Windows installer
|
||||
2. Deploy to Windows Server
|
||||
3. Configure IIS reverse proxy
|
||||
4. Import prod database
|
||||
5. Done
|
||||
|
||||
**Timeline:** 2-3 days including testing
|
||||
|
||||
### For New Sites
|
||||
|
||||
**Recommended:** Option 2 (Docker Pre-Built Images)
|
||||
|
||||
**Prerequisites:**
|
||||
- Set up CI/CD to build images
|
||||
- Set up container registry (Docker Hub, Harbor, or AWS ECR)
|
||||
- Create `deploy.sh` script
|
||||
|
||||
**Then sites can:**
|
||||
```bash
|
||||
# 1. Copy .env.example to .env and edit
|
||||
cp .env.example .env
|
||||
|
||||
# 2. Run deployment
|
||||
./deploy.sh
|
||||
|
||||
# 3. Access UI
|
||||
https://your-server:5001
|
||||
```
|
||||
|
||||
**Timeline for CI/CD setup:** 1-2 weeks
|
||||
|
||||
### For Current Docker Source Build
|
||||
|
||||
**Status:** Development/customization only, not for production
|
||||
|
||||
**Use when:**
|
||||
- Testing new features
|
||||
- Developing plugins
|
||||
- Debugging issues
|
||||
- Learning the system
|
||||
|
||||
**Don't use when:**
|
||||
- Deploying to operations
|
||||
- Training new ops staff
|
||||
- Time-sensitive deployments
|
||||
|
||||
## Current Documentation Issues
|
||||
|
||||
### Too Many Guides
|
||||
|
||||
- `LOCAL-DEV-SETUP.md` - Windows development (port 3307, run_dev.py)
|
||||
- `DOCKER-SETUP.md` - Docker source build (complex, 15+ steps)
|
||||
- `PRODUCTION-READINESS.md` - This file (meta/assessment)
|
||||
- `scripts/backup/README.md` - Backup system (3 containers, restic)
|
||||
|
||||
**Problem:** Which one do I follow?
|
||||
|
||||
**Solution:** Create decision tree:
|
||||
- Developer on Windows -> LOCAL-DEV-SETUP.md
|
||||
- Ops deploying to prod -> (DOESN'T EXIST YET - need simplified guide)
|
||||
- Custom development -> DOCKER-SETUP.md
|
||||
- Backup questions -> scripts/backup/README.md
|
||||
|
||||
### Missing Guides
|
||||
|
||||
- **PRODUCTION-DEPLOY.md** - Step-by-step for operations (IIS or Docker pull)
|
||||
- **QUICK-START.md** - 5-minute version for evaluation
|
||||
- **TROUBLESHOOTING.md** - Common issues, not scattered across files
|
||||
- **UPGRADE-GUIDE.md** - How to update between versions
|
||||
|
||||
## What We Did Today (Honest Assessment)
|
||||
|
||||
[OK] **Merged wilm->wj** - 98 commits, wilm features now in wj
|
||||
[OK] **Fixed local dev** - `run_dev.py` works for Windows development
|
||||
[OK] **Documented Docker** - Comprehensive but too complex
|
||||
[OK] **Added backup docs** - System works but needs expertise
|
||||
[WARN] **Docker "production ready"** - Technically works, operationally complex
|
||||
[FAIL] **Simple deployment** - Still requires too many manual steps
|
||||
|
||||
## Next Steps (Priority Order)
|
||||
|
||||
### Immediate (This Week)
|
||||
|
||||
1. **Create QUICK-START.md**
|
||||
- 5-minute eval version
|
||||
- Uses pre-built image (if available) OR local dev
|
||||
- Gets people to working system fast
|
||||
|
||||
2. **Test Windows Installer**
|
||||
- Build installer for wj branch
|
||||
- Deploy to test Windows Server
|
||||
- Document any issues
|
||||
|
||||
3. **Create deployment decision tree**
|
||||
- Flowchart: what's my situation -> which method to use
|
||||
- Don't make users read 4 docs to figure out where to start
|
||||
|
||||
### Short Term (Next Sprint)
|
||||
|
||||
4. **Set up CI/CD for image builds**
|
||||
- GitHub Actions or Jenkins
|
||||
- Build on every push to wj
|
||||
- Push to internal registry
|
||||
|
||||
5. **Create `deploy.sh` script**
|
||||
- One command: `./deploy.sh --env prod`
|
||||
- Handles certs, plugins, migrations automatically
|
||||
|
||||
6. **Simplify backup system**
|
||||
- Add "simple mode" - just mysqldump + tar
|
||||
- Keep restic for advanced sites
|
||||
- Document tradeoffs
|
||||
|
||||
### Long Term (Next Month)
|
||||
|
||||
7. **Auto-plugin detection**
|
||||
- Boot-time schema scan
|
||||
- No manual `apply-profile`
|
||||
|
||||
8. **Consolidate documentation**
|
||||
- One entry point
|
||||
- Clear paths for each audience
|
||||
- Tested by someone who hasn't seen the system
|
||||
|
||||
## Conclusion
|
||||
|
||||
**Is ShopDB production-ready?**
|
||||
|
||||
- **The code:** Yes, it's solid
|
||||
- **Docker source build deployment:** No, too complex
|
||||
- **Windows installer deployment:** Yes, with improvements
|
||||
- **Docker pre-built images deployment:** Yes, but needs CI/CD setup
|
||||
|
||||
**Bottom line:** The application is mature. The deployment story needs work. Focus should shift from "making Docker work" to "making deployment simple."
|
||||
|
||||
**Recommendation:** Invest 2-3 weeks in deployment automation (CI/CD, pre-built images, deploy.sh) rather than continuing to document complex manual processes.
|
||||
29
docs/deployment/README.md
Normal file
29
docs/deployment/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# ShopDB Deployment Documentation
|
||||
|
||||
Complete deployment guides for development, Docker, and production environments.
|
||||
|
||||
## Quick Start
|
||||
|
||||
**Local development**: `LOCAL-DEV-SETUP.md`
|
||||
**Docker deployment**: `DOCKER-SETUP.md`
|
||||
**VM deployment**: `DEPLOY-TO-VM.md`
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
- `DOCKER-SETUP.md` - Complete Docker setup and configuration
|
||||
- `DOCKER-PREBUILT-APPROACH.md` - Workaround for corporate proxy blocking npm in Docker
|
||||
- `PRODUCTION-READINESS.md` - Production deployment checklist
|
||||
|
||||
## Development
|
||||
|
||||
- `LOCAL-DEV-SETUP.md` - Setting up local development environment (Flask + Vite)
|
||||
|
||||
## VM Deployment
|
||||
|
||||
- `DEPLOY-TO-VM.md` - Deploying the C# endpoint agent to test VMs
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- `.claude/setup/` - MCP servers, VM configuration, and development tools
|
||||
- `.claude/maintenance/` - Operational checklists and maintenance procedures
|
||||
- `../shopdb-flask.wiki/ADR-004-deployment-topology.md` - Architecture decision record for deployment topology
|
||||
Reference in New Issue
Block a user