From 18e581b13a8c2c244050df23f6297e3ee59978a4 Mon Sep 17 00:00:00 2001 From: Proudlock <270015376@geaerospace.com> Date: Sun, 6 Sep 2026 23:59:27 -0400 Subject: [PATCH] Add comprehensive local development setup documentation - Complete Windows dev environment setup guide - Database configuration (port 3307 explained) - Plugin installation and enabling process - Admin password reset procedures - Troubleshooting common issues - Zscaler proxy certificate extraction - run_dev.py usage and why it's needed - Common development tasks - Architecture notes and file structure - Git workflow documentation --- LOCAL-DEV-SETUP.md | 362 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 362 insertions(+) create mode 100644 LOCAL-DEV-SETUP.md diff --git a/LOCAL-DEV-SETUP.md b/LOCAL-DEV-SETUP.md new file mode 100644 index 00000000..3b515c09 --- /dev/null +++ b/LOCAL-DEV-SETUP.md @@ -0,0 +1,362 @@ +# ShopDB Local Development Setup (Windows) + +Complete guide for setting up ShopDB Flask development environment on Windows. + +## Prerequisites + +- Python 3.13+ +- Docker Desktop (for MySQL and testing) +- Git +- Node.js 20+ (for frontend development) + +## Quick Start + +```bash +# 1. Clone and enter repo +cd C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb + +# 2. Create Python virtual environment +python -m venv venv +./venv/Scripts/activate +pip install -r requirements.txt + +# 3. Start MySQL in Docker +docker compose up -d db + +# 4. Configure environment +cp .env.docker .env +# Edit .env and set DATABASE_URL (see below) + +# 5. Install and enable plugins +export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev" +./venv/Scripts/flask.exe plugin install backups computers employees endpoints geenforce guides knowledgebase machines measuringtools network notifications printedparts printers slides tools usb warranty +./venv/Scripts/flask.exe plugin enable + +# 6. Start Flask +./venv/Scripts/python.exe run_dev.py +``` + +Visit http://localhost:5001 and login with `admin` / `admin`. + +## Detailed Setup + +### 1. Database Setup + +MySQL runs in Docker on **port 3307** (not 3306) to avoid conflicts with any local MySQL instance. + +```bash +# Start MySQL container +docker compose up -d db + +# Verify it's running +docker ps --filter name=shopdb-dev-db +# Should show: 127.0.0.1:3307->3306/tcp + +# Test connection +docker exec shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' -e "SELECT 1;" +``` + +### 2. Environment Configuration + +Create `.env` file with: + +```bash +# Database (port 3307 because Docker maps it there) +DATABASE_URL=mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev + +# Flask +FLASK_ENV=development +FLASK_DEBUG=1 + +# Secrets (dev only) +SECRET_KEY=docker-dev-secret-key-change-for-production +JWT_SECRET_KEY=docker-dev-jwt-secret-change-for-production + +# CORS +CORS_ORIGINS=http://localhost:5173,http://localhost:5001 + +# API Port +API_PORT=5001 + +# ShopDB Label +SHOPDB_ENV_LABEL=LOCAL-DEV + +# Log Level +LOG_LEVEL=DEBUG + +# Backup Storage (for local dev) +SHOPDB_BACKUPS_ROOT=C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb/backups +``` + +**Important:** Change `YOUR_USERNAME` to your actual Windows username. + +### 3. Import Database (Optional) + +If you have a production database dump: + +```bash +# Import latest.sql +docker exec -i shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' shopdb_flask_dev < latest.sql + +# Run migrations +export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev" +./venv/Scripts/flask.exe db upgrade +``` + +### 4. Plugin Installation + +All 17 plugins must be installed before they can be enabled: + +```bash +# Set DATABASE_URL for CLI commands +export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev" + +# Install all plugins +for plugin in backups computers employees endpoints geenforce guides knowledgebase machines measuringtools network notifications printedparts printers slides tools usb warranty; do + ./venv/Scripts/flask.exe plugin install $plugin +done + +# Enable all plugins +for plugin in backups computers employees endpoints geenforce guides knowledgebase machines measuringtools network notifications printedparts printers slides tools usb warranty; do + ./venv/Scripts/flask.exe plugin enable $plugin +done +``` + +**Why this is necessary:** +- Plugin state is tracked in `instance/plugins.json`, not the database +- When you import a prod database, it has plugin tables but no plugin registry +- Installing registers plugins in the JSON file +- Enabling makes them active (but requires app restart) + +### 5. Reset Admin Password + +If you imported a prod database and don't know the password: + +```bash +# Generate new password hash +./venv/Scripts/python.exe -c "from werkzeug.security import generate_password_hash; print(generate_password_hash('admin'))" + +# Reset via MySQL +docker exec shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' shopdb_flask_dev -e " +UPDATE users +SET passwordhash = 'PASTE_HASH_HERE', + failedlogins = 0, + lockeduntil = NULL +WHERE username = 'admin'; +" +``` + +Or use the simpler method: +```bash +docker exec shopdb-dev-db mysql -u root -p'root-dev-password-not-for-production' shopdb_flask_dev -e " +UPDATE users +SET passwordhash = 'scrypt:32768:8:1\$B4LhpPrM6qxijInK\$b99b47a0bb5fd45761a1823f401f70a3ceccf71a5d38dc433378aa07426ef8eaf7e679ff3dd7b0ce9772600f83368ac642e9419b9beaa49f87483318bae34547', + failedlogins = 0, + lockeduntil = NULL +WHERE username = 'admin'; +" +``` + +Password is now: `admin` / `admin` + +### 6. Starting Flask + +**Always use `run_dev.py`** (not `flask run` directly): + +```bash +./venv/Scripts/python.exe run_dev.py +``` + +This script: +- Loads `.env` file with DATABASE_URL +- Sets all environment variables +- Starts Flask without auto-reloader (prevents env var loss) +- Binds to 0.0.0.0:5001 + +**Do NOT use:** +- `flask run` - Won't load `.env` properly +- `flask run --reload` - Loses environment variables + +### 7. Frontend Development (Optional) + +To develop the Vue frontend: + +```bash +cd frontend +npm ci +npm run dev +``` + +Frontend dev server runs on http://localhost:5173 and proxies API requests to Flask on port 5001. + +For production build (served by Flask): +```bash +cd frontend +npm run build +# Outputs to frontend/dist/ +``` + +## Troubleshooting + +### "Can't connect to MySQL on 127.0.0.1:3306" + +**Cause:** Flask is trying port 3306 instead of 3307. + +**Fix:** +1. Make sure `.env` has `DATABASE_URL=...@localhost:3307/...` +2. Use `run_dev.py` to start Flask (NOT `flask run`) +3. If problem persists, restart Flask completely + +### "Cannot import name '_backups_configured'" + +**Cause:** Merge conflict between wj and wilm backup implementations. + +**Fix:** This was fixed in commit `8a1fe0ada8` - pull latest wj branch. + +### "Plugin X is not installed" + +**Cause:** Plugin registry (`instance/plugins.json`) doesn't exist or is incomplete. + +**Fix:** Run plugin installation commands (section 4 above). + +### "Access is denied: '/backups'" + +**Cause:** Backup system trying to use Linux path on Windows. + +**Fix:** +1. Add to `.env`: `SHOPDB_BACKUPS_ROOT=C:/Users/YOUR_USERNAME/Documents/github/flask-shopdb/backups` +2. Create directory: `mkdir backups` +3. Restart Flask with `run_dev.py` + +### Docker npm install fails (Zscaler proxy) + +**Cause:** Corporate proxy intercepts HTTPS, npm doesn't trust certificate. + +**Fix:** Extract Zscaler root CA certificate: + +```powershell +# Run this PowerShell script +$request = [System.Net.HttpWebRequest]::Create('https://registry.npmjs.org/') +$request.Method = 'HEAD' +$request.Timeout = 10000 +$response = $request.GetResponse() +$root = [System.Security.Cryptography.X509Certificates.X509Certificate2]::new($request.ServicePoint.Certificate) +$builder = [System.Text.StringBuilder]::new() +[void]$builder.AppendLine('-----BEGIN CERTIFICATE-----') +[void]$builder.AppendLine([System.Convert]::ToBase64String($root.RawData, 'InsertLineBreaks')) +[void]$builder.AppendLine('-----END CERTIFICATE-----') +Set-Content -Path 'C:\Users\270015376\Documents\github\flask-shopdb\corp-root-ca.pem' -Value $builder.ToString() -Encoding ASCII +``` + +Then rebuild Docker: `docker compose build --no-cache` + +## Common Tasks + +### Run Tests +```bash +pytest +pytest tests/test_specific.py +pytest -v -s # Verbose with print output +``` + +### Run Migrations +```bash +export DATABASE_URL="mysql+pymysql://root:root-dev-password-not-for-production@localhost:3307/shopdb_flask_dev" +./venv/Scripts/flask.exe db upgrade +``` + +### Create Migration +```bash +./venv/Scripts/flask.exe db migrate -m "Description of changes" +``` + +### Check Plugin Status +```bash +./venv/Scripts/flask.exe plugin list +``` + +### Enable/Disable Plugin +```bash +./venv/Scripts/flask.exe plugin enable slides +./venv/Scripts/flask.exe plugin disable slides +``` + +### Trigger Backup +```bash +# Via API (requires admin JWT token) +curl -X POST http://localhost:5001/api/shopdbbackups \ + -H "Authorization: Bearer YOUR_TOKEN" + +# Or via Docker +docker compose -f docker-compose.backup.yml up backup +``` + +## Architecture Notes + +### Why Port 3307? +Docker MySQL maps internal port 3306 to host port 3307 to avoid conflicts with any local MySQL installation. This is configured in `docker-compose.yml`: +```yaml +ports: + - "127.0.0.1:3307:3306" +``` + +### Why run_dev.py? +Flask's `flask run` command uses an auto-reloader that spawns a subprocess. This subprocess loses environment variables loaded from `.env`. The `run_dev.py` wrapper: +1. Loads `.env` via python-dotenv BEFORE importing Flask +2. Uses `--no-reload` flag to prevent subprocess spawning +3. Ensures DATABASE_URL and other vars persist + +### Plugin Architecture +- Plugins are Python packages in `plugins/` directory +- Plugin state tracked in `instance/plugins.json` (not database) +- Plugin tables tracked via Alembic migration chains +- Each plugin has own migrations in `plugins//migrations/` + +### Backup System +- Designed for Docker compose deployment +- API queues backups via sentinel file (`.backup-requested`) +- Actual backup execution requires Docker sidecar container +- On Windows dev, API can queue but won't execute without Docker + +## File Structure + +``` +flask-shopdb/ +├── .env # Local environment config (gitignored) +├── run_dev.py # Local development launcher +├── shopdb/ # Flask application +│ ├── __init__.py # App factory +│ ├── config.py # Configuration classes +│ ├── core/ # Core models and APIs +│ └── plugins/ # Plugin manager +├── plugins/ # Plugin packages +│ ├── backups/ +│ ├── computers/ +│ └── ... +├── frontend/ # Vue 3 SPA +│ ├── src/ +│ └── dist/ # Production build (gitignored) +├── instance/ # Runtime state (gitignored) +│ ├── plugins.json # Plugin registry +│ └── ... +├── migrations/ # Core Alembic migrations +├── scripts/ # Deployment and backup scripts +├── backups/ # Local backup storage (gitignored) +└── tests/ # Test suite +``` + +## Git Workflow + +- `main` branch - Published/public version (GitHub) +- `wj` branch - Production deployment (West Jefferson site) +- `wilm` branch - Development branch (upstream from benjamin.baker) +- `dev` branch - Ops deployment branch + +**Never commit to `main` directly** - it's published to public wiki. + +## Additional Resources + +- [Backup Setup](scripts/backup/README.md) +- [Plugin Development](docs/PLUGIN-HOOKS.md) +- [ADRs](wiki/ADRs/) +- [API Docs](http://localhost:5001/api/docs) (when Flask is running)