shopdb-flask/docs/DEVELOPMENT-SETUP.md

<!-- docmeta
purpose: howto
audience: developer
status: current
summary: Clone to a running dev site, the three gates you run before pushing, and how to contribute a change back.
loadwith: none
-->

# Development setup: from clone to first change

The goal of this page: a new developer clones the repo and has a working dev
site plus a change they can see in the browser, in one sitting. Reference
material lives elsewhere - naming rules in `CONTRIBUTING.md`, every config
variable in the CONFIG guide, plugin authoring in the PLUGIN docs - this is
just the on-ramp.

**Most developers here are on Windows in VS Code** - commands below are
PowerShell first, with the bash equivalent in a comment where they differ.
Install **Git for Windows** (it ships Git Bash, which VS Code and the git
hooks use to run the shell-based naming check) and **VS Code** with the
extensions this repo recommends (you'll be prompted - section 2c).

Two ways to run it. **Docker** (Docker Desktop on Windows) is the fastest to a
working site. **Manual (venv + Node)** is the daily driver - frontend
hot-reloads, backend restarts on save. Do Docker once to confirm the box is
sane, then use manual for day-to-day work.

If you only want to LOOK at a running site and are not going to change the
code, do not do any of this: [DEV-BUILD-TEST](DEV-BUILD-TEST.md) pulls a
prebuilt image and needs nothing but Docker.

---

## 0. Prerequisites

| Need | Version | Check |
| --- | --- | --- |
| Python | 3.14 (64-bit) - matches CI, the container image and the Windows installer wheelhouse | `python --version` |
| Node.js | 18+ | `node --version` |
| MySQL | 8.4 LTS (or Docker, below) | `mysql --version` |
| Git | any recent | `git --version` |

On Windows, install all of them with winget (accept each license, then
reopen the terminal so PATH updates):

```powershell
winget install Git.Git                    # includes Git Bash (naming hook needs it)
winget install Python.Python.3.14
winget install OpenJS.NodeJS.LTS          # LTS; may install v24, fine for this SPA
winget install Microsoft.VisualStudioCode
winget install Oracle.MySQL               # or Docker.DockerDesktop for the DB
```

CI runs Node 20; the LTS package may be newer. This Vite/Vue frontend builds
identically across 20-24, so it does not matter. To pin exactly:
`winget install CoreyButler.NVMforWindows` then `nvm install 20; nvm use 20`.

---

## 0b. Corp network (SSL cert) - if you are behind a GE/Zscaler proxy

A proxy that inspects HTTPS (Zscaler on GE PCs) re-signs every connection
with a corporate root CA. `git`, `npm`, `pip`, and Node each keep their own
trust store and do not trust that CA by default, so downloads fail:

| Tool | Symptom |
| --- | --- |
| npm | `UNABLE_TO_GET_ISSUER_CERT_LOCALLY` |
| git | `SSL certificate problem: unable to get local issuer certificate` |
| pip | `SSLError` / `CERTIFICATE_VERIFY_FAILED` |

Fix once - export the corp root CA, point every tool at it. PowerShell
mangles multi-line pastes, so each step below is **one physical line**: paste
it, press Enter, then the next. Do not paste both at once.

```powershell
# 1) Bundle EVERY trusted root into one PEM (one line). Guessing which single cert is the proxy's is fragile; bundling all always includes it.
$sb = New-Object System.Text.StringBuilder; Get-ChildItem Cert:\LocalMachine\Root | ForEach-Object { [void]$sb.AppendLine("-----BEGIN CERTIFICATE-----"); [void]$sb.AppendLine([Convert]::ToBase64String($_.RawData,'InsertLineBreaks')); [void]$sb.AppendLine("-----END CERTIFICATE-----") }; [IO.File]::WriteAllText("$HOME\corp-root-ca.pem", $sb.ToString())
```

Confirm it has many certs (dozens, not 1):
`(Select-String "BEGIN CERTIFICATE" $HOME\corp-root-ca.pem).Count`

```powershell
# 2) Point every tool at it (one line, persistent). NODE_EXTRA_CA_CERTS also fixes Vite / npm run dev.
git config --global http.sslCAInfo "$HOME\corp-root-ca.pem"; npm config set cafile "$HOME\corp-root-ca.pem"; setx NODE_EXTRA_CA_CERTS "$HOME\corp-root-ca.pem"; setx PIP_CERT "$HOME\corp-root-ca.pem"
```

Reopen the terminal so `setx` takes effect. Quick unblock if you cannot
export right now (skips verification - use briefly, then set back):
`npm config set strict-ssl false`, `git config --global http.sslVerify false`.

---

## 1. Get the code

```powershell
git clone https://github.com/ge-aero/shopdb-flask.git
cd shopdb-flask
```

Never work on `main`. Branch for your change:

```powershell
git checkout -b feat/<short-description>
```

---

## 2a. Fast path - Docker (a working site in one command)

```powershell
copy .env.example .env
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, and the MYSQL_* passwords.
# Generate a secret: python -c "import secrets;print(secrets.token_urlsafe(64))"

docker compose up -d --build          # MySQL + the app (frontend built in-image)

# Schema + platform data (idempotent, safe to re-run):
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask seed permissions
docker compose exec api flask seed settings
docker compose exec api flask seed reference-data
docker compose exec api flask seed admin --username admin --email you@example.com
docker compose exec api flask seed demo    # OPTIONAL: sample data (undo: flask seed demo-clear)
```

The app is on the port the compose file maps (see `docker-compose.yml`). Good
for a smoke test; for active development use the manual path so the frontend
hot-reloads.

---

## 2b. Manual path - venv + Node (the daily driver)

### Database

Either point at an existing MySQL 8, or bring one up with just the db service
from compose:

```powershell
docker compose up -d db                # MySQL on 127.0.0.1:3306
```

The mapping is loopback-only and `MYSQL_PORT` moves it. If the box already runs
a local MySQL on 3306, set `MYSQL_PORT=3307` in `.env` and put the same 3307 in
`DATABASE_URL` - a mismatch here is the single most common "cannot connect"
in development, because the error names 3306 and looks like the container is
down when it is simply somewhere else.

Create the database + app user (skip if compose already did via env):

```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY 'devpassword';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```

### Backend

```powershell
python -m venv venv
venv\Scripts\Activate.ps1             # bash/mac: source venv/bin/activate
# If PowerShell blocks the activate script (execution policy), run once:
#   Set-ExecutionPolicy -Scope CurrentUser RemoteSigned
pip install -r requirements-dev.txt

copy .env.example .env                 # bash/mac: cp .env.example .env
# Edit .env - set SECRET_KEY, JWT_SECRET_KEY, and
#   DATABASE_URL=mysql+pymysql://shopdb:devpassword@127.0.0.1:3306/shopdb_flask?charset=utf8mb4
#   CORS_ORIGINS=http://localhost:5173

$env:FLASK_APP = "shopdb"              # bash/mac: export FLASK_APP=shopdb
flask db upgrade                       # core schema
flask plugin upgrade-all               # per-plugin schema (ADR-008)
flask seed permissions
flask seed settings
flask seed reference-data
flask seed admin --username admin --email you@example.com   # password printed once
flask seed demo            # OPTIONAL: sample assets across the plugins (undo: flask seed demo-clear)
```

Enable the plugins you want visible. To turn on everything for development:

PowerShell:

```powershell
foreach ($p in "backups","computers","endpoints","guides","knowledgebase",
               "machines","network","notifications","printers","slides",
               "tools","warranty") {
  flask plugin install $p; flask plugin enable $p
}
```

(bash/mac: a `for p in ...; do flask plugin install "$p"; ...; done` loop.)

Run the backend ON PORT 5001 - the frontend dev server proxies `/api` and
`/static` there (a bare `flask run` uses 5000 and nothing will load). Use the
launcher in the repository root rather than `flask run`:

```powershell
python run_dev.py
```

**Use `run_dev.py`, not `flask run`.** `flask run` reloads by spawning a
subprocess, and the subprocess does not inherit what `python-dotenv` loaded from
`.env` - so the reloaded backend comes up with no `DATABASE_URL` and tries
127.0.0.1:3306 with default credentials. `run_dev.py` loads `.env` BEFORE
importing Flask, then runs `flask run --host=0.0.0.0 --port=5001 --no-reload`.
It prints the `DATABASE_URL` and `SHOPDB_BACKUPS_ROOT` it resolved, which is the
fastest check that `.env` was actually read. The cost is that backend edits need
a manual restart.

**Leave this running.** It does not return to a prompt - that is correct, not a
hang. The server holds this terminal until you stop it. Do NOT press Ctrl+C to
move on; that kills the backend. Open the frontend in a separate terminal (next
section) and leave this one alone. Ctrl+C only when you are done for the day.

### Frontend (a second terminal - leave the backend running)

```powershell
cd frontend
npm install
npm run dev                            # http://localhost:5173
```

Open http://localhost:5173, log in as `admin` with the printed password.

To point the dev proxy at a backend on another machine, set the environment
variable rather than editing `vite.config.js`:

```bash
VITE_DEV_API_TARGET=https://10.0.0.5:5001 npm run dev
```

Certificate verification is off for an `https` target, because those boxes
normally carry a self-signed certificate; `VITE_DEV_SECURE=true` turns it back
on.

> Convenience: instead of two terminals you can run both under a process
> manager (pm2, honcho, foreman). Keep the backend on 5001.

### Working from a copy of a site's database

Restoring a dump instead of seeding is the fastest way to reproduce a real bug,
and it has two traps:

```powershell
docker exec -i <db-container> mysql -u root -p"<MYSQL_ROOT_PASSWORD>" shopdb_flask < dump.sql
flask db upgrade
flask plugin upgrade-all
```

1. **The dump has plugin tables but no plugin registry.** The enabled-plugin
   list lives in `instance/plugins.json`, not in MySQL, so a restored database
   comes up with every table and an empty nav. Install and enable the plugins
   (above), or `flask plugin apply-profile deploy/site-profile-universal.json`.
2. **You will not know the admin password.** Mint a fresh hash and write it
   over the row, clearing the lockout counters at the same time:

   ```powershell
   python -c "from werkzeug.security import generate_password_hash; print(generate_password_hash('admin'))"
   ```

   ```sql
   UPDATE users SET passwordhash = '<paste>', failedlogins = 0, lockeduntil = NULL
    WHERE username = 'admin';
   ```

   Do this only against a development copy. Never paste a hash from a document
   into a real site - one that has been published is a password everybody has.

If the backups feature is in play, `SHOPDB_BACKUPS_ROOT` has to name a path that
exists on the development box. Unset, it defaults to a POSIX path and Windows
answers `Access is denied: '/backups'`.

---

## 2c. VS Code (turnkey)

The repo ships shared VS Code config in `.vscode/` (personal `settings.json`
stays git-ignored):

- **Recommended extensions** - on first open VS Code offers to install them
  (Python + Pylance, Vue Volar, ESLint, Docker). Accept.
- **Run the dev site** - Command Palette > "Tasks: Run Task" >
  **Dev site (backend + frontend)** starts both servers in parallel (backend
  on 5001, frontend on 5173). Individual tasks exist too. If the backend task
  invokes `flask run` rather than `run_dev.py`, expect the `.env` problem
  described in 2b and start the backend by hand instead.
- **Debug the backend** - the Run panel's **Flask API (:5001)** config runs
  the app under the debugger (breakpoints in routes/services, full
  stepping); **Pytest (current file)** debugs the open test file.
- **The CI gate** - task **Check: naming + tests + build** runs the same
  three checks CI runs, before you commit.

Prerequisite: the venv and `npm install` from 2b must be done first (the
tasks call `venv/` and `frontend/node_modules`).

---

## 3. The development loop

1. Make a change. Backend: `flask run` auto-reloads. Frontend: Vite hot-reloads.
2. Before committing, run the three gates. Easiest: in VS Code, Command
   Palette > "Tasks: Run Task" > **Check: naming + tests + build**. By hand
   in PowerShell:
   ```powershell
   venv\Scripts\python -m pytest tests/ -q     # backend
   cd frontend; npx vitest run; npm run build; cd ..
   bash scripts/check-naming-and-style.sh      # naming - runs via Git Bash
   ```
   CI runs all three on every push and pull request, so running them locally
   only saves the round trip. No hook is installed for you; to have the naming
   check run on every `git commit`, enable the shipped one once per clone with
   `git config core.hooksPath .githooks`. The naming check is a shell script,
   so it needs Git Bash (installed with Git for Windows) on Windows.
3. Commit in small, working steps. Subject: short, present tense, plain
   English; body says WHY. Read `CONTRIBUTING.md` before naming anything - the
   naming hook will reject snake_case DB columns, banned shorthand, and
   non-ASCII.

Seeing a change in the real app (not just tests) is the bar for "done" -
drive the actual flow in the browser.

---

## 4. Your first change (suggested)

Add a field to an existing list page. Or build a plugin end to end, starting at
[PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md).

---

<a id="contributing"></a>
## 5. Contributing back

This is the whole contributor workflow, for a core change and for a plugin
alike.

1. **Branch, never work on `main`** (section 1), and commit in working steps -
   each commit a checkpoint that passes its own tests.
2. **Run the three gates before every push** (section 3).
3. **Push and open a Pull Request against `main`.**

   ```powershell
   git push -u origin feat/<short-description>
   ```

   In the description: what changed, which plugin hooks it implements, any
   contract additions (these need a `__contract_version__` bump plus a
   `docs/PLUGIN-HOOKS.md` update in the SAME pull request - a docs-drift test
   fails the build otherwise), and any deliberate security posture. An endpoint
   that is open on purpose needs a decision record, not a comment.

4. **What a reviewer looks for:**
   - contract purity - core imported only via `shopdb.api` and
     `shopdb.plugins.base` (there is a guard test)
   - the naming convention, per `CONTRIBUTING.md`
   - a per-plugin migration chain, its `PLUGIN_TABLE_OWNERS` entry, and its
     expected-head declaration in `tests/test_plugin_migrations.py`
   - permissions both DECLARED and ENFORCED
   - tests for the invariants, not just the happy path
   - `default_enabled` correct for the plugin's nature

5. **After approval**, the maintainer lands the change on the mainline and the
   next published release commit includes it; your pull request is then closed
   as merged. Day-to-day history lives on the development mainline and the
   published repository carries the release line, so do not be surprised when
   your commits arrive squashed or folded into a release commit.

---

## Common setup problems

| Symptom | Cause / fix |
| --- | --- |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| `flask db upgrade` / `flask plugin upgrade-all` error 1071 (key too long) | MySQL 5.6 on the default Antelope file format, where an index prefix caps at 767 bytes - one utf8mb4 `VARCHAR(255)` is 1020. Start mysqld with `--innodb-file-format=Barracuda --innodb-large-prefix=ON` (both are also settable live with `SET GLOBAL`), keep `?charset=utf8mb4` on `DATABASE_URL`, and note that tables CREATED before those flags were on are silently `ROW_FORMAT=Compact` rather than `DYNAMIC` - 5.6 downgrades the row format with a warning instead of an error, so they keep the 767-byte cap until rebuilt with `ALTER TABLE <t> ROW_FORMAT=DYNAMIC`. MySQL 5.7+ and 8.0 need none of this. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
| Naming hook rejects a commit | you used snake_case on a DB-mirrored field or a banned acronym - see `CONTRIBUTING.md`. |
| Plugin toggle throws an internal error | app cannot write `instance/` (the plugin registry lives there) - fix directory permissions. |
| `Can't connect to MySQL on 127.0.0.1:3306` while the container is clearly up | the backend never read `.env` (started with `flask run` instead of `run_dev.py`), or `MYSQL_PORT` moved the container and `DATABASE_URL` still says 3306. |
| `Access is denied: '/backups'` on Windows | `SHOPDB_BACKUPS_ROOT` unset, so the backups code fell back to a POSIX path. Point it at a directory on this box and create it. |
| Backups can be requested but never run | expected outside Docker. The API queues a backup with a sentinel file; the work is done by a sidecar container, so a venv-only box queues and nothing executes. |