shopdb-flask/docs/PLUGIN-EXTERNAL-REPO.md

<!-- docmeta
purpose: howto
audience: adopter, developer
status: current
summary: Build, pin and CI-test a plugin that lives in your own git repo and drops into a site's plugins/ directory.
loadwith: PLUGIN-HOOKS.md
-->

# Developing a plugin in its own repo

This guide is for a sister GE Aerospace site (or any third party) building a
shopdb-flask plugin in a git repo it owns, outside the framework tree. It is
the "external plugin" path in [ADR-003](adr/ADR-003-plugin-distribution.md):
the framework ships a bundled set, and you drop your own plugin into
`<framework>/plugins/<name>/` by clone, submodule, or symlink. No pip packaging
is required for v1 (pip distribution is deferred to v2 per ADR-003).

> **Windows:** use `venv\Scripts\python` and `$env:FLASK_APP="shopdb"` (not
> `export`). Onboarding: [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).

If you have not written a plugin before, start with
[PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) and the hook reference in
[PLUGIN-HOOKS.md](PLUGIN-HOOKS.md). This document only covers the parts that are
different when the plugin lives in its own repo.

## Recommended repo layout

Your repo root holds the plugin directory contents directly, so the whole repo
can be dropped in at `<framework>/plugins/<name>/`. A plugin named `shipping`
in a repo named `sitecode-shipping` looks like this:

```
sitecode-shipping/            # your git repo root == the plugin directory
  manifest.json           # required: name, version, description, core_version
  plugin.py               # required: your BasePlugin subclass
  __init__.py
  models/
    __init__.py           # exports every model
    shipping.py
  api/
    __init__.py
    routes.py             # Flask Blueprint returned by get_blueprint()
  migrations/             # per-plugin Alembic chain (ADR-008), if you own tables
    env.py
    script.py.mako
    versions/
      0001_shipping_baseline.py
  tests/                  # your own tests (the CI harness runs these)
    test_shipping.py
  README.md               # what it tracks, who maintains it, where to file issues
```

Notes:

- `manifest.json` is the single source of truth for metadata (ADR-002). The
  `name` field must match the directory name the site installs it under and
  follows the framework naming convention (lowercase concatenated, no
  underscores or dashes). Prefix site-specific names with the site code when a
  collision across sites is possible, e.g. `sitecode-shipping`
  (see [PLUGINS.md](PLUGINS.md) naming policy and ADR-003).
- `migrations/` is only needed if your plugin owns tables. A plugin built
  outside the tree never had its tables created by the framework's core chain,
  so its `0001` is a REAL baseline that creates them, not a stamp-only anchor.
  That distinction is explained once, in [PLUGINS.md](PLUGINS.md#migrations).
- Import core code only through `shopdb.api` and `shopdb.plugins.base`. The
  surface, and what happens when you reach past it, are in
  [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md#import-surface).

## Local dev workflow

Symlinking lets you edit in your own repo while the framework loads the plugin
live. The loader discovers a symlinked directory the same as a real one.

```bash
# 1. Clone the framework and your plugin repo side by side.
git clone https://github.com/ge-aero/shopdb-flask.git
git clone https://github.com/ge-aero/sitecode-shipping.git

# 2. Symlink your repo into the framework's plugins/ directory.
#    The link name is the plugin name from your manifest.json.
cd shopdb-flask
ln -s ../../sitecode-shipping plugins/shipping
#    (use an absolute path if you prefer: ln -s "$(pwd)/../sitecode-shipping" plugins/shipping)
#
# Windows: use a directory junction instead of ln -s. In an ADMIN prompt
# (or with Developer Mode on) from the shopdb-flask dir:
#    mklink /D plugins\shipping ..\..\sitecode-shipping
# The plugin loader treats a junction the same as a real directory.

# 3. Set up the framework as usual.
python3 -m venv venv
venv/bin/pip install -r requirements-dev.txt

# 4. Install (enable) your plugin.
venv/bin/flask plugin install shipping

# 5. If your plugin owns tables, apply its migrations.
venv/bin/flask db upgrade            # core chain
venv/bin/flask plugin upgrade-all    # plugin chains, including yours

# 6. Run tests.
venv/bin/python -m pytest tests/test_plugin_contract.py
venv/bin/python -m pytest ../sitecode-shipping/tests
```

Edits in `../sitecode-shipping` are picked up on the next framework restart, because
the symlink points back at your working tree.

### If your plugin owns tables, step 5 needs a change to the framework

This is the one place an external plugin is not self-contained, and it is better
said plainly than discovered at the first migration.

`flask plugin upgrade-all` builds each plugin's metadata from
`PLUGIN_TABLE_OWNERS` in `shopdb/plugins/alembic_template.py`, and raises rather
than guessing when a plugin has no entry:

```
RuntimeError: PLUGIN_TABLE_OWNERS has no entry for plugin 'shipping'.
Update shopdb/plugins/alembic_template.py.
```

So a plugin that owns tables needs its table names registered in that dictionary
in the framework repository. It is deliberate - the registry is what stops one
plugin's migration touching another's tables, and `tests/test_plugin_migrations.py`
tests it - but it does mean a table-owning external plugin is a two-repository
change: yours, plus a one-line addition upstream.

Two ways to live with it:

- **Send the entry upstream.** One line in `PLUGIN_TABLE_OWNERS` plus one in
  `EXPECTED_HEAD_REVISION`, and your plugin is a normal citizen from then on.
- **Own no tables.** A plugin that stores nothing of its own - a report, a
  dashboard card, a settings page over existing models - has nothing to
  register and stays entirely in your repository. More plugins fit this than
  expect to.

## core_version pinning for sister sites

The framework is pre-1.0. Under semver, any 0.x minor bump is allowed to break
the contract, and this project uses that latitude (see the history in
[CONTRACT-STABILITY.md](CONTRACT-STABILITY.md)). So pin a TIGHT range in your
`manifest.json`, per ADR-002 (pip-style `>=,<`), admitting only the contract
minor you tested against and not the whole 0.x line.

Read the current contract version from [PROJECT-MAP.md](PROJECT-MAP.md), which
is generated from `shopdb/__init__.py` - a version typed into this page is
wrong within a fortnight, and a plugin pinned to a stale one is refused at
startup. Take that minor and the next one up: if the map says `0.31.0`, your
range is `>=0.31.0,<0.32.0`:

```json
{
  "name": "shipping",
  "version": "1.0.0",
  "description": "Tracks shipping-station scanners and label printers",
  "core_version": ">=<the map's minor>,<<the next minor>",
  "dependencies": []
}
```

Do NOT pin the loose `>=0.2.0,<1.0.0` default that `PluginMeta` falls back to.
That default exists so bundled plugins keep loading across minor bumps; an
external plugin should be deliberate and re-test before widening its range.

What happens at load time on a mismatch (ADR-002):

| Environment | Behavior on core_version mismatch |
|-------------|-----------------------------------|
| dev / test  | The loader re-raises. Startup fails loud so you notice immediately. |
| production  | The loader logs an error, marks the plugin incompatible, and excludes it from registration. The rest of the app still starts. |

When you move a site to a newer framework, bump your `core_version` upper bound
only after the harness below passes against the new ref.

## CI recipe

`scripts/test-external-plugin.sh` (in the framework repo) stands up a throwaway
framework at a pinned ref, drops your plugin in as a symlink, and runs the
framework contract tests plus your own `tests/`. It has two modes:

- **CI / remote** (default): clones the framework at `FRAMEWORK_REF` and builds
  a fresh venv. Needs network access to git and PyPI.
- **Local / offline**: set `LOCAL_FRAMEWORK` to a framework checkout on disk.
  The script exports that checkout at HEAD with `git archive` and reuses its
  existing venv, so it runs with no internet. Useful for air-gapped verification.

```bash
# CI: test the plugin in the current repo against a pinned tag
PLUGIN_DIR=. FRAMEWORK_REF=v0.5.0 scripts/test-external-plugin.sh

# Offline: test against a framework checkout already on disk
LOCAL_FRAMEWORK=/opt/shopdb-flask PLUGIN_DIR=. scripts/test-external-plugin.sh
```

Its header comment documents every input (`PLUGIN_DIR`, `FRAMEWORK_REF`,
`FRAMEWORK_URL`, `LOCAL_FRAMEWORK`); read it in the file rather than from a copy
on a wiki page, because a copied script is one that stops matching the script
that runs. In order it resolves the plugin name from `manifest.json`, stands up
the throwaway framework, symlinks your plugin into its `plugins/`, clears
`instance/plugins.json`, runs the framework's contract tests, then runs your
plugin's own `tests/` if it has any. Nonzero exit means the plugin is not
compatible with that framework ref.

### What the harness does and does not check

The symlinked plugin is discovered and loaded by the plugin loader when the app
starts under test. That validates the things that break a real install: the
manifest parses, the `core_version` range admits the framework's
`__contract_version__` (an out-of-range plugin makes startup fail loud, so the
contract test run errors and the script exits nonzero), models expose
`__tablename__`, and hooks return the right shapes.

The import-surface scan (`test_plugins_only_import_contract_surface`) covers
symlinked plugins too: the scanner resolves each plugin directory before
walking it, because `Path.rglob` alone does not descend symlinks (pinned by
`test_import_scan_covers_symlinked_plugins`). You can additionally keep an
equivalent import-surface assertion in your own `tests/`, so violations fail
in your repo's CI even when run without the framework harness. A minimal
version:

```python
# tests/test_import_surface.py
import re
from pathlib import Path

ALLOWED = ('shopdb.api', 'shopdb.plugins.base')
IMPORT_RE = re.compile(r'^\s*(?:from (shopdb[\w.]*) import|import (shopdb[\w.]*))', re.MULTILINE)

def test_only_contract_surface_imports():
    root = Path(__file__).resolve().parent.parent
    bad = []
    for path in root.rglob('*.py'):
        if '__pycache__' in path.parts or 'migrations' in path.parts:
            continue
        for m in IMPORT_RE.finditer(path.read_text()):
            mod = m.group(1) or m.group(2)
            if mod and not any(mod == a or mod.startswith(a + '.') for a in ALLOWED):
                bad.append(f'{path.name}: {mod}')
    assert not bad, 'core imports outside shopdb.api / shopdb.plugins.base: ' + '; '.join(bad)
```

## GitHub Actions example

For a plugin repo hosted on GitHub, a workflow calling the harness against a
pinned framework tag (config only, adjust URLs and ref to your setup):

```yaml
name: plugin-contract

on:
  push:
  pull_request:

jobs:
  contract:
    runs-on: ubuntu-latest
    env:
      FRAMEWORK_REF: v0.5.0
      FRAMEWORK_URL: https://github.com/ge-aero/shopdb-flask.git
    steps:
      - name: Check out the plugin
        uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.14'
      - name: Fetch the harness from the framework
        run: |
          git clone --depth 1 --branch "$FRAMEWORK_REF" "$FRAMEWORK_URL" /tmp/framework
      - name: Run the contract harness against this plugin
        run: |
          PLUGIN_DIR="$GITHUB_WORKSPACE" \
          FRAMEWORK_REF="$FRAMEWORK_REF" \
          FRAMEWORK_URL="$FRAMEWORK_URL" \
          bash /tmp/framework/scripts/test-external-plugin.sh
```

## See also

- [PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) - generate and install a plugin fast
- [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md) - the full hook and import-surface reference
- [CONTRACT-STABILITY.md](CONTRACT-STABILITY.md) - what is settled vs still churning before 1.0
- [ADR-002](adr/ADR-002-plugin-versioning.md) - contract versioning and core_version ranges
- [ADR-003](adr/ADR-003-plugin-distribution.md) - the bundled vs external distribution model
- [ADR-008](adr/ADR-008-plugin-migration-ownership.md) - per-plugin migration ownership