Files
shopdb-flask/shopdb/__init__.py
cproudlock 4a8bd138a9
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
feat(import): load a site's data from spreadsheets
Adopting a site means getting its asset register in. The HTTP import API suits a
site with a source system and someone to script against it; a sister site with a
spreadsheet and no developer needs something else, and that is the common case.

FOREIGN KEYS TAKE NAMES. This is the whole design. A CSV row has to say where an
asset is, and the database stores locationid, an integer. Requiring the number
means importing locations, reading back the generated ids and pasting them into
the asset sheet - a workflow nobody finishes. Every foreign key here accepts
either a numeric id or the referenced row's name:

    assetnumber,assettypeid,statusid,locationid
    CMM-01,Measuring Tool,Active,Gage Lab

The column keeps its database name, per CONTRIBUTING.md; the value is whatever
the operator actually knows. Names resolve across files in one run, so
assets.csv can reference a location that only exists because locations.csv was
read moments earlier. A name that does not resolve is reported with its line,
column and value, not as a foreign key violation from three layers down.

Dry run is the default, and writes go into the transaction either way - the
rollback is what makes it a dry run. Skipping the writes instead made every
cross-file reference fail, which is the one thing a folder-wide check exists to
verify. Validation covers every row before anything is written, so a typo on
line 400 cannot leave 399 rows imported. Files are matched on a natural key, so
correcting a spreadsheet and re-running updates rather than duplicates.

TEMPLATES ARE GENERATED, NOT MAINTAINED. "flask csv templates" builds them from
the live schema, annotated with required/optional and which file each foreign
key refers to. The prompt for this was a hand-written template set that had
invented columns on seven of eleven tables and named a table that does not
exist, while looking entirely plausible - and described an import mechanism
(a Data Import page, a flask import-csv command) that had never existed. A test
fails the build if a generated template ever offers a column the schema lacks.

User accounts are deliberately not importable: passwords do not belong in a
spreadsheet in either direction.

Verified end to end against MySQL 5.6 - a folder dry run catching one bad
reference, the fix, the commit, and a re-run reporting updates rather than
inserts. 16 tests.
2026-08-04 09:13:03 -04:00

275 lines
9.4 KiB
Python

"""Flask application factory."""
import os
import logging
from flask import Flask, send_from_directory
from .config import config
from .extensions import db, jwt, init_extensions
from .plugins import plugin_manager
# Platform contract version. See ADR-001 for the contract surface and
# ADR-002 for the bump rules. Plugins declare a compatible range in
# their manifest.json `core_version` field. Pre-1.0 (0.x) means the
# contract is still settling; sister sites should pin tight ranges.
# 0.3.0: shopdb.api expanded to the full plugin import surface (db, cache,
# model bases, core models, response + pagination helpers, employee_connection)
# so plugins no longer import internal core paths. Additive, hence minor bump.
# 0.4.0: removed the never-implemented get_searchable_fields hook (search is a
# core concern over the asset model) and wired the get_dashboard_widgets hook to
# a real consumer (/api/dashboard/widgets). Pre-1.0 contract reduction.
# 0.6.0: added the get_reports hook, consumed by GET /api/reports to merge
# plugin report cards into the Reports hub. Additive optional hook, minor bump.
# 0.7.0: added the four ADR-010 frontend-contribution hooks (get_settings_cards,
# get_asset_panels, get_map_overlays, get_asset_presentation), consumed by the
# GET /api/pluginui/* endpoints. Four additive optional hooks, one minor bump.
# 0.9.0: added the dualpath single-machine collapse helpers to shopdb.api
# (resolve_dualpath_pairs, dualpath_single_machine_enabled), consumed by the
# machines plugin list/detail to collapse dual-bay pairs. Two additive names,
# minor bump.
# 0.10.0: added the get_permissions hook so a plugin declares the RBAC
# permissions its own routes enforce, instead of core accumulating them in one
# catalog. Consumed by full_permission_catalog() (core + enabled plugins),
# which backs seeding, the role grid, and API-token scope validation. Additive
# optional hook, minor bump.
# 0.11.0: added service_token_authorized(scope) to shopdb.api so a plugin's
# unattended endpoints (e.g. the GE-Enforce fetch API) can authorize a scoped
# managed service token without importing core token internals. Additive name
# on the import surface, minor bump.
__contract_version__ = '0.15.0'
# Product release version (see ADR-007). The product version and the
# plugin-contract version above are distinct series with independent
# bump rules. Not part of the shopdb.api contract surface, so it is
# not re-exported there.
__version__ = '0.7.0'
def create_app(config_name: str = None) -> Flask:
"""
Application factory.
Args:
config_name: Configuration name ('development', 'production', 'testing')
Returns:
Configured Flask application
"""
if config_name is None:
config_name = os.environ.get('FLASK_ENV', 'development')
app = Flask(__name__, instance_relative_config=True)
config_class = config.get(config_name, config['default'])
# Production must validate its env-driven config before boot.
if config_name == 'production' and hasattr(config_class, 'validate'):
config_class.validate()
app.config.from_object(config_class)
# Load instance config if exists
app.config.from_pyfile('config.py', silent=True)
# Per-plugin collector keys (ADR-006) are dynamic env-vars
# (COLLECTOR_API_KEY_<PLUGINNAME>) that from_object cannot pick up because
# they are not class attributes. Copy them in explicitly so per-plugin
# credential isolation works in real deploys, not just tests.
for envname, envvalue in os.environ.items():
if envname.startswith('COLLECTOR_API_KEY_') and envvalue:
app.config[envname] = envvalue
# Ensure instance folder exists
os.makedirs(app.instance_path, exist_ok=True)
# Configure logging
configure_logging(app)
# Initialize extensions
init_extensions(app)
# Initialize plugin manager
with app.app_context():
plugin_manager.init_app(app, db)
# Register core blueprints
register_blueprints(app)
# Personal API token auth shim: recognize `Bearer shopdb_pat_...` before
# any JWT decode and mint a request-scoped JWT for the token's owner.
from .utils.apitoken_auth import install_apitoken_auth
install_apitoken_auth(app)
# Register CLI commands
register_cli_commands(app)
# Register error handlers
register_error_handlers(app)
# Serve Vue frontend
register_frontend_routes(app)
# JWT user loader (identity is a string in JWT, convert to int for DB lookup)
@jwt.user_lookup_loader
def user_lookup_callback(_jwt_header, jwt_data):
from .core.models import User
identity = jwt_data["sub"]
return db.session.get(User, int(identity))
return app
CORE_BLUEPRINT_NAMES = (
'auth',
'assets',
'modeltypes',
'plugins',
'vendors',
'models',
'businessunits',
'locations',
'operatingsystems',
'dashboard',
'dashboarddefaults',
'applications',
'supportteams',
'search',
'reports',
'collector',
'settings',
'auditlogs',
'users',
'customfields',
'setup',
'pluginui',
'apitokens',
'docs',
)
def register_blueprints(app: Flask):
"""Register core API blueprints from CORE_BLUEPRINT_NAMES.
Each entry maps to an attribute `<name>_bp` exported by
`shopdb.core.api` and a URL prefix `/api/<name>`. Adding a new
core resource is one entry in CORE_BLUEPRINT_NAMES, not a 3-line
edit in this function.
"""
from .core import api as api_module
api_prefix = '/api'
for name in CORE_BLUEPRINT_NAMES:
attr_name = f'{name}_bp'
if not hasattr(api_module, attr_name):
raise RuntimeError(
f'Core blueprint "{attr_name}" missing from shopdb.core.api. '
f'Either add it or remove "{name}" from CORE_BLUEPRINT_NAMES.'
)
bp = getattr(api_module, attr_name)
app.register_blueprint(bp, url_prefix=f'{api_prefix}/{name}')
def register_cli_commands(app: Flask):
"""Register Flask CLI commands."""
from .plugins.cli import plugin_cli
from .cli import db_cli, seed_cli, relationships_cli, csv_cli
app.cli.add_command(plugin_cli)
app.cli.add_command(db_cli)
app.cli.add_command(seed_cli)
app.cli.add_command(relationships_cli)
app.cli.add_command(csv_cli)
def register_error_handlers(app: Flask):
"""Register error handlers."""
from .utils.responses import error_response, ErrorCodes
from .exceptions import ShopDBException
@app.errorhandler(ShopDBException)
def handle_shopdb_exception(error):
http_codes = {
'NOT_FOUND': 404,
'UNAUTHORIZED': 401,
'FORBIDDEN': 403,
'CONFLICT': 409,
'VALIDATION_ERROR': 400,
}
http_code = http_codes.get(error.code, 400)
return error_response(
error.code,
error.message,
details=error.details,
http_code=http_code
)
@app.errorhandler(404)
def not_found_error(error):
return error_response(
ErrorCodes.NOT_FOUND,
'Resource not found',
http_code=404
)
@app.errorhandler(500)
def internal_error(error):
return error_response(
ErrorCodes.INTERNAL_ERROR,
'An internal error occurred',
http_code=500
)
@app.errorhandler(401)
def unauthorized_error(error):
return error_response(
ErrorCodes.UNAUTHORIZED,
'Authentication required',
http_code=401
)
def register_frontend_routes(app: Flask):
"""Serve Vue frontend static files."""
frontend_dist = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'frontend', 'dist')
@app.route('/', defaults={'path': ''})
@app.route('/<path:path>')
def serve_frontend(path):
# Don't serve API routes as frontend
if path.startswith('api/'):
from .utils.responses import error_response, ErrorCodes
return error_response(ErrorCodes.NOT_FOUND, 'API endpoint not found', http_code=404)
# Try to serve a static asset directly. send_from_directory handles
# the safe-join + 404 itself; no explicit existence probe needed
# (the probe was a path-traversal risk surface).
if path:
try:
response = send_from_directory(frontend_dist, path)
# Asset filenames carry a content hash, so a given URL never
# changes - cache them hard. Everything else stays revalidated.
if path.startswith('assets/'):
response.headers['Cache-Control'] = 'public, max-age=31536000, immutable'
else:
response.headers['Cache-Control'] = 'no-cache'
return response
except Exception:
pass
# index.html names the hashed chunks, so a stale copy points at files a
# deploy has already deleted and the SPA stops navigating. Always
# revalidate it.
response = send_from_directory(frontend_dist, 'index.html')
response.headers['Cache-Control'] = 'no-cache'
return response
def configure_logging(app: Flask):
"""Configure application logging."""
log_level = app.config.get('LOG_LEVEL', 'INFO')
logging.basicConfig(
level=getattr(logging, log_level),
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)