Act on the wj architectural review: races, limits, pool and contract gaps
Findings F2, F4, F5, F6, F7, F8, F9, F10, F11 and F13 from wj-architectural-review.md. F1/F1a (the non-sargable LOWER pattern) and F3 (the warranty sync holding a pooled connection) are being handled separately and are untouched here. - A collector check-in that raced another for the same PC lost everything: the unique index on the application fingerprint raised, nothing caught it, and the whole check-in rolled back, discarding state, events and hardware, answering the agent with form copy meant for a person. The insert now runs in a savepoint, reconciles what the other transaction already wrote, one row per savepoint so a second collision skips a row rather than the batch, and the check-in commits with a warning. This is the shape the delete side of the same function already had. - Enrollment is deliberately unauthenticated, so it now carries a per-address rate limit and a cap on pending requests, and the cap is a setting an admin can see and change. A flood can no longer bury the approval queue or grow the table without bound. - The fleet alert poll no longer formats debug strings on every request. - BusinessUnit is on the contract surface (0.34.0), so the endpoints plugin stops reaching into a core table with raw SQL. - A test walks every asset type descriptor and proves the endpoint it names really refuses a caller without the declared permission. - The plugin settings routes now need a permission; they were telling anyone which modules a site runs and where its components live. - The connection pool is sized explicitly with a short timeout, and the arithmetic against the server's connection ceiling is written next to the worker count instead of being left to defaults nobody chose. - Fleet alerts carry both the correct lowercase keys and the old camelCase pair. Trays older than 1.7.0 match keys case-sensitively and would have gone quiet on every PC not yet upgraded; the old pair goes when they are. - The dev server proxy points at localhost again, with environment variables for the remote-box workflow, and one developer's address is out of the server certificate page.
This commit is contained in:
@@ -10,6 +10,18 @@ class ConfigError(Exception):
|
||||
"""Raised when required configuration is missing or unsafe."""
|
||||
|
||||
|
||||
# Connection pool per worker process. Env-driven so a site can size it against
|
||||
# its own MySQL max_connections without editing code; see the arithmetic in
|
||||
# docker-compose.yml next to the worker count.
|
||||
#
|
||||
# Deliberately NOT named SQLALCHEMY_POOL_SIZE and friends: those were
|
||||
# Flask-SQLAlchemy 2.x config keys, removed in 3.x, and reusing the names would
|
||||
# suggest the extension still reads them. It reads SQLALCHEMY_ENGINE_OPTIONS.
|
||||
DB_POOL_SIZE = int(os.environ.get('DB_POOL_SIZE', 10))
|
||||
DB_MAX_OVERFLOW = int(os.environ.get('DB_MAX_OVERFLOW', 10))
|
||||
DB_POOL_TIMEOUT = int(os.environ.get('DB_POOL_TIMEOUT', 5))
|
||||
|
||||
|
||||
def _required_env(varname):
|
||||
"""Read an env var; raise ConfigError if missing or empty."""
|
||||
value = os.environ.get(varname)
|
||||
@@ -32,9 +44,32 @@ class Config:
|
||||
'mysql+pymysql://root:password@localhost:3306/shopdb_flask',
|
||||
)
|
||||
SQLALCHEMY_TRACK_MODIFICATIONS = False
|
||||
|
||||
# Connection pool, sized on purpose rather than left to SQLAlchemy.
|
||||
#
|
||||
# The defaults (pool_size 5, max_overflow 10, pool_timeout 30) were never
|
||||
# chosen: they simply applied, and nothing in the repo related them to the
|
||||
# gunicorn worker count or to the MySQL max_connections ceiling. The
|
||||
# arithmetic that ties the three together is written next to the worker
|
||||
# count in docker-compose.yml.
|
||||
#
|
||||
# Under gevent a pooled connection is held for the duration of whatever the
|
||||
# greenlet is doing, so a request that sits inside a slow external HTTP call
|
||||
# WITH THE SESSION OPEN pins a connection for that whole call. Hundreds of
|
||||
# greenlets can enter such a path at once and drain the pool, at which point
|
||||
# every other request blocks on checkout. Hence the fail-fast timeout: 5
|
||||
# seconds, not 30. A pool that is empty for five seconds is not going to be
|
||||
# fine in thirty, and a fast 500 is diagnosable where a 30-second hang looks
|
||||
# like a dead database.
|
||||
#
|
||||
# See PLUGIN-DEVELOPMENT / the plugin docs: a plugin must not hold a DB
|
||||
# session across external I/O.
|
||||
SQLALCHEMY_ENGINE_OPTIONS = {
|
||||
'pool_pre_ping': True,
|
||||
'pool_recycle': 300,
|
||||
'pool_size': DB_POOL_SIZE,
|
||||
'max_overflow': DB_MAX_OVERFLOW,
|
||||
'pool_timeout': DB_POOL_TIMEOUT,
|
||||
# Pin the client connection to utf8mb4 so reads/writes match the
|
||||
# utf8mb4 schema regardless of the server default charset. TestingConfig
|
||||
# (SQLite) replaces this whole dict, so the charset arg never reaches it.
|
||||
@@ -192,6 +227,13 @@ class TestingConfig(Config):
|
||||
# Off by default so login-heavy fixtures do not trip the limiter; tests
|
||||
# that exercise rate limiting flip it on via app.config override.
|
||||
AUTH_RATELIMIT_ENABLED = False
|
||||
# Same reason: the enrollment suites post to /enroll dozens of times, and
|
||||
# the limiter counts in a process-wide cache that outlives a test, so with
|
||||
# it on by default a budget spent in one module surfaces as an unrelated
|
||||
# failure in another - and which modules share a worker depends on the
|
||||
# xdist distribution, so the failure moves around. The tests that are about
|
||||
# the limiter turn it on themselves.
|
||||
ENDPOINTS_ENROLL_RATELIMIT_ENABLED = False
|
||||
SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
|
||||
SQLALCHEMY_ENGINE_OPTIONS = {
|
||||
'connect_args': {'check_same_thread': False},
|
||||
|
||||
Reference in New Issue
Block a user