Act on the wj architectural review: races, limits, pool and contract gaps
Findings F2, F4, F5, F6, F7, F8, F9, F10, F11 and F13 from wj-architectural-review.md. F1/F1a (the non-sargable LOWER pattern) and F3 (the warranty sync holding a pooled connection) are being handled separately and are untouched here. - A collector check-in that raced another for the same PC lost everything: the unique index on the application fingerprint raised, nothing caught it, and the whole check-in rolled back, discarding state, events and hardware, answering the agent with form copy meant for a person. The insert now runs in a savepoint, reconciles what the other transaction already wrote, one row per savepoint so a second collision skips a row rather than the batch, and the check-in commits with a warning. This is the shape the delete side of the same function already had. - Enrollment is deliberately unauthenticated, so it now carries a per-address rate limit and a cap on pending requests, and the cap is a setting an admin can see and change. A flood can no longer bury the approval queue or grow the table without bound. - The fleet alert poll no longer formats debug strings on every request. - BusinessUnit is on the contract surface (0.34.0), so the endpoints plugin stops reaching into a core table with raw SQL. - A test walks every asset type descriptor and proves the endpoint it names really refuses a caller without the declared permission. - The plugin settings routes now need a permission; they were telling anyone which modules a site runs and where its components live. - The connection pool is sized explicitly with a short timeout, and the arithmetic against the server's connection ceiling is written next to the worker count instead of being left to defaults nobody chose. - Fleet alerts carry both the correct lowercase keys and the old camelCase pair. Trays older than 1.7.0 match keys case-sensitively and would have gone quiet on every PC not yet upgraded; the old pair goes when they are. - The dev server proxy points at localhost again, with environment variables for the remote-box workflow, and one developer's address is out of the server certificate page.
This commit is contained in:
@@ -167,6 +167,31 @@ services:
|
||||
# connections, is the binding resource.
|
||||
- --worker-class
|
||||
- gevent
|
||||
# DATABASE CONNECTION ARITHMETIC. Read this before changing the number.
|
||||
#
|
||||
# workers x (pool_size + max_overflow) + migrate + admin vs MySQL
|
||||
# max_connections
|
||||
#
|
||||
# Each worker process owns its own SQLAlchemy pool, so the ceiling is per
|
||||
# worker multiplied by the worker count. With the defaults in
|
||||
# shopdb/config.py (DB_POOL_SIZE 10, DB_MAX_OVERFLOW 10):
|
||||
#
|
||||
# 4 x (10 + 10) = 80
|
||||
# + the migrate service (a handful, and it has exited before api
|
||||
# starts, so it only overlaps during a manual upgrade)
|
||||
# + whatever a human has open (mysqldump, a client on the host)
|
||||
# = about 90 against the mysql:8.4 default max_connections of 151.
|
||||
#
|
||||
# The `command:` on the db service above sets charset and collation only,
|
||||
# so that 151 is the server default and nothing else in this repo relates
|
||||
# the two numbers. Raise --workers to 8 and the ceiling is 160 before any
|
||||
# overhead, which is ABOVE 151: MySQL then refuses connections, and the
|
||||
# failure shows up as errors from the database rather than anything in the
|
||||
# app. So change one of these and change the others deliberately:
|
||||
# more workers means either a smaller pool or --max-connections on db.
|
||||
#
|
||||
# pool_timeout is 5s, not the 30s default, so exhaustion fails fast and
|
||||
# legibly instead of hanging every request for half a minute.
|
||||
- --workers
|
||||
- "4"
|
||||
# Per-worker greenlet ceiling. 4 x 2000 is far above any realistic fleet;
|
||||
|
||||
Reference in New Issue
Block a user