Release the database connection across Dell warranty lookups

Both Dell routes held a pooled connection open for the duration of the
vendor HTTP call. sync_dell was the severe case: bulk_lookup asks Dell
about 100 service tags per request and calls them sequentially, so a
fleet-sized sync pinned one connection for minutes. On gevent workers
that exhausts the pool and fails every request in the application, not
just warranty.

Copy the values the lookup needs into locals, commit to return the
connection, then reacquire for the writes. The locals are load-bearing:
a commit expires loaded attributes, so reading them afterwards would
quietly check a connection back out and restore the defect.

Preload the existing warranty links in one query. The write loop ran a
query per asset plus one per link, which would have traded the long
network hold for a long write transaction.

Transaction boundaries do not show up in behavioural tests, so the new
tests assert session state at the moment the provider is called. Both
were confirmed to fail against the unfixed code, including the
post-commit attribute-access variant that leaves all other tests green.

Single commit and the existing error handling are unchanged, so the
sync stays all-or-nothing.
This commit is contained in:
benjamin.baker
2026-09-19 08:54:59 -04:00
parent e00d86827b
commit 530f6a60f8
2 changed files with 200 additions and 6 deletions

View File

@@ -281,14 +281,27 @@ def refresh_warranty(warrantyid):
warranty = db.session.get(Warranty, warrantyid)
if not warranty:
return error_response(ErrorCodes.NOT_FOUND, 'Warranty not found', http_code=404)
provider = get_provider(warranty.provider)
# Copy to locals and end the transaction before the vendor call: the lookup
# can block for the full HTTP timeout, and holding a pooled connection
# across it starves the pool under load. Reading ORM attributes after the
# commit would check a connection straight back out.
providername = warranty.provider
servicetag = warranty.servicetag
vendor = warranty.vendor
db.session.commit()
provider = get_provider(providername)
try:
result = provider.lookup(warranty.servicetag, warranty.vendor)
result = provider.lookup(servicetag, vendor)
except (ProviderNotConfigured, WarrantyLookupError) as exc:
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc), http_code=400)
if not result:
return error_response(ErrorCodes.VALIDATION_ERROR,
'This warranty is manual - nothing to refresh.', http_code=400)
warranty = db.session.get(Warranty, warrantyid)
if not warranty:
return error_response(ErrorCodes.NOT_FOUND, 'Warranty not found', http_code=404)
if result.get('servicelevel'):
warranty.servicelevel = result['servicelevel']
if result.get('startdate'):
@@ -342,11 +355,31 @@ def sync_dell():
'candidates': 0, 'matched': 0, 'created': 0, 'updated': 0,
}, message='No PCs need a warranty lookup.')
# by_tag is plain strings and ints, so it survives the commit. Release the
# connection before bulk_lookup: Dell is called 100 tags at a time and a
# fleet-sized sync would otherwise pin a pooled connection for minutes.
db.session.commit()
try:
results = provider.bulk_lookup(list(by_tag.keys()))
except (ProviderNotConfigured, WarrantyLookupError) as exc:
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc), http_code=400)
# Preload every warranty already linked to a candidate asset in one query.
# Done per asset this was a query per asset plus one per link, which would
# just trade the long network hold for a long write transaction.
matched_assetids = [assetid
for tag, assetids in by_tag.items() if results.get(tag)
for assetid in assetids]
existing_by_assetid = {}
if matched_assetids:
rows = (db.session.query(WarrantyAsset.assetid, Warranty)
.join(Warranty, Warranty.warrantyid == WarrantyAsset.warrantyid)
.filter(WarrantyAsset.assetid.in_(matched_assetids))
.all())
for assetid, warranty in rows:
existing_by_assetid.setdefault(assetid, []).append(warranty)
created = updated = matched = 0
now = datetime.now(timezone.utc).replace(tzinfo=None)
for tag, assetids in by_tag.items():
@@ -357,15 +390,13 @@ def sync_dell():
for assetid in assetids:
# Reuse an existing Dell warranty for this asset if there is one.
existing = None
for link in WarrantyAsset.query.filter_by(assetid=assetid).all():
candidate = db.session.get(Warranty, link.warrantyid)
for candidate in existing_by_assetid.get(assetid, []):
# Reuse a warranty that is Dell by ANY signal, not just
# provider: a manually-added or imported Dell warranty carries
# provider 'manual' (create default) but the same service tag or
# a "Dell" vendor. Matching only provider=='dell' made re-check
# duplicate every one of those instead of updating it.
if candidate and (
candidate.provider == 'dell'
if (candidate.provider == 'dell'
or (candidate.servicetag or '').strip().upper() == tag
or (candidate.vendor or '').strip().lower() == 'dell'):
existing = candidate