ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refuse a value the database cannot store, instead of quietly changing it

Browse Source 
Wave 2 of the remediation plan. Eight findings from the feature test run were
the same defect twice: a write that reports success while storing something
other than what was sent.

The first half is a date parser that swallowed its own error. Two plugins
carried an identical private _parse_date returning None on ValueError, so an
unparseable date and a deliberately cleared one produced the same write.
PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty
updated" and stored NULL, dropping the record out of every expiry surface on
the strength of a typo; the measuring-tool calibration date did the same and
flipped the tool to 'unknown', out of the calibration report and the map
overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract
surface: an explicitly empty value clears the field, anything that is not a
date is refused with a 400 naming the field, and those are different intents
with different answers. The vendor-lookup paths keep a lenient wrapper on
purpose - a provider changing its date shape must not fail the refresh of an
otherwise good record, and no person typed that value.

The second half is why the truncation findings were possible at all. MySQL was
running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with
no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted
them. A 257-character username became its first 100 characters (and the next
distinct one then collided with it); an over-long asset number was cut at 50; a
malformed releasedate became '0000-00-00' and permanently broke every read of
that application. All with a 201. The mode is now set on the CONNECTION rather
than left to the server, so a site whose my.cnf disagrees still gets it.

That trades silent corruption for a DataError, which uncaught is a 500 - and a
500 says the server is broken when the truth is that the value does not fit. An
error handler maps it to a 400 naming the column where MySQL names one.
Verified by hand against MySQL 5.6: POST /api/users with a 257-character
username now answers 400 "That value is too long for username." and stores
nothing, where it previously stored a truncated row and answered 201.

Twelve tests. The date guard is verified by planting: restore the swallowing
parser and four of them fail. The DataError tests raise the error directly
because the suite runs on SQLite, which does not enforce VARCHAR length at all -
there is no value that would trigger it there, which is precisely why this went
unnoticed.

Scope: product
This commit is contained in:
cproudlock
2026-09-03 13:58:41 -04:00
parent aa8028f96e
commit 12392a361e
9 changed files with 419 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
plugins/measuringtools/api/routes.py
View File

@@ -20,6 +20,7 @@ from shopdb.api import (
success_response, error_response, paginated_response, ErrorCodes,
get_pagination_params, paginate_query,
require_permission, apply_import_timestamps,
DateValueError, parse_date_fields,
)
from ..models import MeasuringTool, MeasuringToolType, derive_status, STATUS_COLORS
@@ -27,16 +28,6 @@ from ..models import MeasuringTool, MeasuringToolType, derive_status, STATUS_COL
measuringtools_bp = Blueprint('measuringtools', __name__)
def _parse_date(value):
"""Accept 'YYYY-MM-DD' (or None/empty) -> date or None."""
if not value:
return None
try:
return datetime.strptime(str(value)[:10], '%Y-%m-%d').date()
except (ValueError, TypeError):
return None
def _merged(tool, today=None):
"""Asset core dict with the measuringtools extension nested under 'measuringtool'."""
result = tool.asset.to_dict() if tool.asset else {}
@@ -240,6 +231,12 @@ def _asset_type_id():
def create_tool():
"""Create a measuring tool: one Asset core row + one extension row."""
data = request.get_json() or {}
# Refused, not nulled - see the note at the update site.
try:
dates = parse_date_fields(
data, ('lastcalibrationdate', 'nextcalibrationdate'))
except DateValueError as bad:
return error_response(ErrorCodes.VALIDATION_ERROR, str(bad))
if not data.get('assetnumber'):
return error_response(ErrorCodes.VALIDATION_ERROR, 'assetnumber is required')
if Asset.query.filter_by(assetnumber=data['assetnumber']).first():
@@ -275,8 +272,8 @@ def create_tool():
assetid=asset.assetid,
measuringtooltypeid=data.get('measuringtooltypeid'),
calibrationintervaldays=data.get('calibrationintervaldays'),
lastcalibrationdate=_parse_date(data.get('lastcalibrationdate')),
nextcalibrationdate=_parse_date(data.get('nextcalibrationdate')),
lastcalibrationdate=dates.get('lastcalibrationdate'),
nextcalibrationdate=dates.get('nextcalibrationdate'),
calibrationprovider=(data.get('calibrationprovider') or '').strip() or None,
notes=(data.get('notes') or '').strip() or None,
)
@@ -309,6 +306,12 @@ def update_tool(tool_id: int):
return error_response(ErrorCodes.NOT_FOUND,
f'Measuring tool {tool_id} not found', http_code=404)
data = request.get_json() or {}
# Refused, not nulled - see the note at the update site.
try:
dates = parse_date_fields(
data, ('lastcalibrationdate', 'nextcalibrationdate'))
except DateValueError as bad:
return error_response(ErrorCodes.VALIDATION_ERROR, str(bad))
asset = tool.asset
# An UPDATE may not blank the asset number. Create validates it and the
@@ -339,9 +342,11 @@ def update_tool(tool_id: int):
if getattr(tool, key) != data[key]:
changes[key] = {'old': getattr(tool, key), 'new': data[key]}
setattr(tool, key, data[key])
for key in ('lastcalibrationdate', 'nextcalibrationdate'):
if key in data:
setattr(tool, key, _parse_date(data[key]))
# Refused, not nulled: a mistyped calibration date used to store NULL and
# answer 200, which flipped the tool to 'unknown' and dropped it out of the
# calibration report and the map overlay.
for key, parsed in dates.items():
setattr(tool, key, parsed)
if changes:
AuditLog.log('updated', 'MeasuringTool', entityid=tool.measuringtoolid,

49
plugins/warranty/api/routes.py
View File

@@ -19,6 +19,7 @@ from shopdb.api import (
db, Asset,
success_response, error_response, ErrorCodes,
require_permission,
DateValueError, parse_date, parse_date_fields,
)
from ..models import Warranty, WarrantyAsset
@@ -42,13 +43,16 @@ def _proof_dir():
return os.path.join(current_app.instance_path, 'warrantyproofs')
def _parse_date(value):
"""Accept 'YYYY-MM-DD' (or None/empty) -> date or None."""
if not value:
return None
def _vendor_date(value):
"""A date off a VENDOR response, which this code does not control.
Kept lenient on purpose, unlike a date a person typed: a provider that
starts returning a shape we do not parse must not fail the refresh of an
otherwise good record. The field is simply left as it was.
"""
try:
return datetime.strptime(value[:10], '%Y-%m-%d').date()
except (ValueError, TypeError):
return parse_date(value)
except DateValueError:
return None
@@ -203,13 +207,20 @@ def create_warranty():
vendor = (data.get('vendor') or '').strip()
if not vendor:
return error_response(ErrorCodes.VALIDATION_ERROR, 'vendor is required')
# Refused, not nulled. A mistyped end date used to store NULL and answer
# 200, which dropped the warranty out of every expiry surface on the
# strength of a typo.
try:
dates = parse_date_fields(data, ('startdate', 'enddate'))
except DateValueError as bad:
return error_response(ErrorCodes.VALIDATION_ERROR, str(bad))
warranty = Warranty(
vendor=vendor,
servicetag=(data.get('servicetag') or '').strip() or None,
provider=(data.get('provider') or 'manual').strip().lower(),
servicelevel=(data.get('servicelevel') or '').strip() or None,
startdate=_parse_date(data.get('startdate')),
enddate=_parse_date(data.get('enddate')),
startdate=dates.get('startdate'),
enddate=dates.get('enddate'),
notes=(data.get('notes') or '').strip() or None,
)
_apply_links(warranty, data.get('assetids'))
@@ -225,6 +236,10 @@ def update_warranty(warrantyid):
if not warranty:
return error_response(ErrorCodes.NOT_FOUND, 'Warranty not found', http_code=404)
data = request.get_json() or {}
try:
dates = parse_date_fields(data, ('startdate', 'enddate'))
except DateValueError as bad:
return error_response(ErrorCodes.VALIDATION_ERROR, str(bad))
if 'vendor' in data:
warranty.vendor = (data['vendor'] or '').strip() or warranty.vendor
if 'servicetag' in data:
@@ -233,10 +248,8 @@ def update_warranty(warrantyid):
warranty.provider = (data['provider'] or 'manual').strip().lower()
if 'servicelevel' in data:
warranty.servicelevel = (data['servicelevel'] or '').strip() or None
if 'startdate' in data:
warranty.startdate = _parse_date(data['startdate'])
if 'enddate' in data:
warranty.enddate = _parse_date(data['enddate'])
for field, parsed in dates.items():
setattr(warranty, field, parsed)
if 'notes' in data:
warranty.notes = (data['notes'] or '').strip() or None
if 'isactive' in data:
@@ -279,9 +292,9 @@ def refresh_warranty(warrantyid):
if result.get('servicelevel'):
warranty.servicelevel = result['servicelevel']
if result.get('startdate'):
warranty.startdate = _parse_date(result['startdate'])
warranty.startdate = _vendor_date(result['startdate'])
if result.get('enddate'):
warranty.enddate = _parse_date(result['enddate'])
warranty.enddate = _vendor_date(result['enddate'])
warranty.lastcheckeddate = datetime.now(timezone.utc).replace(tzinfo=None)
db.session.commit()
return success_response(_warranty_payload(warranty), message='Warranty refreshed')
@@ -363,8 +376,8 @@ def sync_dell():
existing.provider = 'dell'
existing.vendor = existing.vendor or 'Dell'
existing.servicelevel = found.get('servicelevel')
existing.startdate = _parse_date(found.get('startdate'))
existing.enddate = _parse_date(found.get('enddate'))
existing.startdate = _vendor_date(found.get('startdate'))
existing.enddate = _vendor_date(found.get('enddate'))
existing.servicetag = tag
existing.lastcheckeddate = now
updated += 1
@@ -372,8 +385,8 @@ def sync_dell():
warranty = Warranty(
vendor='Dell', provider='dell', servicetag=tag,
servicelevel=found.get('servicelevel'),
startdate=_parse_date(found.get('startdate')),
enddate=_parse_date(found.get('enddate')),
startdate=_vendor_date(found.get('startdate')),
enddate=_vendor_date(found.get('enddate')),
lastcheckeddate=now,
)
warranty.links.append(WarrantyAsset(assetid=assetid))