Reset to page one when a filter changes, and let the catalog carry a real type

Two unrelated things found while looking at blank printer types.

Selecting a filter while past page one returned an empty list. The filter asked
the server for page 5 of a result set that now had one page, and the screen said
nothing matched. useListQuery already resets the page - setSearch and setExtra
both do - but the filter dropdowns bypassed it and called the loader directly.
Nine list pages now route through applyFilter, which calls setPage(1) when it
needs to and loads directly when already on page one, so the composable's URL
watcher does not also fire and fetch twice.

scripts/retype_models.py addresses why printer types cannot be derived. The
catalog types every printer model "Printer": true, and useless, since it does not
say whether the product is a laser, a plotter or a label printer. That answer is
a property of the model - every VersaLink C405 is a laser MFP - but nothing
recorded it, so nothing could derive it. Recording it on the MODEL means the
existing backfill fills every printer by exact name match, and a printer added
later inherits the right type the moment its model is chosen.

It exports the models needing a decision to CSV with a type suggested from the
model number, a person corrects the column, and applying it is a dry run unless
given --commit. A suggested type is refused unless it already exists in that
asset class's own vocabulary, which is what keeps the later name match working.

The suggestion order matters and got this wrong first time: a generic plotter
pattern matched "Zebra ZT411" and filed a label printer as a plotter. Brands now
come before generic patterns, and the review step exists precisely because a
confident wrong guess would type every asset using that model.

Verified on the development database: 24 printer models need a decision, 22 got
a sensible suggestion, applying them let all 42 printers match a printertype by
name, and the transaction rolled back cleanly.

frontend/src/views/settings/ModelsList.vue

@@ -14,7 +14,7 @@
         placeholder="Search models..."
         @input="debouncedSearch"
       />
-      <select v-model="vendorFilter" class="form-control" @change="loadModels">
+      <select v-model="vendorFilter" class="form-control" @change="applyFilter">
         <option value="">All Vendors</option>
         <option v-for="v in vendors" :key="v.vendorid" :value="v.vendorid">
           {{ v.vendor }}
@@ -272,6 +272,19 @@ onMounted(async () => {
   ])
 })
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadModels()
+}
+
 async function loadModels() {
   loading.value = true
   try {

plugins/applications/frontend/views/ApplicationsList.vue

@@ -14,7 +14,7 @@
         placeholder="Search applications..."
         @input="debouncedSearch"
       />
-      <select v-model="filter" class="form-control" @change="loadApplications">
+      <select v-model="filter" class="form-control" @change="applyFilter">
         <option value="installable">Installable Applications</option>
         <option value="all">All Applications</option>
         <option value="hidden">Hidden Applications</option>
@@ -117,6 +117,19 @@ onMounted(() => {
   loadApplications()
 })
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadApplications()
+}
+
 async function loadApplications() {
   loading.value = true
   try {

plugins/knowledgebase/frontend/views/KnowledgeBaseList.vue

@@ -19,7 +19,7 @@
         placeholder="Search articles..."
         @input="debouncedSearch"
       />
-      <select v-model="topicFilter" class="form-control" @change="loadArticles">
+      <select v-model="topicFilter" class="form-control" @change="applyFilter">
         <option value="">All Topics</option>
         <option
           v-for="app in topics"
@@ -134,6 +134,19 @@ onMounted(async () => {
   ])
 })
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadArticles()
+}
+
 async function loadArticles() {
   loading.value = true
   try {

plugins/network/frontend/views/NetworkDevicesList.vue

@@ -33,13 +33,13 @@
         placeholder="Search by hostname, asset #, serial..."
         @input="debouncedSearch"
       />
-      <select v-model="vendorFilter" class="form-control" @change="loadDevices">
+      <select v-model="vendorFilter" class="form-control" @change="applyFilter">
         <option value="">All Vendors</option>
         <option v-for="v in vendors" :key="v.vendorid" :value="v.vendorid">
           {{ v.vendor }}
         </option>
       </select>
-      <select v-model="locationFilter" class="form-control" @change="loadDevices">
+      <select v-model="locationFilter" class="form-control" @change="applyFilter">
         <option value="">All Locations</option>
         <option v-for="loc in locations" :key="loc.locationid" :value="loc.locationid">
           {{ loc.locationname }}
@@ -193,6 +193,19 @@ async function loadLocations() {
   }
 }
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadDevices()
+}
+
 async function loadDevices() {
   loading.value = true
   try {

plugins/network/frontend/views/SubnetsList.vue

@@ -14,13 +14,13 @@
         placeholder="Search subnets..."
         @input="debouncedSearch"
       />
-      <select v-model="vlanFilter" class="form-control" @change="loadSubnets">
+      <select v-model="vlanFilter" class="form-control" @change="applyFilter">
         <option value="">All VLANs</option>
         <option v-for="vlan in vlans" :key="vlan.vlanid" :value="vlan.vlanid">
           VLAN {{ vlan.vlannumber }} - {{ vlan.name }}
         </option>
       </select>
-      <select v-model="typeFilter" class="form-control" @change="loadSubnets">
+      <select v-model="typeFilter" class="form-control" @change="applyFilter">
         <option value="">All Types</option>
         <option value="ipv4">IPv4</option>
         <option value="ipv6">IPv6</option>
@@ -359,6 +359,19 @@ async function loadLocations() {
   }
 }
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadSubnets()
+}
+
 async function loadSubnets() {
   loading.value = true
   try {

plugins/network/frontend/views/VLANsList.vue

@@ -14,7 +14,7 @@
         placeholder="Search VLANs..."
         @input="debouncedSearch"
       />
-      <select v-model="typeFilter" class="form-control" @change="loadVLANs">
+      <select v-model="typeFilter" class="form-control" @change="applyFilter">
         <option value="">All Types</option>
         <option value="data">Data</option>
         <option value="voice">Voice</option>
@@ -220,6 +220,19 @@ onMounted(() => {
   loadVLANs()
 })
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadVLANs()
+}
+
 async function loadVLANs() {
   loading.value = true
   try {

plugins/notifications/frontend/views/NotificationsList.vue

@@ -15,13 +15,13 @@
       placeholder="Search notifications..."
       @input="debouncedSearch"
     />
-    <select v-model="selectedType" class="form-control" @change="loadNotifications">
+    <select v-model="selectedType" class="form-control" @change="applyFilter">
       <option value="">All Types</option>
       <option v-for="type in types" :key="type.notificationtypeid" :value="type.notificationtypeid">
         {{ type.typename }}
       </option>
     </select>
-    <select v-model="currentFilter" class="form-control" @change="loadNotifications">
+    <select v-model="currentFilter" class="form-control" @change="applyFilter">
       <option value="">All</option>
       <option value="current">Current Only</option>
       <option value="pinned">Pinned Only</option>
@@ -125,6 +125,19 @@ async function loadTypes() {
   }
 }
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadNotifications()
+}
+
 async function loadNotifications() {
   loading.value = true
   try {

plugins/printedparts/frontend/views/PrintedItemsList.vue

@@ -19,11 +19,11 @@
         @input="debouncedSearch"
       />
       <label class="lowstock-filter">
-        <input v-model="lowstockOnly" type="checkbox" @change="loadItems" />
+        <input v-model="lowstockOnly" type="checkbox" @change="applyFilter" />
         Low stock only
       </label>
       <label class="lowstock-filter">
-        <input v-model="includeRetired" type="checkbox" @change="loadItems" />
+        <input v-model="includeRetired" type="checkbox" @change="applyFilter" />
         Include retired
       </label>
     </div>
@@ -108,6 +108,19 @@ let searchTimeout = null
 
 onMounted(loadItems)
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadItems()
+}
+
 async function loadItems() {
   loading.value = true
   try {

plugins/usb/frontend/views/USBList.vue

@@ -15,7 +15,7 @@
         @input="debouncedSearch"
       />
       <label class="checkbox-label">
-        <input type="checkbox" v-model="showAvailableOnly" @change="loadDevices" />
+        <input type="checkbox" v-model="showAvailableOnly" @change="applyFilter" />
         Available Only
       </label>
     </div>
@@ -172,6 +172,19 @@ onMounted(() => {
   loadDevices()
 })
 
+// A filter change must go back to page 1. Selecting a filter while on page 5
+// asked the server for page 5 of a result set that now has one page, and the
+// list came back empty as though the filter matched nothing.
+//
+// setPage(1) writes the URL, which the composable's watcher picks up and
+// answers with onChange - so calling the loader as well would fetch twice.
+// Load directly only when already on page 1, where nothing changes and the
+// watcher stays silent.
+function applyFilter() {
+  if (page.value > 1) setPage(1)
+  else loadDevices()
+}
+
 async function loadDevices() {
   loading.value = true
   try {

scripts/retype_models.py

@@ -0,0 +1,230 @@
+"""Give catalog models a useful type, so their assets can inherit it.
+
+The catalog types every printer model as "Printer" - true, and useless. It says
+the product is a printer, not whether it is a laser, a plotter or a label
+printer. That finer answer is a property of the MODEL (every VersaLink C405 ever
+made is a laser MFP), but nothing records it, so nothing can derive it.
+
+Recording it in the catalog rather than on each printer means:
+
+  - backfill_vendor_from_model.py then fills every printer automatically, since
+    it matches an asset's type to its model's type BY EXACT NAME;
+  - a printer added later inherits the right type as soon as its model is
+    chosen, with no second step, ever;
+  - the catalog stops claiming a plotter and a label printer are the same thing.
+
+Two passes, because guessing at a model's type unreviewed is how a plotter ends
+up filed as a laser:
+
+    python scripts/retype_models.py --export printer-types.csv
+    ... open it, correct the newtype column, save ...
+    python scripts/retype_models.py --apply printer-types.csv
+    python scripts/retype_models.py --apply printer-types.csv --commit
+
+A newtype is REFUSED unless it already exists in that asset class's own type
+vocabulary - "Laser" is accepted for printers because printertypes has it,
+"Laserjet" is not. That is what keeps the name match working afterwards.
+
+Defaults to printers. --class machines|computers|network for the others.
+"""
+
+import argparse
+import csv
+import os
+import re
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+# Each asset class, its own type vocabulary, and the modeltypes.category that a
+# newly created model type should carry.
+CLASSES = {
+    'printers':  ('printers',       'printertypes',       'printertype',       'Printer'),
+    'machines':  ('machines',       'machinetypes',       'machinetype',       'Equipment'),
+    'computers': ('computers',      'computertypes',      'computertype',      'PC'),
+    'network':   ('networkdevices', 'networkdevicetypes', 'networkdevicetype', 'Network'),
+}
+
+# Model-number keywords that suggest a type. A SUGGESTION only - it lands in the
+# CSV for a person to accept or correct, and is never applied on its own.
+# ORDER MATTERS - first match wins, so the specific brands come before the
+# generic patterns. A bare t\d{3,4} for plotters used to match "Zebra ZT411" and
+# file a label printer as a plotter, which is exactly the sort of confident
+# mistake the review step exists to catch. The pattern is now explicit instead.
+HINTS = [
+    (r'zebra|\bzt\d|\bzd\d|\bgk\d|\bgx\d|label|datamax|intermec', 'Label'),
+    (r'designjet|plotter|latex|\bdj\s?t\d{3,4}', 'Plotter'),
+    (r'datacard|zxp|card|cr80', 'Card'),
+    (r'thermal|tsp\d|tm-t', 'Thermal'),
+    (r'deskjet|officejet|inkjet|pixma', 'Inkjet'),
+    (r'\bml-\d|dot ?matrix|lq-\d|fx-\d', 'Dot Matrix'),
+    # Multifunction before plain laser: an MFP is also a laser, and the more
+    # specific answer is the useful one.
+    (r'mfp|mfc|workcentre|altalink|versalink|\bm\d{3}f', 'MFP'),
+    (r'laserjet|laser|phaser|\bls\b|\bp\d{4}', 'Laser'),
+]
+
+# Types the catalog uses that say nothing about the product. LocationOnly is the
+# classic-ASP placeholder that also left 134 machines untyped.
+PLACEHOLDERS = {'printer', 'locationonly', 'pc', 'equipment', 'network', 'computer', ''}
+
+
+def suggest(modelnumber, description=''):
+    haystack = ('%s %s' % (modelnumber or '', description or '')).lower()
+    for pattern, answer in HINTS:
+        if re.search(pattern, haystack):
+            return answer
+    return ''
+
+
+def vocabulary(connection, typetable, typename):
+    from sqlalchemy import text
+    rows = connection.execute(text('SELECT %s FROM %s ORDER BY 1' % (typename, typetable))).fetchall()
+    return [r[0] for r in rows if r[0]]
+
+
+def export(connection, classname, path, include_all):
+    from sqlalchemy import text
+    assettable, typetable, typename, _ = CLASSES[classname]
+    rows = connection.execute(text("""
+        SELECT mo.modelnumberid          AS modelnumberid,
+               mo.modelnumber            AS modelnumber,
+               COALESCE(v.vendor, '')    AS vendor,
+               COALESCE(mo.description, '') AS description,
+               COALESCE(mt.modeltype, '')   AS currenttype,
+               COUNT(a.modelnumberid)    AS assetcount
+        FROM {assettable} a
+        JOIN models mo          ON a.modelnumberid = mo.modelnumberid
+        LEFT JOIN vendors v     ON mo.vendorid = v.vendorid
+        LEFT JOIN modeltypes mt ON mo.modeltypeid = mt.modeltypeid
+        GROUP BY mo.modelnumberid, mo.modelnumber, v.vendor, mo.description, mt.modeltype
+        ORDER BY assetcount DESC, mo.modelnumber
+    """.format(assettable=assettable))).fetchall()
+
+    known = {v.lower() for v in vocabulary(connection, typetable, typename)}
+    out = []
+    for row in rows:
+        current = (row.currenttype or '').strip()
+        # Already a real answer in this class's vocabulary: nothing to decide.
+        if not include_all and current.lower() in known:
+            continue
+        out.append({
+            'modelnumberid': row.modelnumberid,
+            'modelnumber': row.modelnumber,
+            'vendor': row.vendor,
+            'assets': row.assetcount,
+            'currenttype': current,
+            'suggested': suggest(row.modelnumber, row.description),
+            'newtype': suggest(row.modelnumber, row.description),
+        })
+
+    with open(path, 'w', newline='', encoding='utf-8-sig') as handle:
+        writer = csv.DictWriter(handle, fieldnames=[
+            'modelnumberid', 'modelnumber', 'vendor', 'assets',
+            'currenttype', 'suggested', 'newtype'])
+        writer.writeheader()
+        writer.writerows(out)
+
+    print('Wrote %s with %d model(s) to review.' % (path, len(out)))
+    print('Valid values for newtype (%s):' % classname)
+    print('    %s' % ', '.join(vocabulary(connection, typetable, typename)))
+    print('')
+    print('The newtype column is pre-filled with a guess from the model number.')
+    print('CHECK EVERY ROW - a wrong guess types every asset using that model.')
+    print('Clear newtype to leave a model alone.')
+    return 0
+
+
+def apply(connection, classname, path, commit):
+    from sqlalchemy import text
+    _, typetable, typename, category = CLASSES[classname]
+    known = {v.lower(): v for v in vocabulary(connection, typetable, typename)}
+
+    with open(path, newline='', encoding='utf-8-sig') as handle:
+        rows = list(csv.DictReader(handle))
+
+    planned, refused = [], []
+    for row in rows:
+        newtype = (row.get('newtype') or '').strip()
+        if not newtype:
+            continue
+        if newtype.lower() not in known:
+            refused.append((row.get('modelnumber', '?'), newtype))
+            continue
+        planned.append((int(row['modelnumberid']), row.get('modelnumber', '?'),
+                        known[newtype.lower()], int(row.get('assets') or 0)))
+
+    for modelnumber, newtype in refused:
+        print('  REFUSED %-28s "%s" is not in %s' % (modelnumber, newtype, typetable))
+    if refused:
+        print('  (add it to %s first if it is a real type, or correct the spelling)' % typetable)
+        print('')
+
+    print('%d model(s) would be retyped, covering %d asset(s):'
+          % (len(planned), sum(p[3] for p in planned)))
+    for _, modelnumber, newtype, count in planned[:40]:
+        print('    %-32s -> %-12s (%d asset(s))' % (modelnumber[:32], newtype, count))
+    if len(planned) > 40:
+        print('    ... and %d more' % (len(planned) - 40))
+
+    if not commit:
+        print('')
+        print('DRY RUN - nothing written. Re-run with --commit to apply.')
+        return 0
+
+    for modelnumberid, _, newtype, _ in planned:
+        typeid = connection.execute(
+            text('SELECT modeltypeid FROM modeltypes WHERE modeltype = :n'),
+            {'n': newtype}).scalar()
+        if typeid is None:
+            # The name is valid for this asset class but the catalog has no such
+            # model type yet. Create it, tagged with this class's category.
+            connection.execute(
+                text('INSERT INTO modeltypes (modeltype, category) VALUES (:n, :c)'),
+                {'n': newtype, 'c': category})
+            typeid = connection.execute(
+                text('SELECT modeltypeid FROM modeltypes WHERE modeltype = :n'),
+                {'n': newtype}).scalar()
+        connection.execute(
+            text('UPDATE models SET modeltypeid = :t WHERE modelnumberid = :m'),
+            {'t': typeid, 'm': modelnumberid})
+
+    print('')
+    print('Committed. %d model(s) retyped.' % len(planned))
+    print('Now run:  python scripts/backfill_vendor_from_model.py --commit')
+    print('which copies the type onto every asset using those models.')
+    return 0
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__,
+                                     formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument('--class', dest='classname', default='printers',
+                        choices=sorted(CLASSES), help='asset class (default printers)')
+    parser.add_argument('--export', metavar='CSV', help='write models needing a type')
+    parser.add_argument('--apply', metavar='CSV', help='apply a reviewed file')
+    parser.add_argument('--all', action='store_true',
+                        help='with --export, include models that already have a valid type')
+    parser.add_argument('--commit', action='store_true', help='write the changes')
+    args = parser.parse_args()
+
+    if not args.export and not args.apply:
+        parser.error('give --export or --apply')
+
+    from shopdb import create_app
+    from shopdb.extensions import db
+
+    app = create_app()
+    with app.app_context():
+        connection = db.session.connection()
+        if args.export:
+            code = export(connection, args.classname, args.export, args.all)
+        else:
+            code = apply(connection, args.classname, args.apply, args.commit)
+            if args.commit:
+                db.session.commit()
+    return code
+
+
+if __name__ == '__main__':
+    sys.exit(main())