Toner: count cartridge changes that happened, and rate a burst as one burst

Two things reported from the floor, one cause each.

"5 CHANGES IN 90 DAYS, THAT'S HARD TO BELIEVE." It was. A replacement was any
+10 rise between readings, with no check on where it landed, so two shapes that
are not swaps scored as swaps: a supply reading 0 or near-0 while it was out of
the machine and then reading normally again, and a coarse gauge ticking back up
after a reseat or a power cycle.

A swap must now also LAND near full, because that is what a new cartridge reads,
and a single dip that RECOVERS to roughly where it came from is dropped before
anything looks at it. The dip filter keys on shape rather than cause, which is
why it holds for all of them - a supply pulled out to be shaken, a door open
mid-poll, or a site whose preprocessing maps the Printer MIB's unknown
sentinels onto 0. It is NOT a Zabbix timeout: an item that does not answer
records nothing rather than writing a zero. A genuine near-empty reading before
a real swap does not recover, it jumps to full, so it survives and its swap
still counts.

find_replacements and current_run now read one predicate. When they disagreed, a
phantom rise reset the run and threw away the history the estimate needed - so
the bad count was quietly damaging the rate as well, which is why both were
wrong at once. Expect replacement counts to FALL and per-cartridge history to
lengthen.

A BURST BIASED THE RATE FOR THE LIFE OF THE CARTRIDGE. The rate was the slope
between the first and last reading of the run, and two endpoints cannot tell
"steady" from "burst then stopped". A cartridge that lost 20 percent in two days
and then barely moved for a month read as 0.83 percent/day forever after, so the
report kept promising it would run out long after printing slowed. It is now the
median of the per-interval rates: the burst is one interval among many rather
than one of two points. Rising intervals are dropped as noise; flat ones stay in
at zero, because a cartridge that did not move is real information. If every
interval is flat or rising yet the run dropped overall, it falls back to the
whole-run slope rather than reporting nothing.

Where the intervals disagree by 5x or more the rate carries a marker and an
explanation on hover. The number is still the best estimate available; the flag
stops it reading as a measurement.

The "Changed" column is "Replacements", and its cell says "2 in 90d" rather than
"2 / 90d", which was read as a date, a ratio and a version number.

plugins/printers/frontend/views/TonerForecast.vue

@@ -90,7 +90,7 @@
                 <th class="collevel">Level</th>
                 <th>Runs out</th>
                 <th>Rate</th>
-                <th>Changed</th>
+                <th title="Cartridge changes detected, over the days of history behind this row">Replacements</th>
               </tr>
             </thead>
             <tbody>
@@ -123,11 +123,20 @@
                   <span class="days" :class="c.band">{{ daysText(c) }}</span>
                 </td>
                 <td class="muted small">
-                  {{ c.burnrateperday != null ? c.burnrateperday + '%/day' : '-' }}
+                  <template v-if="c.burnrateperday != null">
+                    {{ c.burnrateperday }}%/day<!--
+                    --><span v-if="c.rateunstable" class="unstable"
+                             title="This cartridge's usage has varied a lot between readings - a burst then a lull, or the reverse. The estimate is the best available, not a measurement.">~</span>
+                  </template>
+                  <template v-else>-</template>
                 </td>
                 <td class="muted small">
-                  {{ c.replacements || 0 }}
-                  <span v-if="c.basisdays">/ {{ c.basisdays }}d</span>
+                  <!-- "2 / 90d" read as a date, a ratio, or a version to
+                       everyone who saw it. Say the unit. -->
+                  <template v-if="c.basisdays">
+                    {{ c.replacements || 0 }} in {{ c.basisdays }}d
+                  </template>
+                  <template v-else>{{ c.replacements || 0 }}</template>
                 </td>
               </tr>
             </tbody>
@@ -424,6 +433,10 @@ onMounted(load)
 .days.empty { color: var(--danger); }
 .days.soon { color: var(--warning); }
 .small { font-size: 0.8rem; }
+
+/* Marks a rate the intervals do not agree on. Deliberately quiet - it qualifies
+   the number beside it rather than competing with the urgency bands. */
+.unstable { margin-left: 2px; font-weight: 600; cursor: help; }
 .empty-state { padding: 2rem; text-align: center; }
 .footnote { margin-top: 1rem; font-size: 0.85rem; }
 </style>

plugins/printers/services/supply_history.py

@@ -20,6 +20,19 @@ from datetime import datetime, timezone
 # new cartridge rather than noise.
 REPLACEMENT_RISE = 10
 
+# A replacement must also LAND high. A fresh cartridge reads near full, so a
+# rise that stops mid-range is a gauge bouncing, not a swap. Without this the
+# count was any +10 between readings, and the two common noise shapes both
+# scored: a supply reading 0 or near-0 while it was out of the machine, then the
+# real level again (0 -> 60 counts as +60), and a coarse gauge ticking back up
+# after a reseat or a power cycle. That is how one cartridge claimed five
+# changes in ninety days.
+#
+# A site that fits PART-USED cartridges will under-count with this rule. That is
+# the right way round: a missed swap widens the run and slows the estimate,
+# while a phantom swap resets the run and throws the estimate away entirely.
+NEW_CARTRIDGE_LEVEL = 80
+
 # Below this many readings a slope is arithmetic, not evidence. Two points
 # through a coarse gauge can "prove" any rate at all.
 MIN_POINTS_FOR_ESTIMATE = 4
@@ -29,6 +42,12 @@ MIN_POINTS_FOR_ESTIMATE = 4
 # says "never", which is worse than saying nothing.
 MIN_DROP_FOR_ESTIMATE = 2
 
+# How far the fastest and slowest intervals may differ before a single rate
+# stops being a fair summary. A cartridge that ran at 10 percent/day for two
+# days and 0.2 percent/day since is not described by any one number, and a
+# precise-looking figure invites more trust than it has earned.
+RATE_SPREAD_FACTOR = 5
+
 # At or below this, the cartridge is done and the arithmetic stops being the
 # useful answer. A supply sitting at 1% that drains a tenth of a point a day
 # computes to ten days; a printer at 1% is out of toner as far as anyone
@@ -58,18 +77,65 @@ def normalise(points):
             continue
         out.append((datetime.fromtimestamp(seconds, tz=timezone.utc), level))
     out.sort(key=lambda p: p[0])
+    return drop_spikes(out)
+
+
+def drop_spikes(points, rise=REPLACEMENT_RISE):
+    """Remove one-reading dips that RECOVER to where they came from.
+
+    A single reading far below both neighbours, then a recovery, is a big
+    upward step that scores as a cartridge change. That is how a cartridge
+    claimed five changes in ninety days.
+
+    NOT a Zabbix timeout: an item that does not answer records nothing, it does
+    not write a zero. The dip is a value the device really reported - a supply
+    pulled out to be shaken and reseated, a door open mid-poll, or a site whose
+    preprocessing maps the Printer MIB's "unknown" sentinels (-1/-2/-3, which
+    cannot land in the unsigned history table) onto 0.
+
+    The filter keys on SHAPE rather than cause, which is why it holds for all of
+    them: a level that comes back to where it was did not get a new cartridge.
+
+    Only a dip that comes BACK to roughly its previous level is removed. A
+    genuine near-empty reading before a swap (30, 5, 100) does not recover - it
+    jumps to full - so it is kept, and the swap after it still counts.
+    """
+    if len(points) < 3:
+        return points
+    out = [points[0]]
+    for index in range(1, len(points) - 1):
+        previous = points[index - 1][1]
+        current = points[index][1]
+        following = points[index + 1][1]
+        dipped = (previous - current) >= rise and (following - current) >= rise
+        recovered = abs(following - previous) <= rise
+        if dipped and recovered:
+            continue
+        out.append(points[index])
+    out.append(points[-1])
     return out
 
 
-def find_replacements(points, rise=REPLACEMENT_RISE):
+def _is_replacement(previous, current, rise=REPLACEMENT_RISE,
+                    newlevel=NEW_CARTRIDGE_LEVEL):
+    """One definition of "a cartridge was changed", used by every caller.
+
+    The rise must be big enough to clear gauge noise AND land near full, which
+    is what a new cartridge reads. Both conditions, because either alone admits
+    a shape that is not a swap.
+    """
+    return (current - previous) >= rise and current >= newlevel
+
+
+def find_replacements(points, rise=REPLACEMENT_RISE,
+                      newlevel=NEW_CARTRIDGE_LEVEL):
     """Timestamps where the level jumped up - one per cartridge change.
 
-    Returns [] for a series that only falls. A rise smaller than `rise` is
-    treated as noise, not a replacement.
+    Returns [] for a series that only falls.
     """
     replacements = []
     for (_, previous), (when, current) in zip(points, points[1:]):
-        if current - previous >= rise:
+        if _is_replacement(previous, current, rise, newlevel):
             replacements.append(when)
     return replacements
 
@@ -84,28 +150,78 @@ def current_run(points, rise=REPLACEMENT_RISE):
         return []
     start = 0
     for index in range(1, len(points)):
-        if points[index][1] - points[index - 1][1] >= rise:
+        if _is_replacement(points[index - 1][1], points[index][1], rise):
             start = index
     return points[start:]
 
 
+def interval_rates(points):
+    """Percent-per-day for each consecutive pair, falling intervals only.
+
+    A rise inside a run is gauge noise (a swap would have ended the run), and
+    a flat interval is real information - a cartridge that did not move - so it
+    stays in at zero.
+    """
+    rates = []
+    for (whenprev, prev), (when, current) in zip(points, points[1:]):
+        days = (when - whenprev).total_seconds() / 86400
+        if days <= 0:
+            continue
+        drop = prev - current
+        if drop < 0:
+            continue
+        rates.append(drop / days)
+    return rates
+
+
+def _median(values):
+    ordered = sorted(values)
+    count = len(ordered)
+    if not count:
+        return None
+    middle = count // 2
+    if count % 2:
+        return ordered[middle]
+    return (ordered[middle - 1] + ordered[middle]) / 2
+
+
 def burn_rate(points):
     """Percent consumed per day over these readings, or None.
 
+    THE MEDIAN OF THE PER-INTERVAL RATES, not the slope between the first and
+    last reading. Two endpoints cannot tell "steady" from "burst then stopped":
+    a cartridge that lost 20 percent in two days and then barely moved for a
+    month reads as 0.83 percent/day forever after, so the report keeps promising
+    it will run out long after printing slowed. The burst is one interval among
+    many to a median, and one of two points to a secant.
+
     None means "no honest estimate": too few readings, no elapsed time, or a
     drop too small to distinguish from a gauge that has not moved yet.
     """
     if len(points) < MIN_POINTS_FOR_ESTIMATE:
         return None
-    first_when, first_level = points[0]
-    last_when, last_level = points[-1]
-    days = (last_when - first_when).total_seconds() / 86400
-    if days <= 0:
+    total_days = (points[-1][0] - points[0][0]).total_seconds() / 86400
+    if total_days <= 0:
         return None
-    drop = first_level - last_level
-    if drop < MIN_DROP_FOR_ESTIMATE:
+    # The overall drop still gates the estimate: a gauge sitting on one plateau
+    # has not proved anything yet, whatever the intervals say.
+    if points[0][1] - points[-1][1] < MIN_DROP_FOR_ESTIMATE:
         return None
-    return drop / days
+    rate = _median(interval_rates(points))
+    if not rate:
+        # Every interval flat or rising, yet the run dropped overall - the
+        # movement is all in intervals the median discarded. Fall back to the
+        # whole-run slope rather than reporting nothing.
+        return (points[0][1] - points[-1][1]) / total_days
+    return rate
+
+
+def rate_is_unstable(points, factor=RATE_SPREAD_FACTOR):
+    """True when the intervals disagree enough that one number oversells it."""
+    rates = [r for r in interval_rates(points) if r > 0]
+    if len(rates) < 2:
+        return False
+    return max(rates) >= min(rates) * factor
 
 
 def analyse(points, rise=REPLACEMENT_RISE, currentlevel=None):
@@ -131,6 +247,7 @@ def analyse(points, rise=REPLACEMENT_RISE, currentlevel=None):
     points = normalise(points)
     result = {
         'currentlevel': currentlevel, 'daysleft': None, 'burnrateperday': None,
+        'rateunstable': False,
         'reason': None, 'replacements': 0, 'lastreplaced': None,
         'basisdays': 0, 'points': [],
     }
@@ -182,6 +299,9 @@ def analyse(points, rise=REPLACEMENT_RISE, currentlevel=None):
 
     result['burnrateperday'] = round(rate, 2)
     result['daysleft'] = max(0, int(level / rate))
+    # Say so when the intervals disagree wildly. The number is still the best
+    # estimate available; the flag stops it reading as a measurement.
+    result['rateunstable'] = rate_is_unstable(run)
     return result
 
 

tests/test_plugins/test_supply_history.py

@@ -267,3 +267,75 @@ def test_forecast_endpoint_says_so_when_zabbix_is_absent(client, db):
     assert 'not configured' in body['reason']
     assert body['cartridges'] == []
     assert body['orderlist'] == []
+
+
+# --------------------------------------------------------------------------
+# Noise that used to read as cartridge changes, and bursts that used to bias
+# the rate for the life of the cartridge. Both were reported from the floor:
+# "5 changes in 90 days, that's hard to believe", and a cartridge that dropped
+# 20 percent in two days then barely moved.
+# --------------------------------------------------------------------------
+
+def test_a_poll_returning_zero_is_not_a_cartridge_change():
+    """0 then a real level is an SNMP error or a calibrating printer.
+
+    The rise is +60, which cleared the old threshold on its own. It does not
+    land near full, so it is not a swap.
+    """
+    points = normalise(series([70, 60, 0, 60, 55, 50]))
+    assert find_replacements(points) == []
+
+
+def test_a_gauge_ticking_back_up_mid_range_is_not_a_change():
+    """A coarse gauge after a reseat or a power cycle. Lands at 55, not full."""
+    points = normalise(series([70, 60, 45, 55, 50, 45]))
+    assert find_replacements(points) == []
+
+
+def test_a_real_swap_is_still_counted():
+    """Near-empty to near-full. The shape a cartridge change actually makes."""
+    points = normalise(series([30, 15, 5, 100, 95, 90]))
+    assert len(find_replacements(points)) == 1
+
+
+def test_the_run_starts_at_the_real_swap_not_at_the_noise():
+    """current_run and find_replacements must agree on what a change is.
+
+    They read the same predicate now; when they did not, a phantom rise reset
+    the run and threw away the history the estimate needed.
+    """
+    points = normalise(series([90, 0, 85, 80, 75, 70]))
+    assert find_replacements(points) == []
+    assert len(current_run(points)) == len(points)
+
+
+def test_an_early_burst_does_not_dominate_the_rate_forever():
+    """20 percent in two days, then a month of almost nothing.
+
+    The endpoint slope reads the burst forever: (100-75)/30 = 0.83 %/day, so
+    the report keeps promising the cartridge runs out long after printing
+    stopped. The median sees one fast interval among many quiet ones.
+    """
+    levels = [100, 90, 80] + [80 - i * 0.2 for i in range(1, 28)]
+    rate = burn_rate(normalise(series(levels)))
+    assert rate is not None
+    assert rate < 1.0, rate
+    detail = analyse(series(levels))
+    assert detail['rateunstable'] is True
+
+
+def test_a_steady_cartridge_is_not_flagged_unstable():
+    detail = analyse(series([100, 95, 90, 85, 80, 75, 70]))
+    assert detail['burnrateperday'] == 5.0
+    assert detail['rateunstable'] is False
+
+
+def test_a_real_near_empty_reading_before_a_swap_is_kept():
+    """30, 5, 100 is a cartridge run to the end and changed - not a spike.
+
+    The dip filter must not eat it: the 5 does not RECOVER to 30, it jumps to
+    full, which is the shape of a swap rather than of a bad poll.
+    """
+    points = normalise(series([40, 30, 5, 100, 95, 90]))
+    assert 5.0 in [level for _, level in points]
+    assert len(find_replacements(points)) == 1