Spread a driver rollout across waves so the fleet cannot stampede its own share
Some checks failed
CI / backend (push) Failing after 7m16s
CI / naming (push) Failing after 7m10s
CI / frontend (push) Failing after 7m10s
CI / migrations-mysql (push) Failing after 7m14s

GE-Enforce gives each PC a start offset of SHA256(hostname) % 5 MINUTES and then
repeats every five minutes. That was sized for reading a few KB of manifest
JSON. A driver set is 100 MB for the two universals and 226 MB for a full site,
so the day a driver entry lands, every bay pulls it inside one five-minute
window: roughly 30 GB across 300 bays, at something like 800 Mbps, on the same
share the whole floor needs for everything else. The failure mode is not slow
printers, it is a floor that stops converging.

-WaveStart with -Waves spreads that out. Each PC derives its own wave from its
hostname, so there is no central coordination, no per-bay configuration, and no
list of who has had it yet. The hash is the same idiom Register-GEEnforce.ps1
already uses for its start offset, SHA-256 rather than MD5 because FIPS-enforced
bays disable MD5 outright and would throw.

Measured over 300 hostnames at 10 waves: 23 to 44 bays per wave against a mean of
30, so the peak wave moves about 4.3 GB rather than the 3 GB an average implies.
Hash bucketing is uneven and the peak is what sizes a link, so do not quote the
mean.

THE GATE RUNS BEFORE THE MANIFEST IS READ, because the manifest is on the share
too. A bay that is not due must not touch the share at all - one read is cheap,
300 bays deciding to read in the same five minutes is the entire problem.

It FAILS CLOSED on an unparseable date. Failing open would restore exactly the
stampede this exists to prevent, and 30 GB cannot be un-sent, whereas a typo that
installs nothing says so in the log every cycle and is fixed in a minute.

A bay powered off during its wave installs on its next cycle instead. The wave is
an earliest-time, not a deadline, so nothing needs chasing afterwards.

-TestOnly reports a bay whose wave has not opened as COMPLIANT, because not
installed is genuinely its desired state today; DSC would otherwise call
SetScript every pass to be told to wait. -IgnoreWave is for proving a pilot bay
before opening anything.

Verified on Windows 11 build 26200, six paths: not-due installs nothing and exits
0; TestOnly while not due exits 0; a garbage date exits 1 having installed
nothing; -IgnoreWave installs against a future start; an opened wave installs;
and no wave arguments at all installs, which is what imaging needs.
This commit is contained in:
cproudlock
2026-08-19 17:16:58 -04:00
parent 2d09fa3201
commit 3f7cc37b00

View File

@@ -22,6 +22,28 @@
# SHARE PATHS: on a GE-Enforce site the packages usually live on the SFLD share,
# which is mounted ONLY during the enforcement cycle. Run this as a manifest
# entry inside that cycle, not as its own scheduled task.
#
# WAVES, because a driver package is not a manifest check. GE-Enforce gives each
# PC a start offset of SHA256(hostname) % 5 MINUTES and then repeats every 5
# minutes, which was sized for reading a few KB of JSON. A driver set is 100 MB
# for the two universals and 226 MB for a full site, so the day an entry lands
# the whole fleet pulls it inside one 5-minute window: roughly 30 GB across 300
# bays, on the same share every bay needs for everything else. The failure is not
# slow drivers, it is a floor that stops converging.
#
# -WaveStart with -Waves spreads that out. Each PC derives its own wave from its
# hostname and does nothing until its turn, so there is no central coordination,
# no per-bay configuration and nothing to reconcile afterwards. The hash is the
# same idiom Register-GEEnforce.ps1 uses for its offset, SHA-256 rather than MD5
# because FIPS-enforced bays disable MD5 outright.
#
# A bay switched off during its wave installs on its next cycle instead. Late is
# a non-event here; the wave sets the earliest moment, not a deadline.
#
# Unparseable wave arguments FAIL CLOSED - nothing installs, and it says so every
# cycle. Failing open would restore precisely the stampede this exists to avoid,
# and 30 GB cannot be un-sent, whereas a typo that installs nothing is loud in
# the log and fixed in a minute.
param(
# Defaults to drivers.json beside this script.
@@ -32,8 +54,26 @@ param(
[string[]]$Only = @(),
# Report what is missing and change nothing. This is what a DSC TestScript
# calls: exit 0 means compliant.
[switch]$TestOnly
# calls: exit 0 means compliant. A bay whose wave has not arrived is
# compliant BY DESIGN - not installed is its desired state today.
[switch]$TestOnly,
# Date the rollout opens, e.g. '2026-08-25' or '2026-08-25 22:00'. Empty
# means no gating at all, which is right for imaging time: a bay being built
# is one bay, and it should come off the line complete.
[string]$WaveStart = '',
# How many waves to spread the fleet across. 0 or 1 means everyone at once.
# 300 bays over 10 daily waves is ~30 bays and ~3 GB a day.
[int]$Waves = 0,
# Wave spacing. Hours finishes a 10-wave rollout inside a day; Days is the
# cautious setting when nobody is watching the share.
[ValidateSet('Days', 'Hours')]
[string]$WaveUnit = 'Days',
# Pilot bays skip the gate. Use this to prove one bay before the fleet.
[switch]$IgnoreWave
)
$ErrorActionPreference = 'Continue'
@@ -51,6 +91,42 @@ function Log([string]$msg) {
"$ts [set] $msg" | Tee-Object -FilePath $logFile -Append | Out-Null
}
# The gate runs BEFORE the manifest is read, because the manifest lives on the
# share too. A bay that is not due this wave must not touch the share at all -
# reading it is cheap, but 300 bays deciding to read it in the same 5 minutes is
# how this whole problem starts.
if ($WaveStart -and -not $IgnoreWave) {
$start = [datetime]::MinValue
if (-not [datetime]::TryParse($WaveStart, [ref]$start)) {
Log "ERROR -WaveStart '$WaveStart' is not a date. Refusing to install."
Log ' (failing closed on purpose: guessing here would release the whole fleet at once)'
exit 1
}
$waveCount = $Waves
if ($waveCount -lt 1) { $waveCount = 1 }
# Same hostname hash as the enforcer's own start offset, so a bay's wave is
# stable for the life of its name: it cannot drift between cycles, and a
# rerun never moves a PC into a different wave.
$hostHash = [System.BitConverter]::ToUInt32(
[System.Security.Cryptography.SHA256]::Create().ComputeHash(
[System.Text.Encoding]::UTF8.GetBytes([System.Environment]::MachineName)), 0)
$wave = [int]($hostHash % [uint32]$waveCount)
if ($WaveUnit -eq 'Hours') { $due = $start.AddHours($wave) }
else { $due = $start.AddDays($wave) }
if ((Get-Date) -lt $due) {
Log ("wave {0} of {1} for {2}; not due until {3}. Nothing to do." -f `
$wave, $waveCount, [System.Environment]::MachineName,
$due.ToString('yyyy-MM-dd HH:mm'))
exit 0
}
Log ("wave {0} of {1} opened {2}; proceeding" -f `
$wave, $waveCount, $due.ToString('yyyy-MM-dd HH:mm'))
}
if (-not (Test-Path $Manifest)) {
Log "ERROR manifest not found: $Manifest"
exit 1