Files

cproudlock 902ce5780a Capture-MclCache: read files MCL still has open, skip any it cannot

2026-10-18 09:21:28 -04:00

15 KiB

Raw Blame History

PESetup.exe internals

What GE Image Setup actually does, start to finish. Written from the decompiled assembly, not from observation, because several long-standing beliefs about this tool turned out to be wrong and cost weeks of debugging.

Version documented: 4.0.0.17 (Sources/PESetup.exe, PE32+ native apphost wrapping a .NET 6 single-file bundle, 468 embedded files).

Version in production: 4.0.0.20, as of 2026-08-06 — the media reports it in its own log (AppVersion: 4.0.0.20). Everything in this document was verified against a 4.0.0.20 run on bay 579C144 that day: the media drive resolved to Z:\, W: was created by PrepareDisk and used for every copy destination, the fallback unattend at Deploy\FlatUnattendW10.xml was the one loaded, and driver selection matched win11_optiplexd13mlk7020_a09.zip by model. Most of the decompiled detail below has not been re-derived from the 4.0.0.20 binary, so treat exact line-level claims as 4.0.0.17 and the observed behaviour as current. The exception is "The Patch Tuesday freshness gate" under GATHER DATA, which was decompiled from 4.0.0.20 on 2026-08-21 and does NOT exist in 4.0.0.17.

How to re-derive this

The managed code is not directly readable - the outer PE has no managed metadata, so ILSpy refuses it. Extract PESetup.dll from the bundle first:

# .NET single-file bundle: signature is SHA-256 of ".net core bundle";
# the int64 EIGHT BYTES BEFORE it is the bundle header offset.
SIG = bytes([0x8b,0x12,0x02,0xb9,0x6a,0x61,0x20,0x38,
             0x72,0x7b,0x93,0x02,0x14,0xd7,0xa0,0x32])
i = data.find(SIG)
header = struct.unpack_from('<q', data, i - 8)[0]
# header: uint32 major, uint32 minor, int32 count, 7-bit-prefixed bundle id,
#         then (v2+) 4 x int64 deps/runtimeconfig, then uint64 flags.
# each entry: int64 offset, int64 size, (v6+) int64 compressedSize,
#             byte type, 7-bit-prefixed path.  Compressed entries are raw
#             deflate (zlib.decompress(raw, -15)).

Then ilspycmd -t <TypeName> PESetup.dll.

Paths it hardcodes

From PESetup.Models.GlobalSettings. All are relative to the MEDIA drive (Z:\, whichever winpeapps share startnet mapped) unless stated.

ControlDir            \Deploy\Control
ToolsDir              \Deploy\Tools\GE
SoftwarePackagesDir   \Deploy\Applications
HWPackagesDir         \Deploy\HW_Apps
UnattendFile          \Deploy\Tools\GE\XML\FlatUnattendW10.xml
UnattendFile2         \Deploy\FlatUnattendW10.xml        (fallback)
DiskPartDir           <exe dir>\DiskPart
ConfigSkipPackagesFile      skip.json
ConfigDisableAutoStartFile  disableauto.json
ExpirationDuration    30 days
AutoStartCountDown    31
MinRequiredSpaceWithoutCompression  128849018880  (120 GB)

W: IS HARDCODED, EVERYWHERE. Not derived, not configurable, not a convention this project invented:

CopyPackages    destDir = "W:\\" + SoftwarePackagesDir
CopyHWPackages  destDir = "W:\\" + HWPackagesDir
CopyTools       destDir = "W:\\GE"
CopyDrivers     destinationPath = "W:\\Drivers\\" + <zip name without extension>
ApplyImage      destination = "W:\\"
CreateBCD       "W:\\Windows\\System32\\bcdboot.exe", "W:\\Windows /l en-US"
ApplyPackages   DismApi.OpenOfflineSession("W:\\")
CopyLogs        tags -> "w:\\windows\\system32\\"

PESetup's own disk preparation is what makes W: exist. Any downstream script that hunts for "the applied volume" is solving a problem this tool does not have. diskpart list volume on a machine mid-image shows Volume 0 W Windows NTFS 237GB Healthy.

The five working steps

Keys as they appear in the log: workingstep_gatherdata, workingstep_selectos, workingstep_prepare, workingstep_copy, workingstep_apply, workingstep_finalize, workingstep_reboot.

1. GATHER DATA

Four operations, all automatic:

Operation	What it establishes
`GatherDataMediaDrive`	`ImageInfo.MediaDrive`, e.g. `Z:\`
`GatherDataBootDrive`	`ImageInfo.BootDrive`, always `X:\` (WinPE RAM disk)
`GatherDataSelection`	BIOS version/type, serial, secure boot, OS, languages, model, driver
`GatherDataImageDisk`	picks the physical disk and logs its partitions

GatherDataSelection is where most decisions are made:

Secure boot is mandatory. SecurebootEnabled != 1 fails the step outright.
Reads Control\LanguagePacks.json.
SystemHelpers.GetModel() and GetManufacturer() off WMI.
Reads Control\HardwareDriver.json and Control\hw_applications.json. NOTE: it reads HardwareDriver.json, not the hw_drivers.json that also sits in that folder. Editing the wrong one changes nothing.
Virtual platform check. If ANY driver entry has a manufacturer containing "virtual platform", the tool goes into virtual-only mode: it takes list[0] as the driver and then REQUIRES the machine to look virtual (model/serial/BIOS containing VIRTUAL, VMWARE, XEN, QEMU, VirtualBox, vmw). On real hardware that is a hard failure. A stray "virtual platform" entry in the catalogue therefore breaks imaging for every physical machine.
Otherwise GetDriverByModel (below).
If Control\skip.json exists, package installation is skipped.

The Patch Tuesday freshness gate (4.0.0.20, NOT in 4.0.0.17)

Added after this document was first written, and it is the last thing GatherDataSelection does before it succeeds. Verified against the 4.0.0.20 binary on 2026-08-21.

DateTime lastPatchTuesday = MiscHelpers.GetLastPatchTuesday();
DateTime dateTime = lastPatchTuesday.AddDays(10.0);
if (!(DateTime.Now > lastPatchTuesday) || !(DateTime.Now < dateTime))
{
    bool flag = true;
    if (!string.IsNullOrEmpty(osv.operatingSystemVersion.wim.creationDate)) {
        DateTime d2 = DateTime.ParseExact(..., "yyyy-MM-dd", ...);
        if (d2 > lastPatchTuesday) flag = false;
    }
    if (imageinfo.LastUpdate >= dateTime) flag = false;
    if (flag) {
        SetResult(Failed, "gatherdata_mediadrive_error_patchday");
        imageinfo.IsUpdateRequired = true;
    }
}

GetLastPatchTuesday returns the second Tuesday of this month, or of the previous month when today is on or before it. That makes DateTime.Now > lastPatchTuesday always true, so only the DateTime.Now >= lastPatchTuesday + 10 days half of the guard ever fires. In effect: from Patch Tuesday + 10 onward, the media must prove it is at least that fresh, or imaging fails.

Two things can satisfy it. Neither is a package check; both are dates.

Control\OperatingSystem.json -> wim.creationDate, format yyyy-MM-dd, later than the last Patch Tuesday. Our OperatingSystem.json has no creationDate field at all, so this branch never runs here.
imageinfo.LastUpdate, which is nothing more than the mtime of <media>\Deploy\Control\Media.tag (set in GatherDataMediaDrive. CheckLastWriteTime, the same read that drives the 30-day expiry).

So on this server the gate reduces to one question: is Media.tag's mtime on or after the last Patch Tuesday + 10 days?

The midnight trap. /etc/cron.d/media-tag-refresh touches Media.tag at 00:00:01 server local time. WinPE renders SMB file times about an hour behind the server, which drops a 00:00 stamp back onto the previous day. That is invisible for 29 days a month and fatal on exactly one: the Patch Tuesday + 10 boundary day, when the gate compares against midnight of that very date. Hit on 2026-08-21 (PESetup log showed Last Imaging Source Update: '2026-08-20' against Last Patch Tuesday + 10 days: '2026-08-21'), on a Dell Pro Tower Plus QBT1250 that was otherwise fine: driver matched, secure boot on, media view intact. It recurs every month on Patch Tuesday + 10.

Restamp Media.tag at a mid-day hour, never at midnight. It must also stay in the PAST relative to the bay clock: a future mtime makes CheckLastWriteTime set isBiosTimeCorrect = false and fail with the BIOS-time error instead.

The failure text itself is useless. gatherdata_mediadrive_error_patchday renders as an empty ####ERROR: line in the log, so the three INFO lines above it are the only diagnosis:

Last Imaging Source Update: '<Media.tag mtime>'
Last Patch Tuesday: '<second Tuesday>'
Last Patch Tuesday + 10 days: '<deadline>'
Imaging Source Update required: True

2. SELECT OS

Operator-facing. Warns when the media holds an LTSC image ("99% of the time only for ShopFloor"). Also shows days-to-expiry - the media expires 30 days after build (ExpirationDuration).

3. PREPARE

Operation	What it does
`PrepareDoDisking`	copies `<exedir>\DiskPart\diskpartEFI.txt` to `X:\diskpartEFI<n>.txt`, then `cmd /c DISKPART /S X:\diskpartEFI<n>.txt`
`PrepareUnattend`	loads the unattend, substitutes, saves to `X:\Unattend.xml`

PrepareUnattend in detail:

text = MediaDrive + UnattendFile;              // \Deploy\Tools\GE\XML\FlatUnattendW10.xml
if (!File.Exists(text)) text = MediaDrive + UnattendFile2;   // \Deploy\FlatUnattendW10.xml
xml.Load(text);
xml.InnerXml = xml.InnerXml.Replace("%serialnumber%", imageinfo.Serialnumber);
xml.InnerXml = xml.InnerXml.Replace("*arch*", arch);
SetPackages(arch, xml, imageinfo);             // needs Control\Packages.xml
xml.Save("X:\\Unattend.xml");

Two tokens are substituted: %serialnumber% and *arch*. SetPackages merges in servicing packages from Control\Packages.xml and Control\PackageGroups.xml.

The unattend that is USED is the one on the media at Tools\GE\XML\ if present, otherwise Deploy\FlatUnattendW10.xml. A shopfloor variant only takes effect if it occupies one of those two paths.

4. COPY

Four operations. None of them filters, and none of them reads the unattend.

Operation	Source	Destination	Rule
`CopyTools`	`Z:\Deploy\Tools\GE`	`W:\GE`	whole directory, recursive
`CopyPackages`	`Z:\Deploy\Applications`	`W:\Deploy\Applications`	whole directory, recursive
`CopyHWPackages`	`Z:\Deploy\HW_Apps`	`W:\Deploy\HW_Apps`	whole directory; skipped if `IsVirtual` or `HWApps == null`
`CopyDrivers`	one `.zip` chosen by model	`W:\Drivers\<zipname>`	unzipped, not copied

ExecuteInternalCopy walks GetDirectoriesRecursive(source, includeSubDirs: true) then Directory.GetFiles(item, "*"). Every file, every subdirectory. It clears the read-only attribute on each copy.

Consequences worth designing around:

Anything dropped into Deploy\Applications on the media lands on the target. No manifest, no registration, no unattend reference required.
The copy is FAIL-FAST. One exception on one file returns false and fails the whole step with copy_packages_error. A locked or unreadable file in Applications fails imaging, it does not get skipped.
Progress is computed from total directory size up front, so bulky additions visibly lengthen this phase.

5. APPLY

Operation	What it does
`ApplyImage`	`install.wim` from `OperatingSystem.json`'s `destinationDir`, at `wimindex`, applied to `W:\`
`ApplyPackages`	DISM offline session on `W:\`, adds servicing packages; logs to `X:\ApplyOSPatch_<n>.log`
`ApplyLanguagePacks`	same pattern, `X:\ApplyLanguagePack_<n>.log`
`ApplyUnattend`	DISM offline session on `W:\`, applies `X:\Unattend.xml`; logs to `X:\ApplyUnattend.log`
`CreateBCD`	`W:\Windows\System32\bcdboot.exe W:\Windows /l en-US`
`CopyWinRE`	`reagentc /setreimage /path T:\Recovery\WindowsRE /target W:\Windows`

All DISM work uses W:\imagetemp as scratch and deletes it afterwards.

6. FINALIZE

CopyLogs:

CopyTagFiles(ToolsDir) - every *.tag from the tools dir to w:\windows\system32\. This is how build/media tags reach the OS.
CopyLogFiles("X:\\") and CopyXMLFile("X:\\") - the PESetup log and the generated unattend are preserved.

Driver selection, in full

This is the part most worth understanding, because a miss is nearly silent.

// GatherDataSelection
List<HardwareDriversRootObject> list =
    JSONHelpers.ReadJSON_Driver(MediaDrive + ControlDir + "\\HardwareDriver.json");
...
imageinfo.HWDriver = GetDriverByModel(list, imageinfo.Model);
if (imageinfo.HWDriver == null) {
    // status = Warning, message "driver for [MODEL] not found"
    return operationResult;
}

private static HardwareDriversRootObject? GetDriverByModel(List<...> drivers, string model)
{
    string modelFamily = "";
    if (model.ToUpper().Contains("LATITUDE"))  modelFamily = "Latitude";
    if (model.ToUpper().Contains("OPTIPLEX"))  modelFamily = "Optiplex";
    if (model.ToUpper().Contains("PRECISION")) modelFamily = "Precision";

    return drivers.Where(d => {
        if (!string.IsNullOrEmpty(modelFamily) &&
            !d.family.ToLower().Contains(modelFamily.ToLower())) return false;
        foreach (string token in d.modelswminame.Split(','))
            if (model.ToLower().Contains(token.ToLower())) return true;   // SUBSTRING
        return false;
    }).ToList()?.FirstOrDefault();
}

Then:

sourceFilePath = MediaDrive + HWDriver.destinationDir.Replace("*destinationdir*","")
                 + "\\" + HWDriver.fileName;
destinationPath = "W:\\Drivers\\" + Path.GetFileNameWithoutExtension(sourceFilePath);
StartUnzipAsync(...)

Four traps in that logic

A miss is a WARNING, not a failure. Imaging continues and the machine comes up with no drivers - no NIC, no WiFi, so DNS fails and anything network-dependent at first boot fails with it. Symptoms appear far from the cause.
Matching is substring, first match wins. A token like 7020 matches any model string containing 7020. Order in the JSON decides ties.
The family filter knows only three Dell lines. Anything else - MicroPCs, NUCs, non-Dell - skips the filter and depends entirely on modelswminame substrings being right.
One "virtual platform" entry hijacks the whole catalogue (see GATHER DATA step 5) and fails every physical machine.

What this means for startnet.cmd

Three beliefs encoded in startnet.cmd do not survive contact with the source:

"The applied volume might not be W:." PESetup hardcodes W: in nine places and creates it during disk prep. The volume finder, the diskpart reassignment and the W: wait loop are machinery around a non-problem.
"We must copy our payload from the enrollment share after imaging." Anything in Deploy\Applications on the media is copied to W:\Deploy\Applications by PESetup itself, and is readable at C:\Deploy\Applications at first boot.
"Display MicroPCs failed because the applied volume was not W:." Worth re-testing. A missing HardwareDriver.json match produces the same end-state - no drivers, no network - via a completely different route, and that route only logs a warning.

Files PESetup reads from the media

Deploy\Control\HardwareDriver.json     driver catalogue        (NOT hw_drivers.json)
Deploy\Control\hw_applications.json    per-model applications
Deploy\Control\OperatingSystem.json    OS list, wim path, wimindex
Deploy\Control\LanguagePacks.json      language packs
Deploy\Control\packages.json           servicing packages
Deploy\Control\Packages.xml            unattend package merge
Deploy\Control\PackageGroups.xml       package grouping
Deploy\Control\Media.tag               media identity
Deploy\Control\skip.json               presence = skip package install
Deploy\Control\disableauto.json        presence = disable autostart
Deploy\Tools\GE\XML\FlatUnattendW10.xml   preferred unattend
Deploy\FlatUnattendW10.xml                fallback unattend

15 KiB Raw Blame History