Generic build for other GE sites: no WJ data, one shopfloor type, Windows USB builder
Everything West Jefferson specific moved out to a separate repo (pxe-site-wj): NTLARS/UDC/heat-treat backups keyed by WJ machine numbers, CMM and Wax/Trace bay lists, WJ OpenText profiles, site-config.json, the WJ preinstall list, and the WJ shopfloor PC types (Collections, NoCollections, Common, Keyence, CMM, Genspect, HeatTreat, Wax/Trace, Display, PartMarker) with GE-Enforce, machine numbers and the desktop layout. The part-marker bench tools moved to their own repo. Dead scripts removed: build-proxmox-iso, deploy-bios, sync_hardware_models, mirror-from-gold, pull-bios, startnet-template.cmd. Shopfloor is one generic type, gea-shopfloor-standard, with today's timing: - PPKG at first logon (unattend Order 10, unchanged). - Install-AppList.ps1 runs C:\PreInstall\preinstall.json during setup. - A new SYSTEM task, GE Shopfloor Post-PPKG, runs postppkg.json once after the post-PPKG reboot. It also clears AutoLogonCount after the ShopFloor handover (the Users-level monitor can be denied that HKLM write) and removes the Intune Sync task when done. - Both lists ship empty. A site adds entries and installers. - WinPE choice 3 goes straight to the PPKG menu. Staging, fetch and heal carry only the generic payload. The webapp's shopfloor sub-menu editor is gone, and the imaging tiles name the new stages. - The playbook deploys the empty lists only if missing. It now deploys the PowerShell 7 MSI the shopfloor unattend installs, which it never did before. Blancco site settings move into the webapp (Updates > Blancco releases): business location, licensing Wi-Fi SSID and password. They are rendered into the served preferences.xml and applied to every staged release, and are seeded from site-vars (site_name, blancco_wifi_ssid, blancco_wifi_password) only while none exist. The repo XML carries no site values; the plaintext Wi-Fi password is gone from it. New site_name and site_timezone settings: report and evidence-pack site name, server clock and webapp time zone. No more West Jefferson defaults in webapp/config.py or the dashboard title. Sync-BPRT.ps1 gets the server address filled in at download instead of WJ's GE address. USB on Windows: - scripts/Build-PxeUsb.ps1 runs the same checks as build-usb.sh. It writes one FAT32 CIDATA partition (ISO files plus patched grub.cfg plus payload) with diskpart and reads every file back by SHA-256. - Tested with Windows PowerShell 5.1 in the win11 VM, and the layout installs and first-boots in the pxetest VM. - scripts/stage-usb-share.sh stages the build folder on a share. - site-content/ is optional for both builders. KB docs/kb/How_To_Deploy_PXE_Server_New_Site.html: new-site deployment, Windows only, with a site-vars.yml guide. Checks: webapp tests pass (5 suites); the unattend lint and playbook syntax checks pass; every .ps1 parses; there are no WJ hosts, IPs, names or PC types in playbook/, webapp/, scripts/ or autoinstall/.
This commit is contained in:
cproudlock
595
scripts/Build-PxeUsb.ps1
Normal file
595
scripts/Build-PxeUsb.ps1
Normal file
@@ -0,0 +1,595 @@
|
||||
#Requires -RunAsAdministrator
|
||||
#
|
||||
# Build-PxeUsb.ps1 - Build the PXE server installer USB on a Windows PC.
|
||||
#
|
||||
# The Windows twin of scripts/build-usb.sh: the same checks and the same
|
||||
# payload, from a checkout of this repo plus its payload folders (for
|
||||
# example \\<build PC>\pxe-images\pxe-usb, which scripts/stage-usb-share.sh
|
||||
# fills). No Linux tools are needed.
|
||||
#
|
||||
# Layout (UEFI boot only, Secure Boot works): ONE FAT32 partition labelled
|
||||
# CIDATA holding
|
||||
# - every file of the Ubuntu live-server ISO (boot/grub/grub.cfg gets
|
||||
# 'autoinstall' on the first kernel line and a 5 s timeout), so the
|
||||
# firmware boots its shim and grub straight from the stick, and
|
||||
# - the CIDATA payload: user-data / meta-data (cloud-init NoCloud finds
|
||||
# them by the CIDATA label), packages/, playbook/, webapp/, pip-wheels/,
|
||||
# boot files, boot-tools/, enrollment/, bios/, site-content/ ...
|
||||
# The installer and first-boot.sh find everything by that label, exactly as
|
||||
# on build-usb.sh's two-partition stick. Windows formats FAT32 only up to
|
||||
# 32 GB, so the partition is at most 32 GB; the rest of a bigger stick stays
|
||||
# empty. FAT32 holds files under 4 GiB: boot files, enrollment and drivers
|
||||
# over 3500 MiB are split into .part.NN files the playbook joins again, as
|
||||
# build-usb.sh does; site-content and WinPE images are refused instead.
|
||||
#
|
||||
# Checks before the stick is touched (refuses to build what the server
|
||||
# would reject):
|
||||
# - the ISO's volume id is the one offline-packages/ was resolved for
|
||||
# - the iPXE chain matches the ipxe_*_sha256 pins in the playbook
|
||||
# - every wheel in pip-wheels/ is the pinned version with a pinned hash
|
||||
# - site-content/ matches its SHA256SUMS list and has no file >= 4 GiB
|
||||
# - the site vars set pxe_nic_mac / corp_nic_mac to a MAC or leave them empty
|
||||
# - the disk is a USB disk, not the boot or system disk, and big enough
|
||||
# After copying, every file is read back from the stick and compared with
|
||||
# its source by SHA-256 (-QuickVerify: by size only).
|
||||
#
|
||||
# Usage: double-click Build-PxeUsb.cmd in the build folder (asks for admin
|
||||
# rights, lists the USB disks and asks which one; uses site-vars.yml from
|
||||
# the build folder if there is one, else the defaults). Or, elevated:
|
||||
# .\Build-PxeUsb.ps1 list USB disks
|
||||
# .\Build-PxeUsb.ps1 -DiskNumber 3 -NoSiteVars build on disk 3
|
||||
# .\Build-PxeUsb.ps1 -DiskNumber 3 -SiteVars .\site-vars.yml
|
||||
# Options: -IsoPath FILE, -PayloadRoot DIR, -WinPESource DIR,
|
||||
# -SkipEnrollment -SkipBios -SkipDrivers -SkipBootTools -SkipSiteContent,
|
||||
# -QuickVerify, -Yes (no typed confirmation).
|
||||
#
|
||||
# Exit codes: 0 built and verified, 1 refused or failed (nothing or a
|
||||
# half-written stick: rebuild), 2 usage.
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[int]$DiskNumber = -1,
|
||||
[string]$IsoPath = '',
|
||||
[string]$PayloadRoot = '',
|
||||
[string]$SiteVars = '',
|
||||
[switch]$NoSiteVars,
|
||||
[string]$WinPESource = '',
|
||||
[switch]$SkipEnrollment,
|
||||
[switch]$SkipBios,
|
||||
[switch]$SkipDrivers,
|
||||
[switch]$SkipBootTools,
|
||||
[switch]$SkipSiteContent,
|
||||
[switch]$QuickVerify,
|
||||
[switch]$Yes,
|
||||
# Build-PxeUsb.cmd: ask for the disk, take site-vars.yml from the build
|
||||
# folder (else the defaults), wait for a key at the end.
|
||||
[switch]$Interactive
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
Set-StrictMode -Version 2
|
||||
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
if (-not $PayloadRoot) { $PayloadRoot = $RepoRoot }
|
||||
$PayloadRoot = (Resolve-Path -LiteralPath $PayloadRoot).Path
|
||||
|
||||
$FatMax = [int64]4GB - 1 # largest file FAT32 can hold
|
||||
$ChunkSize = [int64]3500 * 1MB # split size (build-usb.sh FAT32_MAX)
|
||||
$PartMax = [int64]32GB - 64MB # Windows formats FAT32 up to 32 GB
|
||||
$Utf8NoBom = New-Object System.Text.UTF8Encoding($false)
|
||||
|
||||
function Say([string]$m) { Write-Host $m }
|
||||
function Fail([string]$m) {
|
||||
Write-Host "ERROR: $m" -ForegroundColor Red
|
||||
if ($Interactive) { Write-Host ''; Read-Host 'Press Enter to close' | Out-Null }
|
||||
exit 1
|
||||
}
|
||||
|
||||
function Show-UsbDisks {
|
||||
Say 'USB disks:'
|
||||
$d = @(Get-Disk | Where-Object { $_.BusType -eq 'USB' })
|
||||
if ($d.Count -eq 0) { Say ' (none found)'; return }
|
||||
foreach ($x in $d) {
|
||||
Say (' {0,3} {1,-34} {2,8:N1} GB {3}' -f $x.Number, $x.FriendlyName, ($x.Size / 1GB), $x.PartitionStyle)
|
||||
}
|
||||
Say ''
|
||||
Say 'Then: .\Build-PxeUsb.ps1 -DiskNumber N -NoSiteVars (or -SiteVars FILE)'
|
||||
}
|
||||
|
||||
function Get-Sha256([string]$path) {
|
||||
(Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash.ToLowerInvariant()
|
||||
}
|
||||
|
||||
function Get-StreamSha256([System.IO.Stream]$s) {
|
||||
$sha = [System.Security.Cryptography.SHA256]::Create()
|
||||
try { -join ($sha.ComputeHash($s) | ForEach-Object { $_.ToString('x2') }) }
|
||||
finally { $sha.Dispose() }
|
||||
}
|
||||
|
||||
# Every regular file under a folder (follows no links), relative paths with \.
|
||||
function Get-Tree([string]$root, [string[]]$skipDirs = @(), [string[]]$skipExt = @()) {
|
||||
$base = (Resolve-Path -LiteralPath $root).Path.TrimEnd('\')
|
||||
$out = New-Object System.Collections.Generic.List[object]
|
||||
$stack = New-Object System.Collections.Stack
|
||||
$stack.Push($base)
|
||||
while ($stack.Count -gt 0) {
|
||||
$dir = $stack.Pop()
|
||||
foreach ($e in (Get-ChildItem -LiteralPath $dir -Force)) {
|
||||
if ($e.Attributes -band [IO.FileAttributes]::ReparsePoint) { continue }
|
||||
if ($e.PSIsContainer) {
|
||||
if ($skipDirs -notcontains $e.Name) { $stack.Push($e.FullName) }
|
||||
continue
|
||||
}
|
||||
if ($skipExt -contains $e.Extension.ToLowerInvariant()) { continue }
|
||||
$out.Add([pscustomobject]@{ Full = $e.FullName; Rel = $e.FullName.Substring($base.Length + 1); Size = $e.Length })
|
||||
}
|
||||
}
|
||||
, $out
|
||||
}
|
||||
|
||||
# --- the copy plan -------------------------------------------------------------
|
||||
# Each entry: Src, Dest (relative to the stick root), Size, Split (may be split
|
||||
# into .part.NN), Data (bytes written instead of Src: the patched grub.cfg).
|
||||
$Plan = New-Object System.Collections.Generic.List[object]
|
||||
function Add-Plan([string]$src, [string]$dest, [int64]$size, [bool]$split = $false, [byte[]]$data = $null) {
|
||||
$Plan.Add([pscustomobject]@{ Src = $src; Dest = $dest.TrimStart('\'); Size = $size; Split = $split; Data = $data; Hashes = $null })
|
||||
}
|
||||
function Add-File([string]$src, [string]$destDir, [bool]$split = $false) {
|
||||
$f = Get-Item -LiteralPath $src
|
||||
$d = if ($destDir) { Join-Path $destDir $f.Name } else { $f.Name }
|
||||
Add-Plan $f.FullName $d $f.Length $split
|
||||
}
|
||||
function Add-Tree([string]$src, [string]$destDir, [bool]$split = $false, [string[]]$skipDirs = @(), [string[]]$skipExt = @(), [string[]]$skipRel = @()) {
|
||||
foreach ($t in (Get-Tree $src $skipDirs $skipExt)) {
|
||||
if ($skipRel -contains $t.Rel) { continue }
|
||||
Add-Plan $t.Full (Join-Path $destDir $t.Rel) $t.Size $split
|
||||
}
|
||||
}
|
||||
|
||||
function Pause-IfInteractive {
|
||||
if ($Interactive) { Write-Host ''; Read-Host 'Press Enter to close' | Out-Null }
|
||||
}
|
||||
trap {
|
||||
Write-Host "ERROR: $_" -ForegroundColor Red
|
||||
Write-Host (" at line {0}: {1}" -f $_.InvocationInfo.ScriptLineNumber, $_.InvocationInfo.Line.Trim())
|
||||
Pause-IfInteractive
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- no disk given: list and stop (or ask) -------------------------------------
|
||||
if ($DiskNumber -lt 0 -and $Interactive) {
|
||||
Show-UsbDisks
|
||||
$ans = Read-Host 'Disk number of the USB stick to build (Enter to cancel)'
|
||||
if ($ans -notmatch '^\d+$') { Say 'Cancelled.'; Pause-IfInteractive; exit 2 }
|
||||
$DiskNumber = [int]$ans
|
||||
}
|
||||
if ($Interactive -and -not $SiteVars -and -not $NoSiteVars) {
|
||||
$sv = Join-Path $RepoRoot 'site-vars.yml'
|
||||
if (Test-Path -LiteralPath $sv) { $SiteVars = $sv } else { $NoSiteVars = [switch]$true }
|
||||
}
|
||||
if ($DiskNumber -lt 0) { Show-UsbDisks; exit 2 }
|
||||
if ($SiteVars -and $NoSiteVars) { Say 'Give -SiteVars FILE or -NoSiteVars, not both.'; exit 2 }
|
||||
if (-not $SiteVars -and -not $NoSiteVars) {
|
||||
Say 'Give -SiteVars FILE (per-site values, see playbook\site-vars.example.yml)'
|
||||
Say 'or -NoSiteVars (the first boot then picks the imaging NIC itself and uses the defaults).'
|
||||
exit 2
|
||||
}
|
||||
|
||||
$Autoinstall = Join-Path $RepoRoot 'autoinstall'
|
||||
$PlaybookDir = Join-Path $RepoRoot 'playbook'
|
||||
$WebappDir = Join-Path $RepoRoot 'webapp'
|
||||
$OfflineDir = Join-Path $PayloadRoot 'offline-packages'
|
||||
$WheelsDir = Join-Path $PayloadRoot 'pip-wheels'
|
||||
$BootFiles = Join-Path $PayloadRoot 'boot-files'
|
||||
$BootTools = if ($SkipBootTools) { '' } else { Join-Path $PayloadRoot 'boot-tools' }
|
||||
$Enrollment = if ($SkipEnrollment) { '' } else { Join-Path $PayloadRoot 'enrollment' }
|
||||
$BiosDir = if ($SkipBios) { '' } else { Join-Path $PayloadRoot 'bios-staging' }
|
||||
$DriversDir = if ($SkipDrivers) { '' } else { Join-Path $PayloadRoot 'drivers-staging' }
|
||||
$SiteContent = if ($SkipSiteContent) { '' } else { Join-Path $PayloadRoot 'site-content' }
|
||||
|
||||
Say '============================================'
|
||||
Say 'PXE Server USB Builder (Windows)'
|
||||
Say '============================================'
|
||||
Say "Repo : $RepoRoot"
|
||||
Say "Payloads : $PayloadRoot"
|
||||
|
||||
# --- ISO ---------------------------------------------------------------------
|
||||
if (-not $IsoPath) {
|
||||
$iso = Get-ChildItem -LiteralPath $PayloadRoot -Filter 'ubuntu-*-live-server-amd64.iso' -File -ErrorAction SilentlyContinue |
|
||||
Sort-Object Name | Select-Object -Last 1
|
||||
if (-not $iso) { Fail "no -IsoPath and no ubuntu-*-live-server-amd64.iso in $PayloadRoot" }
|
||||
$IsoPath = $iso.FullName
|
||||
}
|
||||
if (-not (Test-Path -LiteralPath $IsoPath -PathType Leaf)) { Fail "ISO not found: $IsoPath" }
|
||||
$IsoPath = (Resolve-Path -LiteralPath $IsoPath).Path
|
||||
Say "ISO : $IsoPath"
|
||||
|
||||
# Volume id from the ISO9660 primary volume descriptor (sector 16, offset 40,
|
||||
# 32 bytes): the same string xorriso -pvd_info prints on Linux.
|
||||
$fs = [IO.File]::OpenRead($IsoPath)
|
||||
try {
|
||||
$buf = New-Object byte[] 2048
|
||||
[void]$fs.Seek(16 * 2048, 'Begin')
|
||||
if ($fs.Read($buf, 0, 2048) -ne 2048 -or $buf[0] -ne 1 -or [Text.Encoding]::ASCII.GetString($buf, 1, 5) -ne 'CD001') {
|
||||
Fail "$IsoPath has no ISO9660 primary volume descriptor"
|
||||
}
|
||||
$IsoVolId = [Text.Encoding]::ASCII.GetString($buf, 40, 32).TrimEnd()
|
||||
} finally { $fs.Dispose() }
|
||||
|
||||
# --- checks (as build-usb.sh) ------------------------------------------------
|
||||
Say ''
|
||||
Say '[1/6] Checking the payload...'
|
||||
foreach ($f in @((Join-Path $Autoinstall 'user-data'), (Join-Path $Autoinstall 'meta-data'),
|
||||
(Join-Path $PlaybookDir 'pxe_server_setup.yml'), (Join-Path $RepoRoot 'scripts\offline-apt-install.sh'),
|
||||
(Join-Path $OfflineDir 'Packages'), (Join-Path $OfflineDir 'explicit.list'))) {
|
||||
if (-not (Test-Path -LiteralPath $f -PathType Leaf)) { Fail "$f not found (offline-packages: built by scripts/download-packages.sh)" }
|
||||
}
|
||||
|
||||
$volFile = Join-Path $OfflineDir 'iso-volid'
|
||||
if (Test-Path -LiteralPath $volFile) {
|
||||
$want = (Get-Content -LiteralPath $volFile -TotalCount 1).Trim()
|
||||
if ($want -ne $IsoVolId) { Fail "offline-packages\ was built for ISO '$want', this ISO is '$IsoVolId'. Use that ISO or rebuild the set." }
|
||||
Say " offline-packages\ matches ISO $IsoVolId"
|
||||
} else {
|
||||
Say " WARNING: offline-packages\iso-volid missing; cannot check the set was resolved against $IsoVolId."
|
||||
}
|
||||
|
||||
if ($SiteVars) {
|
||||
if (-not (Test-Path -LiteralPath $SiteVars -PathType Leaf)) { Fail "-SiteVars file not found: $SiteVars" }
|
||||
$SiteVars = (Resolve-Path -LiteralPath $SiteVars).Path
|
||||
$lines = Get-Content -LiteralPath $SiteVars
|
||||
foreach ($v in 'pxe_nic_mac', 'corp_nic_mac') {
|
||||
$line = @($lines | Where-Object { $_ -match "^${v}:" }) | Select-Object -Last 1
|
||||
if (-not $line) { continue }
|
||||
if ($line -match "^${v}:\s*([`"'][`"'])?\s*(#.*)?$") { continue }
|
||||
if ($line -notmatch "^${v}:\s*[`"']?([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}[`"']?\s*(#.*)?$") {
|
||||
Fail "$SiteVars sets $v to something that is not a MAC (aa:bb:cc:dd:ee:ff) or empty"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# iPXE chain: the playbook's ipxe_*_sha256 pins.
|
||||
$yml = [IO.File]::ReadAllText((Join-Path $PlaybookDir 'pxe_server_setup.yml'))
|
||||
function Get-Pin([string]$name) {
|
||||
$m = [regex]::Match($yml, "(?m)^\s*${name}:\s*`"([0-9a-f]{64})`"")
|
||||
if (-not $m.Success) { Fail "no $name pin in playbook\pxe_server_setup.yml" }
|
||||
$m.Groups[1].Value
|
||||
}
|
||||
$tarball = Join-Path $BootFiles 'ipxeboot.tar.gz'
|
||||
$shim = Join-Path $BootFiles 'ipxe-shimx64.efi'
|
||||
if (-not (Test-Path -LiteralPath $tarball) -or (Get-Sha256 $tarball) -ne (Get-Pin 'ipxe_tarball_sha256')) {
|
||||
Fail "$tarball is missing or does not match ipxe_tarball_sha256 (scripts/fetch-ipxe.sh fetches the pinned release)"
|
||||
}
|
||||
if (-not (Test-Path -LiteralPath $shim) -or (Get-Sha256 $shim) -ne (Get-Pin 'ipxe_shim_sha256')) {
|
||||
Fail "$shim is missing or does not match ipxe_shim_sha256"
|
||||
}
|
||||
$tmpTar = Join-Path ([IO.Path]::GetTempPath()) ('pxeusb-' + [guid]::NewGuid().ToString('N'))
|
||||
New-Item -ItemType Directory -Path $tmpTar | Out-Null
|
||||
try {
|
||||
& tar.exe -xzf $tarball -C $tmpTar 'ipxeboot/x86_64-sb/snponly.efi' 'ipxeboot/x86_64/ipxe.pxe' 2>$null
|
||||
foreach ($p in @(@('ipxeboot\x86_64-sb\snponly.efi', 'ipxe_snponly_sha256'), @('ipxeboot\x86_64\ipxe.pxe', 'ipxe_bios_sha256'))) {
|
||||
$f = Join-Path $tmpTar $p[0]
|
||||
if (-not (Test-Path -LiteralPath $f) -or (Get-Sha256 $f) -ne (Get-Pin $p[1])) { Fail "$($p[0]) in ipxeboot.tar.gz does not match $($p[1])" }
|
||||
}
|
||||
} finally { Remove-Item -LiteralPath $tmpTar -Recurse -Force -ErrorAction SilentlyContinue }
|
||||
Say ' iPXE chain matches the playbook pins'
|
||||
|
||||
# Wheels: each pinned package exactly once, at its pinned version and hash.
|
||||
$reqs = [IO.File]::ReadAllText((Join-Path $WebappDir 'requirements.txt'))
|
||||
function Norm([string]$n) { ($n -replace '[-_.]+', '_').ToLowerInvariant() }
|
||||
$pins = @{}; $hashes = @{}
|
||||
foreach ($m in [regex]::Matches($reqs, '(?m)^([A-Za-z0-9_.-]+)==([^\s\\]+)((?:\s*\\?\s*--hash=sha256:[0-9a-f]{64})+)')) {
|
||||
$n = Norm $m.Groups[1].Value
|
||||
$pins[$n] = $m.Groups[2].Value
|
||||
$hashes[$n] = @([regex]::Matches($m.Groups[3].Value, 'sha256:([0-9a-f]{64})') | ForEach-Object { $_.Groups[1].Value })
|
||||
}
|
||||
if ($pins.Count -eq 0) { Fail 'no hashed pins in webapp\requirements.txt' }
|
||||
if (-not (Test-Path -LiteralPath $WheelsDir)) { Fail "no $WheelsDir (scripts/download-packages.sh builds it)" }
|
||||
$bad = New-Object System.Collections.Generic.List[string]
|
||||
$seen = @{}
|
||||
foreach ($w in (Get-ChildItem -LiteralPath $WheelsDir -File)) {
|
||||
if ($w.Extension -ne '.whl') { $bad.Add("$($w.Name): not a wheel"); continue }
|
||||
$parts = $w.Name.Split('-')
|
||||
$n = Norm $parts[0]; $ver = $parts[1]
|
||||
if (-not $pins.ContainsKey($n)) { $bad.Add("$($w.Name): $n is not pinned") }
|
||||
elseif ($ver -ne $pins[$n]) { $bad.Add("$($w.Name): version $ver, pinned $($pins[$n])") }
|
||||
elseif ($hashes[$n] -notcontains (Get-Sha256 $w.FullName)) { $bad.Add("$($w.Name): sha256 is not a pinned hash of $n") }
|
||||
$seen[$n] = 1 + [int]($seen[$n])
|
||||
}
|
||||
foreach ($n in $pins.Keys) { if ([int]($seen[$n]) -ne 1) { $bad.Add("${n}: $([int]($seen[$n])) wheels (need exactly 1)") } }
|
||||
if ($bad.Count -gt 0) { Fail ("pip-wheels\ does not match webapp\requirements.txt:`n " + ($bad -join "`n ")) }
|
||||
Say " $($pins.Count) wheels match their pinned hashes"
|
||||
|
||||
# site-content: listed in SHA256SUMS, nothing FAT32 cannot hold.
|
||||
$scSums = @{}
|
||||
if ($SiteContent -and -not (Test-Path -LiteralPath $SiteContent)) {
|
||||
# Optional: a site with no other PXE server has none. The server installs
|
||||
# and images without it; that content is added later if the site uses it.
|
||||
Say ' no site-content\ in the build folder (optional): building without it'
|
||||
$SiteContent = ''
|
||||
}
|
||||
if ($SiteContent) {
|
||||
$sumFile = Join-Path $SiteContent 'SHA256SUMS'
|
||||
if (-not (Test-Path -LiteralPath $sumFile)) { Fail 'site-content\ has no SHA256SUMS list; remove the folder or replace it with a complete copy.' }
|
||||
foreach ($l in [IO.File]::ReadAllLines($sumFile)) {
|
||||
if ($l -match '^([0-9a-f]{64})\s+\*?\./(.+)$') { $scSums[($Matches[2] -replace '/', '\')] = $Matches[1] }
|
||||
}
|
||||
$scFiles = @(Get-Tree $SiteContent | Where-Object { $_.Rel -ne 'SHA256SUMS' })
|
||||
if ($scFiles.Count -ne $scSums.Count) { Fail "site-content\ holds $($scFiles.Count) files but SHA256SUMS lists $($scSums.Count). Re-run the capture." }
|
||||
foreach ($t in $scFiles) {
|
||||
if (-not $scSums.ContainsKey($t.Rel)) { Fail "site-content\$($t.Rel) is not in SHA256SUMS" }
|
||||
if ($t.Size -gt $FatMax) { Fail "site-content\$($t.Rel) is 4 GiB or more (FAT32 cannot hold it)" }
|
||||
}
|
||||
Say " site-content\ lists $($scSums.Count) files (hashed when they are read back)"
|
||||
}
|
||||
if ($WinPESource) {
|
||||
if (-not (Test-Path -LiteralPath $WinPESource)) { Fail "WinPE source not found: $WinPESource" }
|
||||
$big = @(Get-Tree $WinPESource | Where-Object { $_.Size -gt $FatMax })
|
||||
if ($big.Count) { Fail "WinPE content has files of 4 GiB or more; import those through the webapp instead: $($big[0].Rel)" }
|
||||
}
|
||||
|
||||
# --- plan --------------------------------------------------------------------
|
||||
Add-File (Join-Path $Autoinstall 'user-data') ''
|
||||
Add-File (Join-Path $Autoinstall 'meta-data') ''
|
||||
Add-File (Join-Path $OfflineDir 'Packages') 'packages'
|
||||
Add-File (Join-Path $OfflineDir 'explicit.list') 'packages'
|
||||
$debs = @(Get-ChildItem -LiteralPath $OfflineDir -Filter '*.deb' -File)
|
||||
foreach ($d in $debs) { Add-Plan $d.FullName (Join-Path 'packages' $d.Name) $d.Length }
|
||||
Add-File (Join-Path $RepoRoot 'scripts\offline-apt-install.sh') ''
|
||||
# The playbook without a repo-local site-vars.yml: the stick carries exactly
|
||||
# -SiteVars (or none).
|
||||
Add-Tree $PlaybookDir 'playbook' $false @() @() @('site-vars.yml')
|
||||
if ($SiteVars) { Add-Plan $SiteVars 'playbook\site-vars.yml' (Get-Item -LiteralPath $SiteVars).Length }
|
||||
Add-File (Join-Path $RepoRoot 'scripts\lint-unattend.py') 'scripts'
|
||||
Add-Tree $WebappDir 'webapp' $false @('__pycache__') @('.pyc', '.pyo')
|
||||
Add-Tree $WheelsDir 'pip-wheels'
|
||||
if (Test-Path -LiteralPath $BootFiles) {
|
||||
foreach ($f in (Get-ChildItem -LiteralPath $BootFiles -File)) { Add-Plan $f.FullName $f.Name $f.Length $true }
|
||||
}
|
||||
if ($BootTools -and (Test-Path -LiteralPath $BootTools)) { Add-Tree $BootTools 'boot-tools' }
|
||||
if ($Enrollment -and (Test-Path -LiteralPath $Enrollment)) { Add-Tree $Enrollment 'enrollment' $true }
|
||||
if ($SiteContent) { Add-Tree $SiteContent 'site-content' }
|
||||
if ($BiosDir -and (Test-Path -LiteralPath $BiosDir)) { Add-Tree $BiosDir 'bios' }
|
||||
if ($DriversDir -and (Test-Path -LiteralPath $DriversDir)) { Add-Tree $DriversDir 'drivers' $true }
|
||||
if ($WinPESource) { Add-Tree $WinPESource 'images' }
|
||||
|
||||
foreach ($p in $Plan) {
|
||||
if ($p.Size -gt $FatMax -and -not $p.Split) { Fail "$($p.Src) is 4 GiB or more and FAT32 cannot hold it" }
|
||||
}
|
||||
$payloadBytes = ($Plan | Measure-Object -Property Size -Sum).Sum
|
||||
$isoBytes = (Get-Item -LiteralPath $IsoPath).Length
|
||||
$need = [int64]$payloadBytes + $isoBytes
|
||||
$need = $need + [int64]($need / 10) + 256MB
|
||||
Say (' payload {0:N1} GB + ISO {1:N1} GB; needs about {2:N1} GB' -f ($payloadBytes / 1GB), ($isoBytes / 1GB), ($need / 1GB))
|
||||
if ($need -gt $PartMax) { Fail ('that is more than the {0:N0} GB FAT32 partition Windows can format; skip a payload (-SkipDrivers, -SkipEnrollment ...)' -f ($PartMax / 1GB)) }
|
||||
|
||||
# --- the disk ------------------------------------------------------------------
|
||||
$disk = Get-Disk -Number $DiskNumber -ErrorAction SilentlyContinue
|
||||
if (-not $disk) { Show-UsbDisks; Fail "no disk $DiskNumber" }
|
||||
if ($disk.IsBoot -or $disk.IsSystem) { Fail "disk $DiskNumber is the boot/system disk" }
|
||||
if ($disk.BusType -ne 'USB') { Fail "disk $DiskNumber is on bus $($disk.BusType), not USB. Refusing." }
|
||||
$partSize = [Math]::Min([int64]$disk.Size - 4MB, $PartMax)
|
||||
if ($partSize -lt $need) { Fail ('disk {0} is {1:N1} GB; the build needs about {2:N1} GB' -f $DiskNumber, ($disk.Size / 1GB), ($need / 1GB)) }
|
||||
|
||||
Say ''
|
||||
Say ('Disk {0}: {1}, {2:N1} GB, {3}' -f $disk.Number, $disk.FriendlyName, ($disk.Size / 1GB), $disk.PartitionStyle)
|
||||
foreach ($v in @(Get-Partition -DiskNumber $DiskNumber -ErrorAction SilentlyContinue | Get-Volume -ErrorAction SilentlyContinue)) {
|
||||
Say (' now holds: {0}: "{1}" {2} {3:N1} GB' -f $v.DriveLetter, $v.FileSystemLabel, $v.FileSystem, ($v.Size / 1GB))
|
||||
}
|
||||
if (-not $Yes) {
|
||||
$ans = Read-Host "EVERYTHING on disk $DiskNumber will be erased. Type the disk number to continue"
|
||||
if ($ans -ne [string]$DiskNumber) { Say 'Aborted.'; Pause-IfInteractive; exit 1 }
|
||||
}
|
||||
|
||||
# --- partition and format ------------------------------------------------------
|
||||
Say ''
|
||||
Say "[2/6] Partitioning disk $DiskNumber (MBR, one FAT32 partition, label CIDATA)..."
|
||||
# diskpart, not Clear-Disk / New-Partition: on removable USB sticks those
|
||||
# often leave the old table half in place ("Not enough available capacity").
|
||||
# Windows formats FAT32 up to 32 GB, so the partition is at most that.
|
||||
$sizeMB = [int64][Math]::Floor($partSize / 1MB)
|
||||
$dpScript = Join-Path ([IO.Path]::GetTempPath()) ('pxeusb-diskpart-' + [guid]::NewGuid().ToString('N') + '.txt')
|
||||
@(
|
||||
"select disk $DiskNumber"
|
||||
'clean'
|
||||
'convert mbr'
|
||||
"create partition primary size=$sizeMB"
|
||||
'active'
|
||||
'format fs=fat32 quick label=CIDATA'
|
||||
'assign'
|
||||
) | Set-Content -LiteralPath $dpScript -Encoding ASCII
|
||||
$dpOut = & diskpart.exe /s $dpScript 2>&1
|
||||
$dpCode = $LASTEXITCODE
|
||||
Remove-Item -LiteralPath $dpScript -Force -ErrorAction SilentlyContinue
|
||||
if ($dpCode -ne 0) {
|
||||
$dpOut | Select-Object -Last 15 | ForEach-Object { Say " $_" }
|
||||
Fail "diskpart failed (exit $dpCode) on disk $DiskNumber"
|
||||
}
|
||||
Start-Sleep -Seconds 3
|
||||
Update-Disk -Number $DiskNumber -ErrorAction SilentlyContinue
|
||||
$part = @(Get-Partition -DiskNumber $DiskNumber | Where-Object { $_.Type -ne 'Reserved' }) | Select-Object -First 1
|
||||
if (-not $part) { Fail "no partition on disk $DiskNumber after diskpart" }
|
||||
if (-not $part.DriveLetter) {
|
||||
$part | Add-PartitionAccessPath -AssignDriveLetter
|
||||
Start-Sleep -Seconds 2
|
||||
$part = Get-Partition -DiskNumber $DiskNumber -PartitionNumber $part.PartitionNumber
|
||||
}
|
||||
if (-not $part.DriveLetter) { Fail 'the new partition got no drive letter' }
|
||||
$vol = Get-Volume -DriveLetter $part.DriveLetter
|
||||
if ($vol.FileSystem -ne 'FAT32' -or $vol.FileSystemLabel -ne 'CIDATA') {
|
||||
Fail "disk $DiskNumber is $($vol.FileSystem) '$($vol.FileSystemLabel)' after diskpart, not FAT32 'CIDATA'"
|
||||
}
|
||||
$Usb = "$($part.DriveLetter):\"
|
||||
Say (' {0} FAT32 "CIDATA" {1:N1} GB' -f $Usb, ($vol.Size / 1GB))
|
||||
|
||||
# --- the ISO -------------------------------------------------------------------
|
||||
Say ''
|
||||
Say '[3/6] Adding the Ubuntu ISO files...'
|
||||
# Drive letter of the mounted ISO: the CD-ROM letter that was not there
|
||||
# before. (Mount-DiskImage -PassThru | Get-Volume fails on some Windows
|
||||
# PowerShell 5.1 builds with "Argument types do not match".)
|
||||
function Get-CdLetters { @(Get-CimInstance Win32_LogicalDisk -Filter 'DriveType=5' | ForEach-Object { $_.DeviceID.Substring(0, 1) }) }
|
||||
$before = Get-CdLetters
|
||||
Mount-DiskImage -ImagePath $IsoPath | Out-Null
|
||||
try {
|
||||
$isoLetter = $null
|
||||
for ($w = 0; $w -lt 30 -and -not $isoLetter; $w++) {
|
||||
$isoLetter = @(Get-CdLetters | Where-Object { $before -notcontains $_ }) | Select-Object -First 1
|
||||
if (-not $isoLetter) {
|
||||
try { $isoLetter = (Get-DiskImage -ImagePath $IsoPath | Get-Volume -ErrorAction Stop).DriveLetter } catch { }
|
||||
}
|
||||
if (-not $isoLetter) { Start-Sleep -Seconds 1 }
|
||||
}
|
||||
if (-not $isoLetter) { Fail "the ISO was mounted but got no drive letter: $IsoPath" }
|
||||
$isoRoot = "${isoLetter}:\"
|
||||
Say " ISO mounted as $isoRoot"
|
||||
$isoTree = Get-Tree $isoRoot
|
||||
$grubRel = 'boot\grub\grub.cfg'
|
||||
$grubSrc = Join-Path $isoRoot $grubRel
|
||||
if (-not (Test-Path -LiteralPath $grubSrc)) { Fail "the ISO has no $grubRel" }
|
||||
if (-not (Test-Path -LiteralPath (Join-Path $isoRoot 'EFI\boot\bootx64.efi'))) { Fail 'the ISO has no EFI\boot\bootx64.efi' }
|
||||
|
||||
# Same edit as build-usb.sh: 'autoinstall' on the default kernel line, 5 s timeout.
|
||||
$grub = [IO.File]::ReadAllText($grubSrc)
|
||||
$patched = $grub.Replace("linux`t/casper/vmlinuz ---", "linux`t/casper/vmlinuz autoinstall ---")
|
||||
$patched = [regex]::Replace($patched, '(?m)^set timeout=30', 'set timeout=5')
|
||||
if ($patched -notmatch 'vmlinuz[^\n]*autoinstall') { Fail 'grub.cfg patch failed: no autoinstall on a kernel line (the ISO grub.cfg format changed)' }
|
||||
$grubBytes = $Utf8NoBom.GetBytes($patched)
|
||||
|
||||
# The payload must not land on an ISO path.
|
||||
$isoNames = @{}
|
||||
foreach ($t in $isoTree) { $isoNames[$t.Rel.ToLowerInvariant()] = 1 }
|
||||
foreach ($p in $Plan) {
|
||||
if ($isoNames.ContainsKey($p.Dest.ToLowerInvariant())) { Fail "payload file $($p.Dest) would overwrite an ISO file" }
|
||||
}
|
||||
$isoPlan = New-Object System.Collections.Generic.List[object]
|
||||
foreach ($t in $isoTree) {
|
||||
if ($t.Size -gt $FatMax) { Fail "ISO file $($t.Rel) is 4 GiB or more" }
|
||||
if ($t.Rel -eq $grubRel) {
|
||||
$isoPlan.Add([pscustomobject]@{ Src = $t.Full; Dest = $t.Rel; Size = [int64]$grubBytes.Length; Split = $false; Data = $grubBytes; Hashes = $null })
|
||||
} else {
|
||||
$isoPlan.Add([pscustomobject]@{ Src = $t.Full; Dest = $t.Rel; Size = $t.Size; Split = $false; Data = $null; Hashes = $null })
|
||||
}
|
||||
}
|
||||
|
||||
# --- copying -----------------------------------------------------------
|
||||
function Copy-PlanItem($p) {
|
||||
$dest = Join-Path $Usb $p.Dest
|
||||
$dir = Split-Path -Parent $dest
|
||||
if (-not (Test-Path -LiteralPath $dir)) { New-Item -ItemType Directory -Path $dir -Force | Out-Null }
|
||||
if ($p.Data) { [IO.File]::WriteAllBytes($dest, $p.Data); return }
|
||||
if (-not ($p.Split -and $p.Size -gt $ChunkSize)) { [IO.File]::Copy($p.Src, $dest, $true); return }
|
||||
# Split like `split -b 3500M -d -a 2 FILE FILE.part.`; the playbook joins them.
|
||||
$p.Hashes = New-Object System.Collections.Generic.List[string]
|
||||
$in = [IO.File]::OpenRead($p.Src)
|
||||
try {
|
||||
$buf = New-Object byte[] (8MB)
|
||||
$n = 0
|
||||
while ($in.Position -lt $in.Length) {
|
||||
$chunk = '{0}.part.{1:D2}' -f $dest, $n
|
||||
$out = [IO.File]::Create($chunk)
|
||||
$sha = [System.Security.Cryptography.SHA256]::Create()
|
||||
try {
|
||||
$left = [Math]::Min($ChunkSize, $in.Length - $in.Position)
|
||||
while ($left -gt 0) {
|
||||
$r = $in.Read($buf, 0, [int][Math]::Min($buf.Length, $left))
|
||||
if ($r -le 0) { throw "short read on $($p.Src)" }
|
||||
$out.Write($buf, 0, $r)
|
||||
[void]$sha.TransformBlock($buf, 0, $r, $null, 0)
|
||||
$left -= $r
|
||||
}
|
||||
[void]$sha.TransformFinalBlock($buf, 0, 0)
|
||||
$p.Hashes.Add((-join ($sha.Hash | ForEach-Object { $_.ToString('x2') })))
|
||||
} finally { $out.Dispose(); $sha.Dispose() }
|
||||
$n++
|
||||
}
|
||||
} finally { $in.Dispose() }
|
||||
Say " split $($p.Dest) into $n parts"
|
||||
}
|
||||
|
||||
# One list (not @() + @(): on Windows PowerShell 5.1 that adds the two
|
||||
# List objects themselves and fails "Argument types do not match").
|
||||
$all = New-Object System.Collections.Generic.List[object]
|
||||
$all.AddRange($isoPlan)
|
||||
$all.AddRange($Plan)
|
||||
$total = [int64](($all | Measure-Object -Property Size -Sum).Sum)
|
||||
$done = [int64]0
|
||||
$i = 0
|
||||
foreach ($p in $isoPlan) {
|
||||
Copy-PlanItem $p
|
||||
$done += $p.Size; $i++
|
||||
if ($i % 50 -eq 0) { Write-Progress -Activity 'Copying to the USB' -Status $p.Dest -PercentComplete ([int](100 * $done / $total)) }
|
||||
}
|
||||
Say " $($isoPlan.Count) ISO files copied (grub.cfg: autoinstall, 5 s timeout)"
|
||||
|
||||
Say ''
|
||||
Say '[4/6] Copying the payload (autoinstall, packages, playbook, webapp, boot files...)...'
|
||||
foreach ($p in $Plan) {
|
||||
Copy-PlanItem $p
|
||||
$done += $p.Size; $i++
|
||||
Write-Progress -Activity 'Copying to the USB' -Status $p.Dest -PercentComplete ([int](100 * $done / $total))
|
||||
}
|
||||
Write-Progress -Activity 'Copying to the USB' -Completed
|
||||
Say (" {0} files, {1:N1} GB ({2} .deb, site vars: {3})" -f $Plan.Count, ($payloadBytes / 1GB), $debs.Count, $(if ($SiteVars) { $SiteVars } else { 'none' }))
|
||||
|
||||
Say ''
|
||||
Say '[5/6] Flushing...'
|
||||
Write-VolumeCache -DriveLetter $part.DriveLetter
|
||||
|
||||
# --- read back ---------------------------------------------------------
|
||||
Say ''
|
||||
Say ('[6/6] Reading every file back from the stick ({0})...' -f $(if ($QuickVerify) { 'sizes' } else { 'SHA-256' }))
|
||||
$errors = New-Object System.Collections.Generic.List[string]
|
||||
$done = [int64]0
|
||||
foreach ($p in $all) {
|
||||
$dest = Join-Path $Usb $p.Dest
|
||||
$done += $p.Size
|
||||
Write-Progress -Activity 'Verifying the USB' -Status $p.Dest -PercentComplete ([int](100 * $done / $total))
|
||||
if ($p.Hashes) {
|
||||
for ($k = 0; $k -lt $p.Hashes.Count; $k++) {
|
||||
$chunk = '{0}.part.{1:D2}' -f $dest, $k
|
||||
if (-not (Test-Path -LiteralPath $chunk)) { $errors.Add("$($p.Dest).part.$k missing"); continue }
|
||||
if (-not $QuickVerify -and (Get-Sha256 $chunk) -ne $p.Hashes[$k]) { $errors.Add("$($p.Dest).part.$k differs") }
|
||||
}
|
||||
continue
|
||||
}
|
||||
if (-not (Test-Path -LiteralPath $dest)) { $errors.Add("$($p.Dest) missing"); continue }
|
||||
if ((Get-Item -LiteralPath $dest).Length -ne $p.Size) { $errors.Add("$($p.Dest) has the wrong size"); continue }
|
||||
if ($QuickVerify) { continue }
|
||||
$got = Get-Sha256 $dest
|
||||
if ($p.Data) {
|
||||
$ms = New-Object IO.MemoryStream (, $p.Data)
|
||||
$want = Get-StreamSha256 $ms
|
||||
} elseif ($p.Dest.StartsWith('site-content\') -and $scSums.ContainsKey($p.Dest.Substring(13))) {
|
||||
$want = $scSums[$p.Dest.Substring(13)] # the captured hash, not today's source
|
||||
} else {
|
||||
$want = Get-Sha256 $p.Src
|
||||
}
|
||||
if ($got -ne $want) { $errors.Add("$($p.Dest) differs from its source") }
|
||||
}
|
||||
Write-Progress -Activity 'Verifying the USB' -Completed
|
||||
} finally {
|
||||
Dismount-DiskImage -ImagePath $IsoPath | Out-Null
|
||||
}
|
||||
|
||||
if ($errors.Count -gt 0) {
|
||||
Say ''
|
||||
Say "The stick is NOT good: $($errors.Count) problem(s):"
|
||||
$errors | Select-Object -First 20 | ForEach-Object { Say " $_" }
|
||||
Fail 'rebuild it (try another stick or port)'
|
||||
}
|
||||
|
||||
Say ''
|
||||
Say '============================================'
|
||||
Say "USB build complete and verified: $Usb"
|
||||
Say '============================================'
|
||||
Say 'Next:'
|
||||
Say ' 1. Eject the stick (Safely Remove).'
|
||||
Say ' 2. On the server: plug in ONLY the imaging switch (GE unplugged), boot the'
|
||||
Say ' stick in UEFI mode (F12 / boot menu). Ubuntu installs itself.'
|
||||
Say ' 3. First boot picks the imaging port, pins it as pxe0 = 172.16.9.1 and runs'
|
||||
Say ' the playbook. Then plug GE into any other port.'
|
||||
Say ' 4. If the console shows a banner, read /var/log/first-boot.log; for the'
|
||||
Say ' imaging port, log in and run: sudo pxe-pick-nic'
|
||||
Pause-IfInteractive
|
||||
exit 0
|
||||
228
scripts/Capture-SiteContent.ps1
Normal file
228
scripts/Capture-SiteContent.ps1
Normal file
@@ -0,0 +1,228 @@
|
||||
#
|
||||
# Capture-SiteContent.ps1 - Pull the site content from a running PXE server
|
||||
# into the pxe-usb build folder, on a Windows PC.
|
||||
#
|
||||
# The Windows twin of scripts/capture-site-content.sh. A new PXE server
|
||||
# needs content that neither the repo nor the MCL image import brings; it
|
||||
# exists only on a working server. This pulls, into the build folder:
|
||||
# site-content\shared\Applications\extra\ Office, Adobe, OpenText, Java,
|
||||
# fonts, Wi-Fi profile, Zscaler (every unattend installs from it;
|
||||
# pxe-status\ is left out, the playbook deploys it)
|
||||
# site-content\images\<image>\Applications\ each image's own files and
|
||||
# folders there: GE CA certificates, the Engineering Layer payload
|
||||
# (without Install-EngineeringLayer.ps1, which the repo carries)
|
||||
# site-content\links.txt the links images keep there
|
||||
# site-content\SHA256SUMS checked by the playbook
|
||||
# bios-staging\ *.exe and models.txt from _shared\BIOS
|
||||
# boot-files\boot.wim
|
||||
# boot-tools\blancco\ the Blancco files already in that folder
|
||||
#
|
||||
# How: ONE ssh session (one password prompt) runs a small script on the
|
||||
# server that collects those files, hashes them there (SHA-256) and streams
|
||||
# them back as a tar archive. Here the archive is unpacked into a scratch
|
||||
# folder and every file is checked against the server's hashes, and against
|
||||
# what FAT32 can hold (under 4 GiB, no two names that differ only in case).
|
||||
# Only when all of that passes does the new content replace the old. The
|
||||
# server is only read.
|
||||
#
|
||||
# Needs: Windows 10/11 (ssh.exe and tar.exe are built in), SSH access to the
|
||||
# server as pxe (from the GE network use the server's GE address).
|
||||
#
|
||||
# Usage: double-click Capture-SiteContent.cmd in the build folder (asks for
|
||||
# the server address), or:
|
||||
# .\scripts\Capture-SiteContent.ps1 -Server 10.x.x.x
|
||||
#
|
||||
# Exit codes: 0 captured and verified, 1 failed (the old content is kept).
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$Server = '',
|
||||
[string]$User = 'pxe',
|
||||
[string]$PayloadRoot = '',
|
||||
[switch]$Interactive
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
Set-StrictMode -Version 2
|
||||
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
if (-not $PayloadRoot) { $PayloadRoot = $RepoRoot }
|
||||
$PayloadRoot = (Resolve-Path -LiteralPath $PayloadRoot).Path
|
||||
$FatMax = [int64]4GB - 1
|
||||
|
||||
function Say([string]$m) { Write-Host $m }
|
||||
function Done([int]$code) {
|
||||
if ($Interactive) { Write-Host ''; Read-Host 'Press Enter to close' | Out-Null }
|
||||
exit $code
|
||||
}
|
||||
function Fail([string]$m) { Write-Host "ERROR: $m" -ForegroundColor Red; Done 1 }
|
||||
trap { Write-Host "ERROR: $_" -ForegroundColor Red; Done 1 }
|
||||
|
||||
foreach ($exe in 'ssh.exe', 'tar.exe') {
|
||||
if (-not (Get-Command $exe -ErrorAction SilentlyContinue)) { Fail "$exe not found (Windows 10 1803 or later has both)" }
|
||||
}
|
||||
if (-not $Server) {
|
||||
if (-not $Interactive) { Fail 'give -Server (the PXE server address; from the GE network its GE address)' }
|
||||
Say 'Address of the running PXE server to copy from.'
|
||||
Say 'From the GE network use its GE address; on the imaging switch 172.16.9.1.'
|
||||
$Server = (Read-Host 'Server').Trim()
|
||||
if (-not $Server) { Fail 'no server given' }
|
||||
}
|
||||
if ($Server -notmatch '^[A-Za-z0-9.\-:]+$') { Fail "not a host name or address: $Server" }
|
||||
|
||||
# The Blancco files to refresh: whatever is already in boot-tools\blancco
|
||||
# (the server also holds files its playbook builds; those are not pulled).
|
||||
$blanccoDir = Join-Path $PayloadRoot 'boot-tools\blancco'
|
||||
$blancco = @()
|
||||
if (Test-Path -LiteralPath $blanccoDir) {
|
||||
$blancco = @(Get-ChildItem -LiteralPath $blanccoDir -Force | ForEach-Object { $_.Name } |
|
||||
Where-Object { $_ -match '^[A-Za-z0-9._+-]+$' })
|
||||
}
|
||||
|
||||
# --- the script that runs on the server (bash, stdout = the tar stream) -----
|
||||
$remote = @'
|
||||
set -euo pipefail
|
||||
SHARE=${CAPTURE_SHARE:-/srv/samba/winpeapps}
|
||||
WWW=${CAPTURE_WWW:-/var/www/html}
|
||||
ST=$(mktemp -d /tmp/pxe-capture.XXXXXX)
|
||||
trap 'rm -rf "$ST"' EXIT
|
||||
SC=$ST/site-content
|
||||
mkdir -p "$SC/shared/Applications" "$SC/images" "$ST/bios-staging" "$ST/boot-files" "$ST/boot-tools/blancco"
|
||||
[ -d "$SHARE/_shared/Applications/extra" ] && ln -s "$SHARE/_shared/Applications/extra" "$SC/shared/Applications/extra"
|
||||
printf '# image<TAB>name in Deploy/Applications<TAB>link target under winpeapps/\n' >"$SC/links.txt"
|
||||
for apps in "$SHARE"/*/Deploy/Applications; do
|
||||
[ -d "$apps" ] || continue
|
||||
img=${apps#"$SHARE"/}; img=${img%%/*}
|
||||
case "$img" in _*) continue ;; esac
|
||||
for e in "$apps"/* "$apps"/.[!.]*; do
|
||||
[ -e "$e" ] || [ -L "$e" ] || continue
|
||||
n=${e##*/}
|
||||
if [ -L "$e" ]; then
|
||||
t=$(readlink "$e")
|
||||
case "$t" in
|
||||
"$SHARE"/_shared/*) printf '%s\t%s\t%s\n' "$img" "$n" "${t#"$SHARE"/}" >>"$SC/links.txt" ;;
|
||||
*) echo "WARNING: $img/Deploy/Applications/$n links outside _shared ($t); not captured" >&2 ;;
|
||||
esac
|
||||
continue
|
||||
fi
|
||||
# a real BPRT folder is an old import's copy of the vendor folder
|
||||
[ "${n,,}" = bprt ] && continue
|
||||
mkdir -p "$SC/images/$img/Applications"
|
||||
ln -s "$e" "$SC/images/$img/Applications/$n"
|
||||
done
|
||||
done
|
||||
for f in "$SHARE"/_shared/BIOS/*.exe "$SHARE"/_shared/BIOS/models.txt; do
|
||||
[ -f "$f" ] && ln -s "$f" "$ST/bios-staging/${f##*/}"
|
||||
done
|
||||
[ -f "$WWW/win11/sources/boot.wim" ] && ln -s "$WWW/win11/sources/boot.wim" "$ST/boot-files/boot.wim"
|
||||
for n in "$@"; do
|
||||
[ -e "$WWW/blancco/$n" ] && ln -s "$WWW/blancco/$n" "$ST/boot-tools/blancco/$n"
|
||||
done
|
||||
EXCL=(--exclude=./site-content/shared/Applications/extra/pxe-status --exclude=Install-EngineeringLayer.ps1)
|
||||
echo "pxe-capture: hashing on the server (a few minutes for about 12 GB)..." >&2
|
||||
( cd "$ST" && find -L . -type f ! -path './site-content/shared/Applications/extra/pxe-status/*' \
|
||||
! -name Install-EngineeringLayer.ps1 ! -name capture-SHA256SUMS -print0 |
|
||||
sort -z | xargs -0 -r sha256sum ) >"$ST/capture-SHA256SUMS.tmp"
|
||||
mv "$ST/capture-SHA256SUMS.tmp" "$ST/capture-SHA256SUMS"
|
||||
echo "pxe-capture: $(wc -l <"$ST/capture-SHA256SUMS") files; sending..." >&2
|
||||
tar -chf - -C "$ST" "${EXCL[@]}" .
|
||||
'@
|
||||
|
||||
$scratch = Join-Path $PayloadRoot '.capture-new'
|
||||
$old = Join-Path $PayloadRoot '.capture-old'
|
||||
foreach ($d in $scratch, $old) {
|
||||
if (Test-Path -LiteralPath $d) { Remove-Item -LiteralPath $d -Recurse -Force }
|
||||
}
|
||||
New-Item -ItemType Directory -Path $scratch | Out-Null
|
||||
$remoteFile = Join-Path ([IO.Path]::GetTempPath()) ('pxe-capture-' + [guid]::NewGuid().ToString('N') + '.sh')
|
||||
[IO.File]::WriteAllText($remoteFile, ($remote -replace "`r", ''), (New-Object System.Text.UTF8Encoding($false)))
|
||||
|
||||
Say '============================================'
|
||||
Say 'PXE site content capture (Windows)'
|
||||
Say '============================================'
|
||||
Say "Server : $User@$Server"
|
||||
Say "Folder : $PayloadRoot"
|
||||
Say "Blancco : $($blancco.Count) known files"
|
||||
Say ''
|
||||
Say "[1/3] Copying from the server (enter the $User password when asked)..."
|
||||
# cmd's pipe passes bytes unchanged; a PowerShell pipe would re-encode the tar.
|
||||
$names = ($blancco | ForEach-Object { "'" + $_ + "'" }) -join ' '
|
||||
$cmdline = "ssh.exe -o StrictHostKeyChecking=accept-new -o ServerAliveInterval=30 $User@$Server `"bash -s -- $names`" < `"$remoteFile`" | tar.exe -xf - -C `"$scratch`""
|
||||
& cmd.exe /c $cmdline
|
||||
$tarCode = $LASTEXITCODE
|
||||
Remove-Item -LiteralPath $remoteFile -Force -ErrorAction SilentlyContinue
|
||||
|
||||
$sumFile = Join-Path $scratch 'capture-SHA256SUMS'
|
||||
if (-not (Test-Path -LiteralPath $sumFile)) {
|
||||
Remove-Item -LiteralPath $scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Fail "nothing usable arrived (ssh/tar exit $tarCode): wrong address or password, or the server is not reachable. Nothing was changed."
|
||||
}
|
||||
|
||||
Say ''
|
||||
Say '[2/3] Checking every file against the server''s SHA-256...'
|
||||
$lines = @([IO.File]::ReadAllLines($sumFile) | Where-Object { $_ })
|
||||
$want = @{}
|
||||
$lower = @{}
|
||||
$errors = New-Object System.Collections.Generic.List[string]
|
||||
foreach ($l in $lines) {
|
||||
if ($l -notmatch '^([0-9a-f]{64}) \./(.+)$') { $errors.Add("bad line in the server's list: $l"); continue }
|
||||
$rel = $Matches[2]
|
||||
$key = $rel.ToLowerInvariant()
|
||||
if ($lower.ContainsKey($key)) { $errors.Add("names differ only in case (one would overwrite the other): $rel / $($lower[$key])") }
|
||||
$lower[$key] = $rel
|
||||
$want[$rel] = $Matches[1]
|
||||
}
|
||||
$i = 0
|
||||
foreach ($rel in $want.Keys) {
|
||||
$i++
|
||||
if ($i % 25 -eq 0) { Write-Progress -Activity 'Verifying' -Status $rel -PercentComplete ([int](100 * $i / $want.Count)) }
|
||||
$p = Join-Path $scratch ($rel -replace '/', '\')
|
||||
if (-not (Test-Path -LiteralPath $p -PathType Leaf)) { $errors.Add("missing: $rel"); continue }
|
||||
$f = Get-Item -LiteralPath $p
|
||||
if ($f.Length -gt $FatMax) { $errors.Add("4 GiB or more (FAT32 cannot hold it): $rel") }
|
||||
if ((Get-FileHash -LiteralPath $p -Algorithm SHA256).Hash.ToLowerInvariant() -ne $want[$rel]) { $errors.Add("differs from the server: $rel") }
|
||||
}
|
||||
Write-Progress -Activity 'Verifying' -Completed
|
||||
if ($errors.Count -gt 0) {
|
||||
Say "$($errors.Count) problem(s); nothing was changed:"
|
||||
$errors | Select-Object -First 20 | ForEach-Object { Say " $_" }
|
||||
Remove-Item -LiteralPath $scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Fail 'capture not usable (long paths? see https://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation)'
|
||||
}
|
||||
Say " $($want.Count) files match"
|
||||
|
||||
# site-content\SHA256SUMS in the format the playbook checks (sha256sum -c,
|
||||
# paths relative to site-content), from the server's own hashes.
|
||||
$sc = New-Object System.Collections.Generic.List[string]
|
||||
foreach ($l in $lines) {
|
||||
if ($l -match '^([0-9a-f]{64}) \./site-content/(.+)$') { $sc.Add("$($Matches[1]) ./$($Matches[2])") }
|
||||
}
|
||||
[IO.File]::WriteAllText((Join-Path $scratch 'site-content\SHA256SUMS'), (($sc -join "`n") + "`n"), (New-Object System.Text.UTF8Encoding($false)))
|
||||
|
||||
Say ''
|
||||
Say '[3/3] Replacing the old content...'
|
||||
New-Item -ItemType Directory -Path $old | Out-Null
|
||||
function Swap([string]$rel) {
|
||||
$new = Join-Path $scratch $rel
|
||||
if (-not (Test-Path -LiteralPath $new)) { return }
|
||||
$cur = Join-Path $PayloadRoot $rel
|
||||
if (Test-Path -LiteralPath $cur) {
|
||||
$bak = Join-Path $old $rel
|
||||
New-Item -ItemType Directory -Path (Split-Path -Parent $bak) -Force | Out-Null
|
||||
Move-Item -LiteralPath $cur -Destination $bak
|
||||
}
|
||||
New-Item -ItemType Directory -Path (Split-Path -Parent $cur) -Force | Out-Null
|
||||
Move-Item -LiteralPath $new -Destination $cur
|
||||
Say " $rel"
|
||||
}
|
||||
Swap 'site-content'
|
||||
Swap 'bios-staging'
|
||||
Swap 'boot-files\boot.wim'
|
||||
foreach ($n in $blancco) { Swap (Join-Path 'boot-tools\blancco' $n) }
|
||||
Remove-Item -LiteralPath $old -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -LiteralPath $scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
|
||||
$size = (Get-ChildItem -LiteralPath (Join-Path $PayloadRoot 'site-content') -Recurse -File | Measure-Object Length -Sum).Sum
|
||||
Say ''
|
||||
Say ('Done: site-content {0:N1} GB, {1} files. Build the USB next (Build-PxeUsb.cmd).' -f ($size / 1GB), $sc.Count)
|
||||
Done 0
|
||||
@@ -1,271 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
build-pctype-media.py - Build a per-PCTYPE PESetup media view out of symlinks.
|
||||
|
||||
PESetup's CopyPackages copies the WHOLE of <media>\\Deploy\\Applications recursively
|
||||
to the target (see docs/PESETUP-INTERNALS.md). It does not filter, so every shopfloor
|
||||
bay built from the shared gea-shopfloor media receives every shopfloor payload
|
||||
regardless of its PC type.
|
||||
|
||||
Rather than mutate the shared media per session - which races, because bays image
|
||||
concurrently and CopyPackages is fail-fast, so pulling a path mid-copy fails the
|
||||
OTHER bay's imaging - give each PC type its own media directory built entirely from
|
||||
symlinks. Samba resolves them server-side (follow symlinks + wide links are already
|
||||
on, and every image root already uses this pattern for Sources, Operating Systems,
|
||||
Out-of-box Drivers and Packages), so WinPE sees ordinary directories.
|
||||
|
||||
Layout produced, per type:
|
||||
|
||||
_media/<pctype>/
|
||||
Sources -> base Sources
|
||||
Tools -> base Tools
|
||||
Deploy/Control -> base Deploy/Control
|
||||
Deploy/FlatUnattendW10.xml -> base Deploy/FlatUnattendW10.xml
|
||||
Deploy/Operating Systems -> base Deploy/Operating Systems
|
||||
Deploy/Out-of-box Drivers -> base Deploy/Out-of-box Drivers
|
||||
Deploy/Packages -> base Deploy/Packages
|
||||
Deploy/Tools -> base Deploy/Tools
|
||||
Deploy/Applications/ real directory holding:
|
||||
<every entry of base Deploy/Applications> -> base entry
|
||||
<every entry of _shared/Applications/pctype/<pctype>> -> that entry
|
||||
|
||||
Everything is a link, so a type costs inodes rather than gigabytes, and the shared
|
||||
media stays untouched and readable while bays image.
|
||||
|
||||
The type list comes from the enrollment share's menu.json, the same file the WinPE
|
||||
picker renders, so the media set cannot drift from the boot menu.
|
||||
|
||||
Usage:
|
||||
./build-pctype-media.py # show the plan, change nothing
|
||||
./build-pctype-media.py --apply # build or refresh the media dirs
|
||||
./build-pctype-media.py --verify # resolve every link, report breaks
|
||||
./build-pctype-media.py --apply --prune # also remove types no longer listed
|
||||
./build-pctype-media.py --base gea-standard --prefix _media-std --apply
|
||||
|
||||
Requires: sshpass (remote mode only)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import posixpath
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
PXE_HOST = "172.16.9.1"
|
||||
PXE_USER = "pxe"
|
||||
PXE_PASS = "pxe"
|
||||
IMAGE_BASE = "/srv/samba/winpeapps"
|
||||
SHARED = IMAGE_BASE + "/_shared"
|
||||
MENU_JSON = "/srv/samba/enrollment/shopfloor-setup/menu.json"
|
||||
|
||||
# Underscore prefix keeps these out of the webapp's image-type listing, the same way
|
||||
# _shared is skipped. They are media views, not image types.
|
||||
MEDIA_PREFIX = "_media"
|
||||
DEFAULT_BASE = "gea-shopfloor"
|
||||
|
||||
# Per-type payload lives here. Absent dir just means "nothing type-specific yet".
|
||||
PCTYPE_OVERLAY = SHARED + "/Applications/pctype"
|
||||
|
||||
# Deploy entries linked straight through from the base image. Anything else in the
|
||||
# base Deploy (the FlatUnattendW10.xml.pre-* backups, stray logs) is deliberately
|
||||
# NOT mirrored - the media view is the clean set PESetup actually reads.
|
||||
DEPLOY_LINKS = [
|
||||
"Control",
|
||||
"FlatUnattendW10.xml",
|
||||
"Operating Systems",
|
||||
"Out-of-box Drivers",
|
||||
"Packages",
|
||||
"Tools",
|
||||
]
|
||||
ROOT_LINKS = ["Sources", "Tools"]
|
||||
|
||||
# PESetup will not get far without these, so they are what --verify resolves.
|
||||
CRITICAL_PATHS = [
|
||||
"Sources/PESetup.exe",
|
||||
"Deploy/Control/HardwareDriver.json",
|
||||
"Deploy/Control/OperatingSystem.json",
|
||||
"Deploy/FlatUnattendW10.xml",
|
||||
"Deploy/Operating Systems",
|
||||
"Deploy/Out-of-box Drivers",
|
||||
]
|
||||
|
||||
|
||||
def ssh_cmd(host, cmd, stdin=None):
|
||||
return subprocess.run(
|
||||
["sshpass", "-p", PXE_PASS, "ssh", "-o", "StrictHostKeyChecking=no",
|
||||
"-o", "LogLevel=ERROR", f"{PXE_USER}@{host}", cmd],
|
||||
capture_output=True, text=True, input=stdin)
|
||||
|
||||
|
||||
def sh_quote(s):
|
||||
return "'" + str(s).replace("'", "'\\''") + "'"
|
||||
|
||||
|
||||
def run_root(host, script, apply_it):
|
||||
"""Server-side work needs root: the share is root-owned.
|
||||
|
||||
The script travels inside the command as a heredoc, not on stdin. "echo pxe |
|
||||
sudo -S bash -s" hands the password pipe to bash as well, so bash reads EOF
|
||||
immediately, runs nothing, and still exits 0 - a silent no-op."""
|
||||
if not apply_it:
|
||||
return None
|
||||
remote = "/tmp/build-pctype-media.$$.sh"
|
||||
wrapped = (
|
||||
"cat > %s <<'PCTYPEMEDIAEOF'\n%s\nPCTYPEMEDIAEOF\n"
|
||||
"echo pxe | sudo -S -p '' bash %s; rc=$?; rm -f %s; exit $rc"
|
||||
% (remote, script, remote, remote))
|
||||
r = ssh_cmd(host, wrapped)
|
||||
if r.returncode != 0:
|
||||
sys.exit("ERROR: server-side step failed:\n%s\n%s" % (r.stdout, r.stderr))
|
||||
return r.stdout
|
||||
|
||||
|
||||
def read_types(host, menu_path):
|
||||
r = ssh_cmd(host, "cat %s" % sh_quote(menu_path))
|
||||
if r.returncode != 0:
|
||||
sys.exit("ERROR: cannot read %s (%s)" % (menu_path, r.stderr.strip()))
|
||||
try:
|
||||
menu = json.loads(r.stdout)
|
||||
except json.JSONDecodeError as e:
|
||||
sys.exit("ERROR: %s does not parse: %s" % (menu_path, e))
|
||||
types = [e["key"] for e in menu.get("shopfloor", [])
|
||||
if e.get("enabled", True) and e.get("key")]
|
||||
if not types:
|
||||
sys.exit("ERROR: no enabled types in %s" % menu_path)
|
||||
return types
|
||||
|
||||
|
||||
def list_dir(host, path):
|
||||
r = ssh_cmd(host, "ls -1 %s 2>/dev/null" % sh_quote(path))
|
||||
return [n for n in r.stdout.splitlines() if n] if r.returncode == 0 else []
|
||||
|
||||
|
||||
def plan_for_type(base_dir, media_dir, pctype, base_apps, overlay_entries):
|
||||
"""Every link this type needs, as (link_path, target) pairs."""
|
||||
links = []
|
||||
for name in ROOT_LINKS:
|
||||
links.append((posixpath.join(media_dir, name),
|
||||
posixpath.join(base_dir, name)))
|
||||
for name in DEPLOY_LINKS:
|
||||
links.append((posixpath.join(media_dir, "Deploy", name),
|
||||
posixpath.join(base_dir, "Deploy", name)))
|
||||
apps_dir = posixpath.join(media_dir, "Deploy", "Applications")
|
||||
for name in base_apps:
|
||||
links.append((posixpath.join(apps_dir, name),
|
||||
posixpath.join(base_dir, "Deploy", "Applications", name)))
|
||||
# Overlay wins: a type-specific entry replaces the base entry of the same name.
|
||||
overlay_dir = posixpath.join(PCTYPE_OVERLAY, pctype)
|
||||
for name in overlay_entries:
|
||||
links.append((posixpath.join(apps_dir, name),
|
||||
posixpath.join(overlay_dir, name)))
|
||||
return links
|
||||
|
||||
|
||||
def build_script(media_dir, links):
|
||||
"""Rebuild the type's dir from scratch. Idempotent, and drops stale links.
|
||||
|
||||
Builds into a scratch dir and swaps, so a bay that maps this path mid-refresh
|
||||
sees either the old tree or the new one, never a half-built one."""
|
||||
tmp = media_dir + ".new"
|
||||
out = ["set -e", "rm -rf %s" % sh_quote(tmp)]
|
||||
dirs = {posixpath.dirname(link) for link, _ in links}
|
||||
for d in sorted(dirs):
|
||||
out.append("mkdir -p %s" % sh_quote(d.replace(media_dir, tmp, 1)))
|
||||
for link, target in links:
|
||||
out.append("ln -s %s %s" % (sh_quote(target),
|
||||
sh_quote(link.replace(media_dir, tmp, 1))))
|
||||
out.append("rm -rf %s" % sh_quote(media_dir + ".old"))
|
||||
out.append("if [ -e %s ]; then mv %s %s; fi"
|
||||
% (sh_quote(media_dir), sh_quote(media_dir), sh_quote(media_dir + ".old")))
|
||||
out.append("mv %s %s" % (sh_quote(tmp), sh_quote(media_dir)))
|
||||
out.append("rm -rf %s" % sh_quote(media_dir + ".old"))
|
||||
return "\n".join(out) + "\n"
|
||||
|
||||
|
||||
def verify(host, media_root, types):
|
||||
"""Resolve the paths PESetup reads. A dangling link here is a dead media view."""
|
||||
checks = []
|
||||
for pctype in types:
|
||||
for rel in CRITICAL_PATHS:
|
||||
checks.append(posixpath.join(media_root, pctype, rel))
|
||||
script = "\n".join("if [ -e %s ]; then echo \"OK %s\"; else echo \"DEAD %s\"; fi"
|
||||
% (sh_quote(p), p, p) for p in checks)
|
||||
r = ssh_cmd(host, "bash -s", stdin=script)
|
||||
dead = [line for line in r.stdout.splitlines() if line.startswith("DEAD")]
|
||||
for line in r.stdout.splitlines():
|
||||
if line.startswith("DEAD"):
|
||||
print(" " + line)
|
||||
print(" %d paths checked, %d dead" % (len(checks), len(dead)))
|
||||
return len(dead)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Build per-PCTYPE PESetup media views out of symlinks.")
|
||||
parser.add_argument("--server", default=PXE_HOST, help="PXE server (default: %s)" % PXE_HOST)
|
||||
parser.add_argument("--base", default=DEFAULT_BASE,
|
||||
help="image type the view is built from (default: %s)" % DEFAULT_BASE)
|
||||
parser.add_argument("--prefix", default=MEDIA_PREFIX,
|
||||
help="directory under winpeapps to hold the views (default: %s)"
|
||||
% MEDIA_PREFIX)
|
||||
parser.add_argument("--menu", default=MENU_JSON, help="menu.json path on the server")
|
||||
parser.add_argument("--types", nargs="+", help="explicit type list, bypassing menu.json")
|
||||
parser.add_argument("--apply", action="store_true", help="write; default is a dry run")
|
||||
parser.add_argument("--prune", action="store_true",
|
||||
help="remove media views whose type is no longer listed")
|
||||
parser.add_argument("--verify", action="store_true",
|
||||
help="resolve the critical paths in each view and exit")
|
||||
args = parser.parse_args()
|
||||
|
||||
base_dir = posixpath.join(IMAGE_BASE, args.base)
|
||||
media_root = posixpath.join(IMAGE_BASE, args.prefix)
|
||||
types = args.types or read_types(args.server, args.menu)
|
||||
|
||||
if args.verify:
|
||||
print("Verifying %d media views under %s" % (len(types), media_root))
|
||||
return 1 if verify(args.server, media_root, types) else 0
|
||||
|
||||
base_apps = list_dir(args.server, posixpath.join(base_dir, "Deploy", "Applications"))
|
||||
if not base_apps:
|
||||
sys.exit("ERROR: %s/Deploy/Applications is empty or unreadable" % base_dir)
|
||||
|
||||
print("base image : %s" % base_dir)
|
||||
print("media root : %s" % media_root)
|
||||
print("types : %d from %s" % (len(types), "--types" if args.types else args.menu))
|
||||
print("base apps : %s" % ", ".join(base_apps))
|
||||
print()
|
||||
|
||||
total_links = 0
|
||||
for pctype in types:
|
||||
media_dir = posixpath.join(media_root, pctype)
|
||||
overlay_entries = list_dir(args.server, posixpath.join(PCTYPE_OVERLAY, pctype))
|
||||
links = plan_for_type(base_dir, media_dir, pctype, base_apps, overlay_entries)
|
||||
total_links += len(links)
|
||||
print("%-32s %2d links%s" % (
|
||||
pctype, len(links),
|
||||
" overlay: " + ", ".join(overlay_entries) if overlay_entries else ""))
|
||||
run_root(args.server, build_script(media_dir, links), args.apply)
|
||||
|
||||
print()
|
||||
if args.prune:
|
||||
existing = list_dir(args.server, media_root)
|
||||
stale = [n for n in existing if n not in types and not n.endswith((".new", ".old"))]
|
||||
for name in stale:
|
||||
print("prune %s" % posixpath.join(media_root, name))
|
||||
run_root(args.server, "rm -rf %s\n" % sh_quote(posixpath.join(media_root, name)),
|
||||
args.apply)
|
||||
if not stale:
|
||||
print("prune: nothing stale")
|
||||
print()
|
||||
|
||||
if not args.apply:
|
||||
print("DRY RUN: %d links across %d types. Re-run with --apply to build."
|
||||
% (total_links, len(types)))
|
||||
return 0
|
||||
|
||||
print("Built %d links across %d types. Verifying." % (total_links, len(types)))
|
||||
return 1 if verify(args.server, media_root, types) else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,28 +0,0 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# build-proxmox-iso.sh - UNSUPPORTED, kept only to say what to use instead.
|
||||
#
|
||||
# This script built a second, hand-maintained copy of the installer
|
||||
# (its own late-commands and first-boot script). It drifted from
|
||||
# autoinstall/user-data: it kept the dead-mirror apt block but dropped the
|
||||
# step that writes the real archive sources back, installed the debs with
|
||||
# "dpkg -i ... || true" instead of the checked offline-apt-install.sh path,
|
||||
# had no site vars (so the two-NIC playbook always stopped), and no
|
||||
# first-boot failure handling. Rather than keep two installers in step,
|
||||
# there is one: build-usb.sh. Its image mode needs no root and gives a disk
|
||||
# image a Proxmox VM boots directly:
|
||||
#
|
||||
# scripts/build-usb.sh --site-vars site-vars.yml --image pxe-usb.img auto \
|
||||
# ubuntu-24.04.x-live-server-amd64.iso
|
||||
# # on the Proxmox host:
|
||||
# qm importdisk <vmid> pxe-usb.img <storage>
|
||||
# # attach it as a second disk (or USB), boot order: system disk, then it
|
||||
#
|
||||
# The VM needs two NICs (imaging bridge with no other DHCP server, and the
|
||||
# corp/GE bridge); put their MACs in site-vars.yml. tests/vm/ does the same
|
||||
# with libvirt.
|
||||
|
||||
echo "build-proxmox-iso.sh is no longer supported (it drifted from autoinstall/user-data)." >&2
|
||||
echo "Use: scripts/build-usb.sh --site-vars FILE --image pxe-usb.img auto UBUNTU.iso" >&2
|
||||
echo "then on Proxmox: qm importdisk <vmid> pxe-usb.img <storage> (see the header of this script)." >&2
|
||||
exit 1
|
||||
@@ -30,10 +30,9 @@
|
||||
# --skip-bios do not copy bios-staging/
|
||||
# --skip-drivers do not copy drivers-staging/
|
||||
# --skip-boot-tools do not copy boot-tools/
|
||||
# --skip-site-content do not copy site-content/ (Office, Adobe, certs,
|
||||
# Engineering Layer...: scripts/capture-site-content.sh).
|
||||
# Without it a new server cannot build standard or
|
||||
# engineer until that content is copied over by hand.
|
||||
# --skip-site-content do not copy site-content/ even if present (optional
|
||||
# content captured from a running server with
|
||||
# scripts/capture-site-content.sh).
|
||||
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml
|
||||
# (per-site values; see playbook/site-vars.example.yml)
|
||||
# --no-site-vars build without site vars (defaults; the first boot
|
||||
@@ -321,11 +320,12 @@ fi
|
||||
# never split: a file FAT32 cannot hold refuses the build instead.
|
||||
if [ -n "$SITE_CONTENT_DIR" ]; then
|
||||
if [ ! -d "$SITE_CONTENT_DIR" ]; then
|
||||
echo "ERROR: no site-content/ in $PAYLOAD_ROOT. Run scripts/capture-site-content.sh --apply"
|
||||
echo " (or pass --skip-site-content: the new server then lacks Office, Adobe, certs"
|
||||
echo " and the Engineering Layer until they are copied by hand)."
|
||||
exit 1
|
||||
# Optional: a site with no other PXE server has none.
|
||||
echo "No site-content/ in $PAYLOAD_ROOT (optional): building without it"
|
||||
SITE_CONTENT_DIR=""
|
||||
fi
|
||||
fi
|
||||
if [ -n "$SITE_CONTENT_DIR" ]; then
|
||||
if [ ! -f "$SITE_CONTENT_DIR/SHA256SUMS" ]; then
|
||||
echo "ERROR: site-content/SHA256SUMS missing. Re-run scripts/capture-site-content.sh --apply"
|
||||
exit 1
|
||||
@@ -683,8 +683,7 @@ echo " Copied playbook/"
|
||||
if [ -n "$SITE_VARS" ]; then
|
||||
echo " Copied $SITE_VARS as playbook/site-vars.yml"
|
||||
else
|
||||
echo " WARNING: --no-site-vars: the first-boot playbook stops until"
|
||||
echo " pxe_nic_mac / corp_nic_mac are set on the server."
|
||||
echo " --no-site-vars: defaults; the first boot picks the imaging NIC by its link"
|
||||
fi
|
||||
# The unattend linter the playbook runs before it replaces the answer files
|
||||
# ({{ usb_mount }}/../scripts/lint-unattend.py).
|
||||
@@ -860,11 +859,12 @@ if [ "$IMAGE_MODE" -eq 1 ]; then
|
||||
echo " or attach it to a VM as a USB disk (see tests/vm/README.md)."
|
||||
else
|
||||
echo "Next steps:"
|
||||
echo " 1. Cable both NICs: the imaging NIC (pxe_nic_mac) to the isolated"
|
||||
echo " imaging switch, the corp NIC (corp_nic_mac) to the GE network"
|
||||
echo " 1. Plug in ONLY the imaging switch (GE unplugged), unless site-vars"
|
||||
echo " pins both MACs"
|
||||
echo " 2. Insert USB into the target machine, boot from it (F12 / boot menu)"
|
||||
echo " 3. Ubuntu auto-installs; on first boot the playbook pins the NICs by"
|
||||
echo " MAC (imaging = static 172.16.9.1, corp = DHCP) and sets up PXE"
|
||||
echo " 3. Ubuntu auto-installs; on first boot the playbook picks and pins the"
|
||||
echo " imaging NIC (static 172.16.9.1) and sets up PXE. Then plug GE into"
|
||||
echo " any other port"
|
||||
echo " 4. Check /var/log/first-boot.log; a failure shows a banner at login"
|
||||
echo " and on the console, and first boot retries at the next reboot"
|
||||
fi
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
#!/bin/bash
|
||||
# deploy-bios.sh - Deploy BIOS update files to a running PXE server
|
||||
# Copies Flash64W.exe, BIOS binaries, models.txt, and check-bios.cmd
|
||||
#
|
||||
# Usage: ./deploy-bios.sh [server-ip]
|
||||
# Default server: 172.16.9.1
|
||||
|
||||
set -e
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")"/.. && pwd)"
|
||||
PXE_SERVER="${1:-172.16.9.1}"
|
||||
PXE_USER="pxe"
|
||||
PXE_PASS="pxe"
|
||||
REMOTE_DIR="/srv/samba/enrollment/BIOS"
|
||||
BIOS_DIR="$REPO_ROOT/bios-staging"
|
||||
MANIFEST="$REPO_ROOT/playbook/shopfloor-setup/BIOS/models.txt"
|
||||
CHECK_SCRIPT="$REPO_ROOT/playbook/shopfloor-setup/BIOS/check-bios.cmd"
|
||||
|
||||
SSH="sshpass -p $PXE_PASS ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 $PXE_USER@$PXE_SERVER"
|
||||
SCP="sshpass -p $PXE_PASS scp -o StrictHostKeyChecking=no -o ConnectTimeout=10"
|
||||
|
||||
# Verify sources exist
|
||||
if [ ! -d "$BIOS_DIR" ] || [ -z "$(ls -A "$BIOS_DIR" 2>/dev/null)" ]; then
|
||||
echo "ERROR: bios-staging/ is empty or missing. Run scripts/capture-site-content.sh --apply first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "$MANIFEST" ]; then
|
||||
echo "ERROR: playbook/shopfloor-setup/BIOS/models.txt not found."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Deploying BIOS files to $PXE_SERVER..."
|
||||
|
||||
# Create remote directory
|
||||
$SSH "sudo mkdir -p '$REMOTE_DIR' && sudo chown $PXE_USER:$PXE_USER '$REMOTE_DIR'"
|
||||
|
||||
# Copy check-bios.cmd and models.txt
|
||||
echo " Copying check-bios.cmd + models.txt..."
|
||||
$SCP "$CHECK_SCRIPT" "$MANIFEST" "$PXE_USER@$PXE_SERVER:$REMOTE_DIR/"
|
||||
|
||||
# Copy BIOS binaries
|
||||
COUNT=$(find "$BIOS_DIR" -name '*.exe' | wc -l)
|
||||
SIZE=$(du -sh "$BIOS_DIR" | cut -f1)
|
||||
echo " Copying $COUNT BIOS binaries ($SIZE)..."
|
||||
$SCP "$BIOS_DIR"/*.exe "$PXE_USER@$PXE_SERVER:$REMOTE_DIR/"
|
||||
|
||||
# Verify
|
||||
REMOTE_COUNT=$($SSH "find '$REMOTE_DIR' -name '*.exe' | wc -l")
|
||||
echo "Done: $REMOTE_COUNT files on $PXE_SERVER:$REMOTE_DIR"
|
||||
@@ -848,7 +848,7 @@ def main():
|
||||
print(f"\n Add these lines to playbook/shopfloor-setup/BIOS/models.txt:")
|
||||
for line in models_txt:
|
||||
print(f" {line}")
|
||||
print(f" Then re-deploy via scripts/deploy-bios.sh.")
|
||||
print(f" Then upload them through the webapp (Updates > Dell drivers and BIOS) or re-run the playbook.")
|
||||
|
||||
# --- Save manifest ---
|
||||
completed, skipped, errors = counters["completed"], counters["skipped"], counters["errors"]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,260 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Pretend to be a Telesis marking controller, so MarkTelesis will talk to us.
|
||||
|
||||
WHY THIS EXISTS
|
||||
We need to know exactly what GE's MarkTelesis.exe puts on the wire before a
|
||||
translator can impersonate a Telesis controller in front of it. A passive
|
||||
tap is useless here: MarkTelesis waits for a reply to every command and
|
||||
gives up after the first timeout, so a listener that never answers captures
|
||||
one frame and nothing else.
|
||||
|
||||
So this ANSWERS. It decodes each frame, logs it, and sends back the ACK
|
||||
shape MarkTelesis is parsing for. That keeps the conversation going long
|
||||
enough to record a whole marking job.
|
||||
|
||||
It is also the first half of the real translator - the same decode, minus
|
||||
the part where the other side becomes a SIC e10.
|
||||
|
||||
THE PROTOCOL, from disassembly of MarkTelesis.exe (2026-08-27)
|
||||
request (host -> controller):
|
||||
EOT SOH <TYPE> STX <data> ETX <BCC:3 ascii digits> CR
|
||||
reply (controller -> host):
|
||||
SOH <TYPE> ACK STX <data> ETX <BCC:3 ascii digits> CR
|
||||
|
||||
BCC = (TYPE + sum of data bytes) & 0xFF, as three ASCII decimal digits.
|
||||
|
||||
NOTE THE MISSING STATION ID. Backup470 framed requests as
|
||||
EOT SOH <TYPE> <2-digit station> STX ... - MarkTelesis does NOT send those
|
||||
two digits. Same protocol family, different dialect. The frame builder at
|
||||
0x401cf0 pushes exactly EOT, SOH, TYPE, STX, data, ETX, BCC, CR and nothing
|
||||
else.
|
||||
|
||||
COMMAND TYPES, all confirmed at their call sites
|
||||
'P' pattern load, data is the pattern name
|
||||
'1'..'9' numbered line send - the 2Line/3Line/9Line dialect. The TYPE byte
|
||||
IS the line number: the loop at 0x4090b0 does `add bl, 0x31`.
|
||||
'V' set field, the TMC400 dialect. Matches Message Type"="V" in
|
||||
WJPRT.reg.
|
||||
'O' online, empty data
|
||||
'R' empty data, believed to be the mark/run trigger. INFERRED, not yet
|
||||
traced to a UI action - one of the things this capture settles.
|
||||
'D' pattern download, data is "<name>;<6-digit size>;128". Same write
|
||||
header as Backup470.
|
||||
|
||||
FORCING THE ERROR PATHS
|
||||
Happy-path cycles teach us almost nothing we cannot already read out of the
|
||||
binary. What we actually need is how MarkTelesis behaves when things go
|
||||
wrong, because the translator has to reproduce it:
|
||||
|
||||
--nak TYPE[,TYPE...] answer these types with NAK instead of ACK
|
||||
--drop TYPE[,TYPE...] do not answer these at all, to see the retry
|
||||
and timeout cadence
|
||||
--delay SECONDS stall before every reply
|
||||
|
||||
Run it once clean, then again with --nak P (pattern not found) and once
|
||||
with --drop R, and the resulting logs cover the cases that would otherwise
|
||||
only show up at a bay with an operator standing there.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
import termios
|
||||
import time
|
||||
|
||||
EOT = 0x04
|
||||
SOH = 0x01
|
||||
STX = 0x02
|
||||
ETX = 0x03
|
||||
ACK = 0x06
|
||||
NAK = 0x15
|
||||
CR = 0x0D
|
||||
|
||||
TYPE_MEANING = {
|
||||
"P": "pattern load",
|
||||
"V": "set field (TMC400 dialect)",
|
||||
"O": "online",
|
||||
"R": "run / mark (inferred)",
|
||||
"D": "pattern download",
|
||||
"I": "status request",
|
||||
}
|
||||
|
||||
|
||||
def describe(type_char):
|
||||
if type_char in TYPE_MEANING:
|
||||
return TYPE_MEANING[type_char]
|
||||
if type_char.isdigit() and type_char != "0":
|
||||
return "line %s text (2Line/3Line/9Line dialect)" % type_char
|
||||
return "unknown"
|
||||
|
||||
|
||||
def bcc(type_char, data):
|
||||
"""(TYPE + sum(data)) & 0xFF, as three ASCII decimal digits."""
|
||||
total = ord(type_char)
|
||||
for byte in data:
|
||||
total += byte
|
||||
return b"%03d" % (total & 0xFF)
|
||||
|
||||
|
||||
class Line:
|
||||
"""One serial endpoint, raw, no flow control."""
|
||||
|
||||
def __init__(self, device, baud=termios.B9600):
|
||||
self.fd = os.open(device, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
|
||||
attrs = termios.tcgetattr(self.fd)
|
||||
cc = list(attrs[6])
|
||||
cc[termios.VMIN] = 0
|
||||
cc[termios.VTIME] = 0
|
||||
termios.tcsetattr(self.fd, termios.TCSANOW,
|
||||
[0, 0,
|
||||
termios.CS8 | termios.CREAD | termios.CLOCAL,
|
||||
0, baud, baud, cc])
|
||||
termios.tcflush(self.fd, termios.TCIOFLUSH)
|
||||
|
||||
def read_byte(self, timeout=None):
|
||||
deadline = None if timeout is None else time.monotonic() + timeout
|
||||
while True:
|
||||
try:
|
||||
chunk = os.read(self.fd, 1)
|
||||
except BlockingIOError:
|
||||
chunk = b""
|
||||
except OSError:
|
||||
return None
|
||||
if chunk:
|
||||
return chunk[0]
|
||||
if deadline is not None and time.monotonic() >= deadline:
|
||||
return None
|
||||
time.sleep(0.002)
|
||||
|
||||
def write(self, data):
|
||||
offset = 0
|
||||
while offset < len(data):
|
||||
offset += os.write(self.fd, data[offset:])
|
||||
|
||||
def close(self):
|
||||
os.close(self.fd)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Answer MarkTelesis as if we were a Telesis controller.")
|
||||
parser.add_argument("-d", "--device", default="/dev/pts/4",
|
||||
help="serial endpoint (the win11 VM's pty)")
|
||||
parser.add_argument("-l", "--log", help="also append the transcript here")
|
||||
parser.add_argument("--nak", default="",
|
||||
help="comma-separated TYPEs to answer with NAK")
|
||||
parser.add_argument("--drop", default="",
|
||||
help="comma-separated TYPEs to leave unanswered")
|
||||
parser.add_argument("--delay", type=float, default=0.0,
|
||||
help="seconds to stall before each reply")
|
||||
parser.add_argument("--status", default="0A;000",
|
||||
help="data to return for a Type I status request."
|
||||
" 0A;000 is a real TMC470 idle reply.")
|
||||
args = parser.parse_args()
|
||||
|
||||
nak_types = set(t.strip() for t in args.nak.split(",") if t.strip())
|
||||
drop_types = set(t.strip() for t in args.drop.split(",") if t.strip())
|
||||
|
||||
log_handle = open(args.log, "a") if args.log else None
|
||||
|
||||
def emit(text):
|
||||
stamp = time.strftime("%H:%M:%S")
|
||||
line = "%s %s" % (stamp, text)
|
||||
print(line, flush=True)
|
||||
if log_handle:
|
||||
log_handle.write(line + "\n")
|
||||
log_handle.flush()
|
||||
|
||||
line = Line(args.device)
|
||||
emit("listening on %s as a Telesis controller" % args.device)
|
||||
if nak_types:
|
||||
emit("will NAK: %s" % ", ".join(sorted(nak_types)))
|
||||
if drop_types:
|
||||
emit("will DROP (no reply): %s" % ", ".join(sorted(drop_types)))
|
||||
emit("waiting for MarkTelesis...")
|
||||
|
||||
frames = 0
|
||||
try:
|
||||
while True:
|
||||
# requests open with EOT. anything else is noise worth seeing.
|
||||
byte = line.read_byte(timeout=None)
|
||||
if byte is None:
|
||||
continue
|
||||
if byte != EOT:
|
||||
emit(" stray byte before EOT: 0x%02X" % byte)
|
||||
continue
|
||||
|
||||
soh = line.read_byte(timeout=2)
|
||||
if soh != SOH:
|
||||
emit(" expected SOH after EOT, got %s"
|
||||
% ("nothing" if soh is None else "0x%02X" % soh))
|
||||
continue
|
||||
|
||||
type_byte = line.read_byte(timeout=2)
|
||||
if type_byte is None:
|
||||
emit(" timed out waiting for TYPE")
|
||||
continue
|
||||
type_char = chr(type_byte)
|
||||
|
||||
stx = line.read_byte(timeout=2)
|
||||
if stx != STX:
|
||||
emit(" expected STX, got %s"
|
||||
% ("nothing" if stx is None else "0x%02X" % stx))
|
||||
continue
|
||||
|
||||
data = bytearray()
|
||||
while True:
|
||||
nxt = line.read_byte(timeout=2)
|
||||
if nxt is None:
|
||||
emit(" timed out reading data")
|
||||
break
|
||||
if nxt == ETX:
|
||||
break
|
||||
data.append(nxt)
|
||||
|
||||
checksum = bytearray()
|
||||
for _ in range(3):
|
||||
nxt = line.read_byte(timeout=2)
|
||||
if nxt is None:
|
||||
break
|
||||
checksum.append(nxt)
|
||||
line.read_byte(timeout=1) # trailing CR
|
||||
|
||||
frames += 1
|
||||
expected = bcc(type_char, data)
|
||||
ok = "ok" if bytes(checksum) == expected else (
|
||||
"MISMATCH, we computed %s" % expected.decode())
|
||||
emit("RX #%d TYPE '%s' (%s) data=%r bcc=%s [%s]"
|
||||
% (frames, type_char, describe(type_char), bytes(data),
|
||||
bytes(checksum).decode("ascii", "replace"), ok))
|
||||
|
||||
if type_char in drop_types:
|
||||
emit(" -> dropping, no reply (forcing a timeout)")
|
||||
continue
|
||||
|
||||
if args.delay:
|
||||
time.sleep(args.delay)
|
||||
|
||||
if type_char in nak_types:
|
||||
reply = bytes([SOH, ord(type_char), NAK, STX, ETX]) \
|
||||
+ bcc(type_char, b"") + bytes([CR])
|
||||
emit(" -> NAK")
|
||||
else:
|
||||
payload = b""
|
||||
if type_char == "I":
|
||||
payload = args.status.encode("ascii")
|
||||
reply = bytes([SOH, ord(type_char), ACK, STX]) + payload \
|
||||
+ bytes([ETX]) + bcc(type_char, payload) + bytes([CR])
|
||||
emit(" -> ACK%s" % (" data=%r" % payload if payload else ""))
|
||||
line.write(reply)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
emit("stopped after %d frames" % frames)
|
||||
finally:
|
||||
line.close()
|
||||
if log_handle:
|
||||
log_handle.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,279 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build SIC e10 marking files with DECLARED variables.
|
||||
|
||||
WHY THIS EXISTS
|
||||
The e10's own protocol cannot create a variable. You can build a marking
|
||||
file over the wire with NEWFILE / INSERTTEXTLINE / SAVEFILE, and you can put
|
||||
"#(V, V01)" in a text line, but that is only a REFERENCE. The variable has
|
||||
to be DECLARED, and there is no INSERTVAR command. Proven the hard way: a
|
||||
file built entirely over the protocol came back with a variable count of
|
||||
zero, and SETVAR answered "variable not found".
|
||||
|
||||
Global variables are numbered 1-10, so #(V, NAME) cannot reach one either.
|
||||
|
||||
That left authoring dependent on SIC's GUI, which would have made migrating
|
||||
a fleet of Telesis patterns a hand operation. This does it from a template
|
||||
instead.
|
||||
|
||||
THE FILE FORMAT, reversed from 28 real files plus one we built over the wire
|
||||
|
||||
header
|
||||
0x00 "MARK_V5.0\\0"
|
||||
0x0A variable count <- the byte that matters. zero here means
|
||||
SETVAR will always fail.
|
||||
0x0B record count
|
||||
0x0C two bytes, purpose unknown, carried through unchanged
|
||||
0x0E offset table: 16-bit big-endian file offsets, one per record.
|
||||
The number of entries is implied - the first entry points at the
|
||||
first record, so entries = (first_offset - 0x0E) / 2.
|
||||
|
||||
variable declaration record
|
||||
[type][len-6][class][00][name, NUL-padded]
|
||||
type 0x02 and 0x04 both seen; class 0x50 ('P') and 0x40 ('@') both seen.
|
||||
Carried through unchanged except for the name, which is rewritten
|
||||
inside its fixed-width padded field so the record length never moves.
|
||||
|
||||
text line record
|
||||
[X(2)][Y(2)][Z(2)][W(2)][H(2)][2 bytes][textlen+13 (2)][2 bytes]
|
||||
[0x83][font name, 12 bytes NUL-padded][text]
|
||||
All coordinates are big-endian tenths of a mm. The field at offset 12
|
||||
is len(text) + 13 in every file checked, which is what makes rewriting
|
||||
the text safe.
|
||||
|
||||
trailer records
|
||||
Short fixed records at the end. Carried through untouched.
|
||||
|
||||
WHAT THIS DOES
|
||||
Takes a known-good file as a template, rewrites the variable names and the
|
||||
text lines, then rebuilds the offset table so the records still line up.
|
||||
Everything it does not understand is copied through byte for byte, which is
|
||||
the only honest way to edit a format nobody has documented.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
MAGIC = b"MARK_V5.0\x00"
|
||||
TEXT_HEADER_LEN = 29 # geometry + marker byte + 12-byte font field
|
||||
TEXT_LEN_OFFSET = 12 # where len(text)+13 lives inside a text record
|
||||
TEXT_LEN_BIAS = 13
|
||||
FONT_OFFSET = 17
|
||||
FONT_FIELD = 12
|
||||
NAME_OFFSET = 4 # where a variable's name starts in its record
|
||||
|
||||
|
||||
def be16(data, index):
|
||||
return (data[index] << 8) | data[index + 1]
|
||||
|
||||
|
||||
def pack_be16(value):
|
||||
return bytes([(value >> 8) & 0xFF, value & 0xFF])
|
||||
|
||||
|
||||
class Pattern:
|
||||
"""A parsed MARK_V5.0 file: a header, a list of records, nothing hidden."""
|
||||
|
||||
def __init__(self, raw):
|
||||
if not raw.startswith(MAGIC):
|
||||
raise ValueError("not a MARK_V5.0 file")
|
||||
self.raw = raw
|
||||
self.varcount = raw[0x0A]
|
||||
self.reccount = raw[0x0B]
|
||||
self.unknown = raw[0x0C:0x0E]
|
||||
|
||||
first = be16(raw, 0x0E)
|
||||
entries = (first - 0x0E) // 2
|
||||
if entries < 1:
|
||||
raise ValueError("implausible offset table")
|
||||
offsets = [be16(raw, 0x0E + 2 * i) for i in range(entries)]
|
||||
|
||||
self.records = []
|
||||
for index, start in enumerate(offsets):
|
||||
nxt = offsets[index + 1] if index + 1 < len(offsets) else len(raw)
|
||||
if nxt <= start:
|
||||
nxt = len(raw)
|
||||
self.records.append(bytearray(raw[start:nxt]))
|
||||
|
||||
# ------------------------------------------------------------ inspection
|
||||
|
||||
def is_variable(self, record):
|
||||
"""A declaration record, identified by its own length arithmetic.
|
||||
|
||||
Do NOT match on byte 0 being 0x02 or 0x04 - that is the name LENGTH,
|
||||
which changes the moment you rename anything, and matching on it makes
|
||||
a renamed variable invisible to the next edit.
|
||||
"""
|
||||
return (len(record) >= NAME_OFFSET + 1
|
||||
and record[2] in (0x40, 0x50)
|
||||
and record[3] == 0x00
|
||||
and len(record) == NAME_OFFSET + record[0] + record[1])
|
||||
|
||||
def is_text(self, record):
|
||||
return (len(record) > FONT_OFFSET
|
||||
and record[FONT_OFFSET - 1] == 0x83)
|
||||
|
||||
def variable_names(self):
|
||||
out = []
|
||||
for rec in self.records:
|
||||
if self.is_variable(rec):
|
||||
out.append(bytes(rec[NAME_OFFSET:]).split(b"\x00")[0]
|
||||
.decode("ascii", "replace"))
|
||||
return out
|
||||
|
||||
def variable_sizes(self):
|
||||
"""Value buffer size per variable - the silent truncation limit."""
|
||||
return [r[1] for r in self.records if self.is_variable(r)]
|
||||
|
||||
def texts(self):
|
||||
out = []
|
||||
for rec in self.records:
|
||||
if self.is_text(rec):
|
||||
out.append(bytes(rec[TEXT_HEADER_LEN:])
|
||||
.decode("ascii", "replace"))
|
||||
return out
|
||||
|
||||
# -------------------------------------------------------------- editing
|
||||
|
||||
def rename_variable(self, index, new_name, value_size=None):
|
||||
"""Rewrite a declaration's name.
|
||||
|
||||
THE RECORD LAYOUT, and the two length bytes both matter:
|
||||
|
||||
[name_len][value_size][class][00][name][value buffer]
|
||||
|
||||
SN 02 0a 50 00 "SN" 16 = 4 + 2 + 10
|
||||
PN 02 0e 50 00 "PN" 20 = 4 + 2 + 14
|
||||
OF 02 0f 40 00 "OF" 21 = 4 + 2 + 15
|
||||
TEMP 04 02 40 00 "TEMP" 10 = 4 + 4 + 2
|
||||
|
||||
Byte 0 is the length of the NAME. Byte 1 is the size of the VALUE
|
||||
buffer that follows it, which is the longest string SETVAR can put in
|
||||
this variable.
|
||||
|
||||
Both were found the hard way. Writing a three-character name while byte
|
||||
0 still said 2 produced a file that pushed cleanly, read back
|
||||
byte-identical and loaded without complaint - and then SETVAR answered
|
||||
"variable not found", because the controller had parsed a name of the
|
||||
wrong length. A silent, entirely plausible-looking failure.
|
||||
"""
|
||||
found = [i for i, r in enumerate(self.records) if self.is_variable(r)]
|
||||
if index >= len(found):
|
||||
raise ValueError("no variable %d in this template" % index)
|
||||
position = found[index]
|
||||
rec = self.records[position]
|
||||
encoded = new_name.encode("ascii")
|
||||
if not 1 <= len(encoded) <= 255:
|
||||
raise ValueError("implausible variable name %r" % new_name)
|
||||
|
||||
size = rec[1] if value_size is None else value_size
|
||||
rebuilt = bytearray(rec[:NAME_OFFSET])
|
||||
rebuilt[0] = len(encoded)
|
||||
rebuilt[1] = size
|
||||
rebuilt += encoded
|
||||
rebuilt += b"\x00" * size
|
||||
self.records[position] = rebuilt
|
||||
|
||||
def set_text(self, index, text):
|
||||
"""Replace a text line's content and fix its internal length field."""
|
||||
found = [i for i, r in enumerate(self.records) if self.is_text(r)]
|
||||
if index >= len(found):
|
||||
raise ValueError("no text line %d in this template" % index)
|
||||
rec = self.records[found[index]]
|
||||
encoded = text.encode("ascii")
|
||||
rec[TEXT_HEADER_LEN:] = encoded
|
||||
rec[TEXT_LEN_OFFSET:TEXT_LEN_OFFSET + 2] = pack_be16(
|
||||
len(encoded) + TEXT_LEN_BIAS)
|
||||
|
||||
def set_position(self, index, x=None, y=None):
|
||||
"""Move a text line. Coordinates are tenths of a mm."""
|
||||
found = [i for i, r in enumerate(self.records) if self.is_text(r)]
|
||||
rec = self.records[found[index]]
|
||||
if x is not None:
|
||||
rec[0:2] = pack_be16(x)
|
||||
if y is not None:
|
||||
rec[2:4] = pack_be16(y)
|
||||
|
||||
# ------------------------------------------------------------- assembly
|
||||
|
||||
def build(self):
|
||||
"""Reassemble, recomputing every offset from the record lengths."""
|
||||
entries = len(self.records)
|
||||
table_start = 0x0E
|
||||
body_start = table_start + 2 * entries
|
||||
|
||||
offsets, cursor = [], body_start
|
||||
for rec in self.records:
|
||||
offsets.append(cursor)
|
||||
cursor += len(rec)
|
||||
|
||||
out = bytearray(MAGIC)
|
||||
out.append(self.varcount)
|
||||
out.append(self.reccount)
|
||||
out += self.unknown
|
||||
for off in offsets:
|
||||
out += pack_be16(off)
|
||||
for rec in self.records:
|
||||
out += rec
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def describe(pattern, label):
|
||||
print("%s" % label)
|
||||
sizes = pattern.variable_sizes()
|
||||
print(" variables declared : %d %s"
|
||||
% (pattern.varcount,
|
||||
", ".join("%s(max %d)" % (n, s)
|
||||
for n, s in zip(pattern.variable_names(), sizes))
|
||||
or "-"))
|
||||
print(" records : %d" % len(pattern.records))
|
||||
for text in pattern.texts():
|
||||
print(" text line : %r" % text)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Build an e10 marking file with declared variables.")
|
||||
parser.add_argument("--template", required=True,
|
||||
help="a known-good marking file with at least as many"
|
||||
" variables as you need. NEW.marking has 2.")
|
||||
parser.add_argument("--var", action="append", default=[],
|
||||
help="new name for the next variable. repeatable.")
|
||||
parser.add_argument("--var-size", type=int, default=127,
|
||||
help="value buffer size for each variable, 1-127."
|
||||
" THE CONTROLLER TRUNCATES SILENTLY to this and"
|
||||
" still answers SETVAR ok, so the default is the"
|
||||
" protocol maximum. Only lower it deliberately.")
|
||||
parser.add_argument("--line", action="append", default=[],
|
||||
help="text for the next line. repeatable."
|
||||
" Use #(V, NAME) to reference a variable.")
|
||||
parser.add_argument("-o", "--out", help="write the file here")
|
||||
parser.add_argument("--show", action="store_true",
|
||||
help="just describe the template and exit")
|
||||
args = parser.parse_args()
|
||||
|
||||
pattern = Pattern(open(args.template, "rb").read())
|
||||
describe(pattern, "TEMPLATE %s" % args.template)
|
||||
if args.show:
|
||||
return
|
||||
|
||||
if not 1 <= args.var_size <= 127:
|
||||
sys.exit("--var-size must be 1..127 (FILE SET VAR carries 1-127 bytes)")
|
||||
for index, name in enumerate(args.var):
|
||||
pattern.rename_variable(index, name, value_size=args.var_size)
|
||||
for index, text in enumerate(args.line):
|
||||
pattern.set_text(index, text)
|
||||
|
||||
built = Pattern(pattern.build())
|
||||
print()
|
||||
describe(built, "BUILT")
|
||||
|
||||
if args.out:
|
||||
with open(args.out, "wb") as handle:
|
||||
handle.write(pattern.build())
|
||||
print()
|
||||
print("wrote %s (%d bytes)" % (args.out, len(pattern.build())))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,153 +0,0 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# mirror-from-gold.sh - Byte-identical mirror of /srv/samba/ from an existing
|
||||
# PXE server (GOLD) onto a freshly-installed PXE server.
|
||||
#
|
||||
# Run this ON THE NEW PXE SERVER, pointing at the GOLD server's IP.
|
||||
# It pulls every Samba share (winpeapps, enrollment, blancco-reports,
|
||||
# clonezilla) wholesale so the new box matches GOLD regardless of whether
|
||||
# content lives in the flat layout or the new taxonomy layout.
|
||||
#
|
||||
# Usage:
|
||||
# sudo ./mirror-from-gold.sh <GOLD_IP> [options]
|
||||
#
|
||||
# Options:
|
||||
# --skip-drivers Do not mirror Out-of-box Drivers trees (saves ~178G).
|
||||
# --skip-dell10 Do not mirror Dell_10 drivers (saves ~179G).
|
||||
# --skip-latitude Do not mirror Latitude drivers (saves ~48G).
|
||||
# --skip-os Do not mirror shared Operating Systems (saves ~22G).
|
||||
# --skip-clonezilla Do not mirror clonezilla backup images (can be huge).
|
||||
# --skip-reports Do not mirror blancco-reports history.
|
||||
# --dry-run Show what would transfer without doing it.
|
||||
#
|
||||
# Prereqs:
|
||||
# - GOLD's pxe user accepts the SSH key this script generates.
|
||||
# - GOLD's filesystem is world-readable for the paths involved
|
||||
# (it is, by default).
|
||||
|
||||
# Note: not using `set -e`. rsync legitimately exits non-zero (e.g. 23
|
||||
# "some files/attrs were not transferred") when source dirs have files
|
||||
# the rsync user cannot read (the OpenText W10shortcuts dir on GOLD is
|
||||
# pxe-upload-group-only). We log and continue instead of aborting.
|
||||
set -uo pipefail
|
||||
|
||||
GOLD="${1:-}"
|
||||
shift || true
|
||||
|
||||
if [ -z "$GOLD" ]; then
|
||||
echo "Usage: $0 <GOLD_IP> [--skip-drivers|--skip-dell10|--skip-latitude|--skip-os|--dry-run]"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
echo "ERROR: must run as root (sudo)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SKIP_DRIVERS=0
|
||||
SKIP_DELL10=0
|
||||
SKIP_LATITUDE=0
|
||||
SKIP_OS=0
|
||||
SKIP_CLONEZILLA=0
|
||||
SKIP_REPORTS=0
|
||||
DRY_RUN=""
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--skip-drivers) SKIP_DRIVERS=1 ;;
|
||||
--skip-dell10) SKIP_DELL10=1 ;;
|
||||
--skip-latitude) SKIP_LATITUDE=1 ;;
|
||||
--skip-os) SKIP_OS=1 ;;
|
||||
--skip-clonezilla) SKIP_CLONEZILLA=1 ;;
|
||||
--skip-reports) SKIP_REPORTS=1 ;;
|
||||
--dry-run) DRY_RUN="--dry-run" ;;
|
||||
*) echo "Unknown option: $1"; exit 1 ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
KEY=/root/.ssh/pxe-mirror-key
|
||||
if [ ! -f "$KEY" ]; then
|
||||
echo "[setup] Generating mirror SSH key at $KEY"
|
||||
mkdir -p /root/.ssh && chmod 700 /root/.ssh
|
||||
ssh-keygen -t ed25519 -N '' -f "$KEY" -q
|
||||
echo
|
||||
echo "Copy the following public key into pxe@$GOLD's ~/.ssh/authorized_keys"
|
||||
echo "(easiest: ssh pxe@$GOLD 'mkdir -p ~/.ssh && chmod 700 ~/.ssh' and then"
|
||||
echo " scp '$KEY.pub' pxe@$GOLD:~/.ssh/authorized_keys, then chmod 600 there)"
|
||||
echo
|
||||
cat "$KEY.pub"
|
||||
echo
|
||||
read -rp "Press enter once the key is installed on GOLD..."
|
||||
fi
|
||||
|
||||
RSH="ssh -i $KEY -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
|
||||
|
||||
# Quick connectivity check
|
||||
echo "[check] Testing key-based SSH to pxe@$GOLD..."
|
||||
$RSH "pxe@$GOLD" "whoami" >/dev/null || {
|
||||
echo "ERROR: SSH to pxe@$GOLD failed. Install $KEY.pub on GOLD first."
|
||||
exit 1
|
||||
}
|
||||
echo " OK"
|
||||
|
||||
mirror() {
|
||||
local label="$1"
|
||||
local src="$2"
|
||||
local dest="$3"
|
||||
shift 3
|
||||
echo
|
||||
echo "==== $label ===="
|
||||
echo " src: pxe@$GOLD:$src"
|
||||
echo " dest: $dest"
|
||||
mkdir -p "$dest"
|
||||
rsync -aHl --delete --info=progress2 --no-inc-recursive $DRY_RUN "$@" \
|
||||
-e "$RSH" \
|
||||
"pxe@$GOLD:$src" "$dest" || \
|
||||
echo " WARNING: rsync exited rc=$? (likely a permissions issue on source); continuing"
|
||||
}
|
||||
|
||||
# ---------- winpeapps share (all image types + _shared) ----------
|
||||
WINPE_EXCLUDES=()
|
||||
[ "$SKIP_OS" = "1" ] && WINPE_EXCLUDES+=(--exclude='_shared/gea-Operating Systems')
|
||||
[ "$SKIP_DRIVERS" = "1" ] && WINPE_EXCLUDES+=(--exclude='_shared/Out-of-box Drivers' --exclude='Out-of-box Drivers')
|
||||
[ "$SKIP_DELL10" = "1" ] && WINPE_EXCLUDES+=(--exclude='Dell_10')
|
||||
[ "$SKIP_LATITUDE" = "1" ] && WINPE_EXCLUDES+=(--exclude='Latitude')
|
||||
|
||||
mirror "winpeapps (all image types, _shared, tools)" \
|
||||
"/srv/samba/winpeapps/" \
|
||||
"/srv/samba/winpeapps/" \
|
||||
"${WINPE_EXCLUDES[@]}"
|
||||
|
||||
# ---------- enrollment share (flat-layout root + taxonomy subdirs) ----------
|
||||
mirror "enrollment (taxonomy + flat-layout content)" \
|
||||
"/srv/samba/enrollment/" \
|
||||
"/srv/samba/enrollment/"
|
||||
|
||||
# ---------- blancco-reports share (historical XML reports) ----------
|
||||
if [ "$SKIP_REPORTS" = "0" ]; then
|
||||
mirror "blancco-reports (erasure report history)" \
|
||||
"/srv/samba/blancco-reports/" \
|
||||
"/srv/samba/blancco-reports/"
|
||||
fi
|
||||
|
||||
# ---------- clonezilla share (disk backup images, can be very large) ----------
|
||||
if [ "$SKIP_CLONEZILLA" = "0" ]; then
|
||||
mirror "clonezilla (disk backup images)" \
|
||||
"/srv/samba/clonezilla/" \
|
||||
"/srv/samba/clonezilla/"
|
||||
fi
|
||||
|
||||
# Permissions: make sure everything we pulled is readable by the share
|
||||
chown -R root:root /srv/samba/enrollment /srv/samba/winpeapps /srv/samba/blancco-reports /srv/samba/clonezilla 2>/dev/null || true
|
||||
find /srv/samba/enrollment /srv/samba/winpeapps -type d -exec chmod 0755 {} \; 2>/dev/null || true
|
||||
find /srv/samba/enrollment /srv/samba/winpeapps -type f -exec chmod 0644 {} \; 2>/dev/null || true
|
||||
find /srv/samba/enrollment -name '*.ppkg' -exec chmod 0755 {} \; 2>/dev/null || true
|
||||
|
||||
echo
|
||||
echo "============================================"
|
||||
echo "Mirror complete. Verify with:"
|
||||
echo " du -sh /srv/samba/winpeapps/_shared/*"
|
||||
echo " du -sh /srv/samba/enrollment/*"
|
||||
echo " df -h /srv"
|
||||
echo "============================================"
|
||||
@@ -15,8 +15,6 @@ What it runs:
|
||||
lint-unattend.py answer files vs the schema limits. One over-length
|
||||
value invalidates the whole file for its pass and
|
||||
stops the bay at a dialog with nothing configured.
|
||||
build-pctype-media.py resolves the critical paths in every per-PCTYPE
|
||||
--verify media view; a dangling symlink is a dead view.
|
||||
firmware coverage models in the driver catalogues that have no entry
|
||||
in the BIOS models.txt, so their firmware is never
|
||||
offered. This one is advisory - the manifest is a
|
||||
@@ -123,8 +121,6 @@ def main():
|
||||
rc = 0
|
||||
rc |= run("driver catalogue", [sys.executable, "scripts/lint-driver-catalogue.py", "--quiet"])
|
||||
rc |= run("unattend answer files", [sys.executable, "scripts/lint-unattend.py", "--quiet"])
|
||||
if not args.quick:
|
||||
rc |= run("per-PCTYPE media views", [sys.executable, "scripts/build-pctype-media.py", "--verify"])
|
||||
rc |= firmware_coverage()
|
||||
|
||||
print("=" * 72)
|
||||
|
||||
@@ -56,24 +56,17 @@ PAIRS = [
|
||||
"/srv/samba/enrollment/shopfloor-setup/Verify-And-Heal-Staging.ps1", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/Fetch-StagingPayload.ps1",
|
||||
"/srv/samba/enrollment/shopfloor-setup/Fetch-StagingPayload.ps1", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/site-config.json",
|
||||
"/srv/samba/enrollment/shopfloor-setup/site-config.json", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/Shopfloor/Install-AppList.ps1",
|
||||
"/srv/samba/enrollment/shopfloor-setup/Shopfloor/Install-AppList.ps1", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/Shopfloor/Register-PostPpkgTask.ps1",
|
||||
"/srv/samba/enrollment/shopfloor-setup/Shopfloor/Register-PostPpkgTask.ps1", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/Shopfloor/Invoke-PostPpkg.ps1",
|
||||
"/srv/samba/enrollment/shopfloor-setup/Shopfloor/Invoke-PostPpkg.ps1", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/BIOS/models.txt",
|
||||
"/srv/samba/winpeapps/_shared/BIOS/models.txt", GIT_OWNED),
|
||||
("playbook/shopfloor-setup/BIOS/check-bios.cmd",
|
||||
"/srv/samba/winpeapps/_shared/BIOS/check-bios.cmd", GIT_OWNED),
|
||||
|
||||
# The config/ copy is the one startnet actually stages to
|
||||
# C:\Enrollment\site-config.json, so it is the copy every bay reads. It had
|
||||
# its own edit history and sat stale since 2026-08-06, which meant bays
|
||||
# imaged after e844ff3 quietly lost the Plant Apps startup item and the
|
||||
# Defect_Tracker pin - while the sibling copy under shopfloor-setup/ showed
|
||||
# perfectly in sync and made the pair look healthy. Reconciled 2026-08-19 by
|
||||
# deploying the repo copy over it. One source file, two destinations, both
|
||||
# git-owned now, so a stale one cannot hide behind the other again.
|
||||
("playbook/shopfloor-setup/site-config.json",
|
||||
"/srv/samba/enrollment/config/site-config.json", GIT_OWNED),
|
||||
|
||||
# All three unattends are reconciled. The live files were ahead of the repo
|
||||
# and the LIVE side won, because these are what boot machines - pushing the
|
||||
# repo over them is precisely the 2026-08-06 outage.
|
||||
|
||||
@@ -1,465 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Do what eDNC and MarkTelesis do, without eDNC or MarkTelesis.
|
||||
|
||||
WHAT THIS IS FOR
|
||||
The whole part-marker integration hinges on one question: what exactly does
|
||||
GE's MarkTelesis.exe put on the wire, and can a translator turn that into
|
||||
something a SIC e10 understands? Answering it by capture needs a Windows VM,
|
||||
an installed Mark 6.2.1, a real Telesis controller and a live eMX prescan.
|
||||
|
||||
We do not need any of that to make progress, because we already know all
|
||||
three pieces:
|
||||
|
||||
1. The part-program file format, from MarkFileSpec.doc.
|
||||
2. The token substitution rules, from the same document.
|
||||
3. The exact wire frame, from disassembling MarkTelesis.exe on
|
||||
2026-08-27 - the frame builder at 0x401cf0 and its callers.
|
||||
|
||||
So this reimplements that path. Feed it a real GE part program and the part
|
||||
number and serial that eDNC would have pulled from the router scan, and it
|
||||
emits the byte-exact Telesis frames MarkTelesis would have sent. Then it can
|
||||
translate those into e10 commands and, optionally, drive a real controller.
|
||||
|
||||
That makes the whole chain testable today, on a desk, with one e10 and no
|
||||
Windows anywhere.
|
||||
|
||||
PART PROGRAM FORMAT (MarkFileSpec.doc, first column is the command character)
|
||||
: file information, "name,snumb". optional.
|
||||
^ pattern name to select on the controller. optional.
|
||||
* menu item. items run in file order.
|
||||
@ operator instruction, max 78 chars.
|
||||
! comment. programmer only, never shown.
|
||||
$ prompt for operator input, max 60 chars.
|
||||
# one choice in a pick list, max 16 chars.
|
||||
/ command line, new-style: /NN/ addresses field NN (TMC100/TMC400).
|
||||
\\ command line, old-style: \\N\\ is line N (2Line/3Line/9Line).
|
||||
Backslash strips leading spaces, forward slash keeps them.
|
||||
|
||||
TOKENS
|
||||
<PARTNO> part number <SERIALNO> serial number
|
||||
<CHECKSUM> dash + checksum char <PN1>..<PN4> part number in 5/5/5/3 chunks
|
||||
<DIM> operator dimension <TEXT> operator text
|
||||
<RDIM1..3> reusable dimension - prompted once, reused after
|
||||
|
||||
Wilmington takes only the last 5 characters of <SERIALNO>. That is a site
|
||||
rule, not a format rule, so it lives behind --site.
|
||||
|
||||
THE WIRE FRAME (disassembled, not guessed)
|
||||
EOT SOH <TYPE> STX <data> ETX <BCC:3 ascii digits> CR
|
||||
BCC = (TYPE + sum of data bytes) & 0xFF
|
||||
|
||||
NO STATION ID. Backup470 sent EOT SOH <TYPE> <2 digits> STX; MarkTelesis
|
||||
does not. Same family, different dialect.
|
||||
|
||||
TYPE letters, each confirmed at its call site:
|
||||
'P' pattern load, data is the pattern name
|
||||
'1'..'9' line N text. The loop at 0x4090b0 does `add bl, 0x31`, so the
|
||||
TYPE byte IS the line number. This is the 2Line dialect, which
|
||||
is what WJPRT runs.
|
||||
'V' set field, TMC400 dialect, field number leads the data
|
||||
'O' online, empty data
|
||||
'R' empty data, believed to be the mark trigger
|
||||
'D' pattern download, "<name>;<6-digit size>;128"
|
||||
|
||||
TRANSLATION TO THE e10
|
||||
'P' <name> -> LOAD FILE 'c'
|
||||
'1'/'2'/... <text>-> FILE SET VAR '7' into V01 / V02 / ...
|
||||
'V' <n><text> -> FILE SET VAR '7' into V<n>
|
||||
'O' -> nothing on the e10. ACK it locally.
|
||||
'R' -> START MARKING 'g'
|
||||
|
||||
The e10 side is deliberately BINARY, never text: the text protocol uses
|
||||
space as a delimiter, and these values are full of spaces - look at line 2
|
||||
of the real GE program, which has four consecutive ones.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
EOT, SOH, STX, ETX, CR = 0x04, 0x01, 0x02, 0x03, 0x0D
|
||||
|
||||
|
||||
# --------------------------------------------------------------- part program
|
||||
|
||||
class Item:
|
||||
def __init__(self, name):
|
||||
self.name = name
|
||||
self.pattern = None
|
||||
self.instructions = []
|
||||
self.commands = [] # (field, template, keep_spaces)
|
||||
self.prompts = [] # prompt text in file order
|
||||
self.choices = {} # prompt index -> [choices]
|
||||
|
||||
|
||||
def parse_part_program(text):
|
||||
"""Parse a MarkEdit part program into items."""
|
||||
items = []
|
||||
general = []
|
||||
fileinfo = None
|
||||
current = None
|
||||
|
||||
for raw in text.splitlines():
|
||||
if not raw.strip():
|
||||
continue
|
||||
tag = raw[0]
|
||||
# NEVER strip a command line. The real GE program marks
|
||||
# "S/N <SERIALNO> 4AM79" - those four spaces are marked onto the
|
||||
# part, and stripping them silently changes what gets engraved.
|
||||
body = raw[1:] if tag in ("/", "\\") else raw[1:].strip()
|
||||
|
||||
if tag == ":":
|
||||
fileinfo = body
|
||||
elif tag == "!":
|
||||
continue # comment, programmer only
|
||||
elif tag == "*":
|
||||
current = Item(body)
|
||||
items.append(current)
|
||||
elif tag == "^":
|
||||
if current:
|
||||
current.pattern = body
|
||||
elif tag == "@":
|
||||
(current.instructions if current else general).append(body)
|
||||
elif tag == "$":
|
||||
if current:
|
||||
current.prompts.append(body)
|
||||
elif tag == "#":
|
||||
if current and current.prompts:
|
||||
current.choices.setdefault(
|
||||
len(current.prompts) - 1, []).append(body)
|
||||
elif tag in ("/", "\\"):
|
||||
# /NN/text or \N\text - the delimiter decides space handling
|
||||
m = re.match(r"([0-9]+)[/\\](.*)", body)
|
||||
if not m:
|
||||
continue
|
||||
field, template = int(m.group(1)), m.group(2)
|
||||
keep_spaces = (tag == "/")
|
||||
if not keep_spaces:
|
||||
template = template.lstrip(" ")
|
||||
if current:
|
||||
current.commands.append((field, template, keep_spaces))
|
||||
|
||||
return fileinfo, general, items
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- tokens
|
||||
|
||||
def serial_checksum(serial):
|
||||
"""Telesis serial checksum character.
|
||||
|
||||
MarkFileSpec describes <CHECKSUM> as a dash followed by the checksum
|
||||
character but never defines the algorithm, and the binary computes it in
|
||||
DNCdll. Until a real job's log shows one, this is clearly marked as
|
||||
unverified rather than quietly guessed at.
|
||||
"""
|
||||
if not serial:
|
||||
return ""
|
||||
total = sum(ord(c) for c in serial) % 36
|
||||
return "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"[total]
|
||||
|
||||
|
||||
def substitute(template, partno, serial, site, operator_inputs, reusable):
|
||||
"""Replace tokens the way MarkTelesis does, returning finished text."""
|
||||
out = template
|
||||
ser = serial[-5:] if site == "wilmington" else serial
|
||||
|
||||
out = out.replace("<PARTNO>", partno)
|
||||
out = out.replace("<SERIALNO>", ser)
|
||||
out = out.replace("<CHECKSUM>", "-" + serial_checksum(serial))
|
||||
out = out.replace("<PN1>", partno[0:5])
|
||||
out = out.replace("<PN2>", partno[5:10])
|
||||
out = out.replace("<PN3>", partno[10:15])
|
||||
out = out.replace("<PN4>", partno[15:18])
|
||||
|
||||
for name in ("RDIM1", "RDIM2", "RDIM3"):
|
||||
token = "<%s>" % name
|
||||
while token in out:
|
||||
if name not in reusable:
|
||||
reusable[name] = next_input(operator_inputs, name)
|
||||
out = out.replace(token, reusable[name], 1)
|
||||
|
||||
for name in ("DIM", "TEXT"):
|
||||
token = "<%s>" % name
|
||||
while token in out:
|
||||
out = out.replace(token, next_input(operator_inputs, name), 1)
|
||||
|
||||
return out
|
||||
|
||||
|
||||
def next_input(queue, label):
|
||||
"""Take the next operator-supplied value, or a visible placeholder."""
|
||||
if queue:
|
||||
return queue.pop(0)
|
||||
return "<%s?>" % label
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- frames
|
||||
|
||||
def bcc(type_char, data):
|
||||
total = ord(type_char) + sum(data)
|
||||
return b"%03d" % (total & 0xFF)
|
||||
|
||||
|
||||
def frame(type_char, data=b""):
|
||||
"""Build one MarkTelesis request frame, byte for byte."""
|
||||
if isinstance(data, str):
|
||||
data = data.encode("ascii")
|
||||
return (bytes([EOT, SOH, ord(type_char), STX]) + data
|
||||
+ bytes([ETX]) + bcc(type_char, data) + bytes([CR]))
|
||||
|
||||
|
||||
def build_sequence(item, partno, serial, site, operator_inputs,
|
||||
marker_type="2Line", clear_first=True):
|
||||
"""Everything MarkTelesis would send for one item, in order."""
|
||||
seq = []
|
||||
reusable = {}
|
||||
|
||||
seq.append(("online", frame("O")))
|
||||
|
||||
if item.pattern:
|
||||
seq.append(("load pattern %s" % item.pattern, frame("P", item.pattern)))
|
||||
|
||||
if clear_first:
|
||||
# TmcClrFlds - blank every field before setting values, so a field the
|
||||
# program does not set cannot mark the previous part's data.
|
||||
for field, _, _ in item.commands:
|
||||
if marker_type == "2Line":
|
||||
seq.append(("clear line %d" % field, frame(str(field), " ")))
|
||||
else:
|
||||
seq.append(("clear field %d" % field,
|
||||
frame("V", "%02d %s" % (field, " "))))
|
||||
|
||||
for field, template, _ in item.commands:
|
||||
text = substitute(template, partno, serial, site,
|
||||
operator_inputs, reusable)
|
||||
if marker_type == "2Line":
|
||||
# the TYPE byte IS the line number
|
||||
seq.append(("line %d = %r" % (field, text), frame(str(field), text)))
|
||||
else:
|
||||
seq.append(("field %d = %r" % (field, text),
|
||||
frame("V", "%02d%s" % (field, text))))
|
||||
|
||||
seq.append(("mark", frame("R")))
|
||||
return seq
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- translation
|
||||
|
||||
# Field number -> e10 variable name. Default is the V01/V02 convention we
|
||||
# intend for authored patterns; override when driving a file whose variables
|
||||
# were named by someone else's GUI.
|
||||
VAR_NAMES = {}
|
||||
|
||||
|
||||
def var_for(field):
|
||||
return VAR_NAMES.get(field, "V%02d" % field)
|
||||
|
||||
|
||||
def translate(type_char, data):
|
||||
"""Turn one Telesis frame into the e10 action it corresponds to.
|
||||
|
||||
Returns (action, args) where action is one of:
|
||||
load, setvar, mark, ack_locally
|
||||
"""
|
||||
text = data.decode("ascii", "replace")
|
||||
if type_char == "P":
|
||||
return ("load", text)
|
||||
if type_char == "O":
|
||||
return ("ack_locally", "online - the e10 has no equivalent")
|
||||
if type_char == "R":
|
||||
return ("mark", "")
|
||||
if type_char.isdigit():
|
||||
return ("setvar", (var_for(int(type_char)), text))
|
||||
if type_char == "V":
|
||||
return ("setvar", (var_for(int(text[:2])), text[2:]))
|
||||
return ("ack_locally", "unhandled type %r" % type_char)
|
||||
|
||||
|
||||
def decode_frame(raw):
|
||||
"""Pull (type, data) back out of a frame we built."""
|
||||
type_char = chr(raw[2])
|
||||
end = raw.index(ETX, 4)
|
||||
return type_char, raw[4:end]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- main
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Simulate what eDNC/MarkTelesis do with a part program.")
|
||||
parser.add_argument("partprogram")
|
||||
parser.add_argument("--partno", default="07482ABC",
|
||||
help="part number eDNC would get from the router scan")
|
||||
parser.add_argument("--serial", default="12345678",
|
||||
help="serial number from the router scan")
|
||||
parser.add_argument("--site", default="westjefferson",
|
||||
choices=["westjefferson", "wilmington"])
|
||||
parser.add_argument("--marker-type", default="2Line",
|
||||
choices=["2Line", "TMC400"],
|
||||
help="which dialect MarkTelesis would emit."
|
||||
" WJPRT runs 2Line.")
|
||||
parser.add_argument("--input", action="append", default=[],
|
||||
help="operator input, in prompt order. repeatable.")
|
||||
parser.add_argument("--no-clear", action="store_true",
|
||||
help="skip the clear-fields pass")
|
||||
parser.add_argument("--e10", metavar="DEVICE", nargs="?", const="",
|
||||
help="also drive a real e10. Give a port, or pass the"
|
||||
" flag bare to use $E10_PORT / the platform"
|
||||
" default.")
|
||||
parser.add_argument("--pattern",
|
||||
help="override the pattern name from the ^ line."
|
||||
" Useful when the e10 file is named differently"
|
||||
" from the Telesis pattern.")
|
||||
parser.add_argument("--var-names",
|
||||
help="comma-separated e10 variable names for fields"
|
||||
" 1,2,3... e.g. PN,SN. Default is V01,V02,...")
|
||||
parser.add_argument("--really-mark", action="store_true",
|
||||
help="fire the stylus for real. WITHOUT this, marking"
|
||||
" runs in simulation mode, force 0, stylus up.")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.var_names:
|
||||
for index, name in enumerate(args.var_names.split(","), start=1):
|
||||
VAR_NAMES[index] = name.strip()
|
||||
|
||||
try:
|
||||
with open(args.partprogram) as handle:
|
||||
fileinfo, general, items = parse_part_program(handle.read())
|
||||
except FileNotFoundError:
|
||||
sys.stdout.flush()
|
||||
sys.stderr.write("no such part program: %s\n" % args.partprogram)
|
||||
here = os.path.join(os.path.dirname(os.path.abspath(__file__)),
|
||||
"..", "partprograms")
|
||||
for folder in (os.path.dirname(args.partprogram) or ".", here):
|
||||
try:
|
||||
found = sorted(f for f in os.listdir(folder)
|
||||
if f.lower().endswith(".txt"))
|
||||
except OSError:
|
||||
continue
|
||||
if found:
|
||||
sys.stderr.write("part programs in %s:\n"
|
||||
% os.path.normpath(folder))
|
||||
for name in found:
|
||||
sys.stderr.write(" %s\n" % name)
|
||||
break
|
||||
else:
|
||||
sys.stderr.write("Give the path to a .txt part program. On Windows"
|
||||
" use backslashes:\n"
|
||||
" python scripts\\simulate-ednc.py"
|
||||
" partprograms\\TEL3L0001.txt ...\n")
|
||||
sys.exit(1)
|
||||
except IsADirectoryError:
|
||||
sys.stdout.flush()
|
||||
sys.stderr.write("%s is a folder, not a part program file\n"
|
||||
% args.partprogram)
|
||||
sys.exit(1)
|
||||
|
||||
print("=" * 72)
|
||||
print("PART PROGRAM: %s" % args.partprogram)
|
||||
if fileinfo:
|
||||
print(" file info %s" % fileinfo)
|
||||
print(" part number %s (from the router scan)" % args.partno)
|
||||
print(" serial %s" % args.serial)
|
||||
print(" marker type %s" % args.marker_type)
|
||||
for line in general:
|
||||
print(" instruction %s" % line)
|
||||
print("=" * 72)
|
||||
|
||||
for item in items:
|
||||
if args.pattern:
|
||||
item.pattern = args.pattern
|
||||
print("\nITEM %s" % item.name)
|
||||
if item.pattern:
|
||||
print(" pattern %s" % item.pattern)
|
||||
for line in item.instructions:
|
||||
print(" operator sees %s" % line)
|
||||
for field, template, keep in item.commands:
|
||||
print(" line %-2d %r%s"
|
||||
% (field, template, "" if keep else " (leading spaces stripped)"))
|
||||
|
||||
seq = build_sequence(item, args.partno, args.serial, args.site,
|
||||
list(args.input), args.marker_type,
|
||||
clear_first=not args.no_clear)
|
||||
|
||||
print("\n WHAT MARKTELESIS WOULD PUT ON THE WIRE")
|
||||
print(" %-28s %s" % ("meaning", "bytes"))
|
||||
print(" " + "-" * 68)
|
||||
for label, raw in seq:
|
||||
print(" %-28s %s" % (label, raw.hex(" ")))
|
||||
|
||||
print("\n TRANSLATED FOR THE e10")
|
||||
for label, raw in seq:
|
||||
type_char, data = decode_frame(raw)
|
||||
action, payload = translate(type_char, data)
|
||||
if action == "load":
|
||||
print(" %-28s LOAD FILE %r" % (label, payload))
|
||||
elif action == "setvar":
|
||||
print(" %-28s SETVAR %s = %r" % (label, payload[0], payload[1]))
|
||||
elif action == "mark":
|
||||
print(" %-28s START MARKING%s"
|
||||
% (label, "" if args.really_mark else " (SIMULATION)"))
|
||||
else:
|
||||
print(" %-28s [%s]" % (label, payload))
|
||||
|
||||
# `is not None`, NOT a truth test. --e10 passed bare sets this to an
|
||||
# empty string, which is falsy, so a truth test silently turns a
|
||||
# hardware run into a dry run - with no error to explain why nothing
|
||||
# reached the controller.
|
||||
if args.e10 is not None:
|
||||
drive_e10(seq, args)
|
||||
|
||||
|
||||
def drive_e10(seq, args):
|
||||
"""Run the translated sequence against a real controller."""
|
||||
sys.path.insert(0, __file__.rsplit("/", 1)[0])
|
||||
import importlib.util
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"e10", __file__.rsplit("/", 1)[0] + "/e10-client.py")
|
||||
e10mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(e10mod)
|
||||
|
||||
port = args.e10 or e10mod.default_port()
|
||||
print("\n DRIVING THE REAL CONTROLLER on %s" % port)
|
||||
with e10mod.E10(port, timeout=5.0) as dev:
|
||||
for label, raw in seq:
|
||||
type_char, data = decode_frame(raw)
|
||||
action, payload = translate(type_char, data)
|
||||
try:
|
||||
if action == "load":
|
||||
dev.load_file(payload)
|
||||
print(" %-26s LOAD FILE ok" % label)
|
||||
elif action == "setvar":
|
||||
dev.set_var(payload[0], payload[1])
|
||||
print(" %-26s SETVAR ok" % label)
|
||||
elif action == "mark":
|
||||
dev.start_marking(simulate=not args.really_mark)
|
||||
print(" %-26s started, waiting for home..." % label)
|
||||
dev.wait_for_mark(on_pause=lambda: print(
|
||||
" PAUSE - press Start on the machine"))
|
||||
print(" %-26s cycle complete" % label)
|
||||
else:
|
||||
print(" %-26s (nothing to send)" % label)
|
||||
except e10mod.E10Error as err:
|
||||
print(" %-26s FAILED: %s" % (label, err))
|
||||
break
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except KeyboardInterrupt:
|
||||
sys.exit(130)
|
||||
except Exception as err: # noqa: BLE001
|
||||
# E10Error is defined in the client module, which is loaded
|
||||
# dynamically, so catch by name rather than importing it up here.
|
||||
if type(err).__name__ in ("E10Error", "E10Fault"):
|
||||
# Flush stdout first. It is block-buffered when piped while stderr
|
||||
# is not, so without this the error appears BEFORE the output it
|
||||
# refers to - and a `| tail` drops it entirely, which looks like
|
||||
# the command failed silently.
|
||||
sys.stdout.flush()
|
||||
sys.stderr.write("e10: %s\n" % err)
|
||||
sys.stderr.flush()
|
||||
sys.exit(1)
|
||||
raise
|
||||
108
scripts/stage-usb-share.sh
Executable file
108
scripts/stage-usb-share.sh
Executable file
@@ -0,0 +1,108 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# stage-usb-share.sh - Put everything Build-PxeUsb.ps1 needs into a folder
|
||||
# on a share, so a Windows PC can build the installer USB from it.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/stage-usb-share.sh [--iso FILE] [--payload-root DIR] [DEST]
|
||||
#
|
||||
# DEST defaults to ~/pxe-images/pxe-usb (the [pxe-images] Samba share; on
|
||||
# the work PC: \\<this PC>\pxe-images\pxe-usb, usually Z:\pxe-usb). It gets:
|
||||
# - the repo as committed (git archive HEAD: uncommitted edits are NOT
|
||||
# staged, so the stick matches a commit);
|
||||
# - the payload folders (offline-packages, pip-wheels, boot-files,
|
||||
# boot-tools, enrollment, bios-staging, drivers-staging, site-content)
|
||||
# as hard links, so they cost no extra disk space;
|
||||
# - the Ubuntu ISO offline-packages/ was built for (hard link);
|
||||
# - STAGED.txt: commit, date, sizes.
|
||||
# Then on the work PC: copy the folder to a local disk (C:\pxe-usb) and
|
||||
# double-click Build-PxeUsb.cmd (KB: docs/kb/How_To_Deploy_PXE_Server_New_Site.html).
|
||||
#
|
||||
# The new copy is built beside the old one and swapped in when complete.
|
||||
# Run capture-site-content.sh --apply first so site-content/ is current.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")"/.. && pwd)"
|
||||
PAYLOAD_ROOT="$REPO_ROOT"
|
||||
ISO=""
|
||||
DEST=""
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--iso) ISO="${2:?}"; shift 2 ;;
|
||||
--payload-root) PAYLOAD_ROOT="$(cd "${2:?}" && pwd)"; shift 2 ;;
|
||||
-h|--help) sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
-*) echo "unknown option $1" >&2; exit 2 ;;
|
||||
*) DEST="$1"; shift ;;
|
||||
esac
|
||||
done
|
||||
DEST="${DEST:-$HOME/pxe-images/pxe-usb}"
|
||||
mkdir -p "$(dirname "$DEST")"
|
||||
DEST="$(cd "$(dirname "$DEST")" && pwd)/$(basename "$DEST")"
|
||||
|
||||
# Only ever replace a folder this script made (it is swapped and deleted).
|
||||
if [ -e "$DEST" ] && [ ! -f "$DEST/STAGED.txt" ]; then
|
||||
echo "ERROR: $DEST exists and was not made by this script (no STAGED.txt); refusing to replace it" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# The ISO the offline set was resolved for (BUILD-INFO names it).
|
||||
if [ -z "$ISO" ]; then
|
||||
name=$(sed -n 's/.*--iso \([^ ]*\.iso\).*/\1/p' "$PAYLOAD_ROOT/offline-packages/BUILD-INFO.txt" 2>/dev/null | head -1)
|
||||
name=$(basename "${name:-}")
|
||||
for d in "$PAYLOAD_ROOT" "$PAYLOAD_ROOT/iso" "${PXETEST_CACHE:-$HOME/.cache/pxetest}"; do
|
||||
[ -n "$name" ] && [ -f "$d/$name" ] && { ISO="$d/$name"; break; }
|
||||
done
|
||||
[ -n "$ISO" ] || { echo "ERROR: give --iso FILE (the offline set was built for ${name:-an unknown ISO})" >&2; exit 1; }
|
||||
fi
|
||||
[ -f "$ISO" ] || { echo "ERROR: ISO not found: $ISO" >&2; exit 1; }
|
||||
|
||||
same_fs() { [ "$(stat -c %d "$1")" = "$(stat -c %d "$2")" ]; }
|
||||
|
||||
NEW="$DEST.new.$$"
|
||||
OLD="$DEST.old.$$"
|
||||
cleanup() { [ -d "$NEW" ] && rm -rf -- "$NEW"; return 0; }
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$NEW"
|
||||
|
||||
cd "$REPO_ROOT"
|
||||
COMMIT=$(git rev-parse --short HEAD)
|
||||
if ! git diff --quiet HEAD -- autoinstall playbook scripts webapp; then
|
||||
echo "NOTE: uncommitted changes in autoinstall/playbook/scripts/webapp are NOT staged (HEAD $COMMIT is)."
|
||||
fi
|
||||
echo "Repo $COMMIT -> $NEW"
|
||||
git archive HEAD | tar -x -C "$NEW"
|
||||
|
||||
link_or_copy() { # SRC DST
|
||||
if same_fs "$1" "$(dirname "$2")"; then cp -al "$1" "$2"; else cp -a "$1" "$2"; fi
|
||||
}
|
||||
for d in offline-packages pip-wheels boot-files boot-tools enrollment bios-staging drivers-staging site-content; do
|
||||
src="$PAYLOAD_ROOT/$d"
|
||||
[ -d "$src" ] || { echo " $d: not in $PAYLOAD_ROOT (skipped)"; continue; }
|
||||
# git archive brings the tracked bits of boot-files/ and boot-tools/; the
|
||||
# payload folder replaces them whole.
|
||||
rm -rf -- "${NEW:?}/$d"
|
||||
link_or_copy "$src" "$NEW/$d"
|
||||
printf ' %-17s %s\n' "$d" "$(du -sh "$NEW/$d" | cut -f1)"
|
||||
done
|
||||
link_or_copy "$ISO" "$NEW/$(basename "$ISO")"
|
||||
echo " ISO $(basename "$ISO")"
|
||||
|
||||
{
|
||||
echo "Staged by scripts/stage-usb-share.sh $(date -Is)"
|
||||
echo "Commit: $(git log -1 --format='%h %s')"
|
||||
echo "ISO: $(basename "$ISO")"
|
||||
echo "Payload root: $PAYLOAD_ROOT"
|
||||
echo
|
||||
echo "On the Windows PC:"
|
||||
echo " 1. Copy this whole folder to a local disk, e.g. C:\\pxe-usb"
|
||||
echo " 2. Double-click C:\\pxe-usb\\Build-PxeUsb.cmd (asks for admin, lists USB disks)"
|
||||
echo " Instructions: docs\\kb\\How_To_Deploy_PXE_Server_New_Site.html"
|
||||
} >"$NEW/STAGED.txt"
|
||||
|
||||
if [ -e "$DEST" ]; then mv -- "$DEST" "$OLD"; fi
|
||||
mv -- "$NEW" "$DEST"
|
||||
[ -e "$OLD" ] && rm -rf -- "$OLD"
|
||||
trap - EXIT
|
||||
echo "Staged: $DEST ($(du -sh "$DEST" | cut -f1) apparent, hard-linked payload)"
|
||||
@@ -1,177 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Sync HardwareDriver.json and user_selections.json across all PXE image types.
|
||||
|
||||
Reads all HardwareDriver.json files, builds a unified driver catalog,
|
||||
then updates each image to include all known hardware models.
|
||||
Run after adding new driver packs to the shared Out-of-box Drivers directory.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from collections import OrderedDict
|
||||
|
||||
WINPEAPPS = Path("/srv/samba/winpeapps")
|
||||
SHARED_DRIVERS = WINPEAPPS / "_shared" / "Out-of-box Drivers"
|
||||
|
||||
|
||||
def normalize_entry(entry):
|
||||
"""Normalize a HardwareDriver.json entry to a consistent format."""
|
||||
norm = {}
|
||||
norm["manufacturer"] = entry.get("manufacturer", "Dell")
|
||||
norm["product"] = entry.get("product") or entry.get("manufacturerfriendlyname", "Dell")
|
||||
norm["family"] = entry.get("family", "")
|
||||
norm["modelswminame"] = entry.get("modelswminame") or entry.get("models", "")
|
||||
norm["modelsfriendlyname"] = entry.get("modelsfriendlyname", "")
|
||||
norm["fileName"] = entry.get("fileName") or entry.get("FileName", "")
|
||||
norm["destinationDir"] = entry.get("destinationDir") or entry.get("DestinationDir", "")
|
||||
norm["url"] = entry.get("url", "")
|
||||
norm["hash"] = entry.get("hash", "")
|
||||
norm["size"] = entry.get("size", 0)
|
||||
norm["modifiedDate"] = entry.get("modifiedDate", "0001-01-01T00:00:00")
|
||||
norm["osId"] = entry.get("osId", "")
|
||||
norm["imagedisk"] = entry.get("imagedisk", 0)
|
||||
return norm
|
||||
|
||||
|
||||
def merge_os_ids(a, b):
|
||||
"""Merge two osId strings (e.g., '18' + '20,21' -> '18,20,21')."""
|
||||
ids = set()
|
||||
for oid in [a, b]:
|
||||
for part in str(oid).split(","):
|
||||
part = part.strip()
|
||||
if part:
|
||||
ids.add(part)
|
||||
return ",".join(sorted(ids, key=lambda x: int(x) if x.isdigit() else 0))
|
||||
|
||||
|
||||
def check_driver_exists(entry):
|
||||
"""Check if the driver zip actually exists in the shared directory."""
|
||||
dest = entry["destinationDir"]
|
||||
dest = dest.replace("*destinationdir*", "")
|
||||
dest = dest.lstrip("\\")
|
||||
dest = dest.replace("\\", "/")
|
||||
# Strip leading path components that are already in SHARED_DRIVERS
|
||||
for prefix in ["Deploy/Out-of-box Drivers/", "Out-of-box Drivers/"]:
|
||||
if dest.startswith(prefix):
|
||||
dest = dest[len(prefix):]
|
||||
break
|
||||
dest = dest.lstrip("/")
|
||||
zip_path = SHARED_DRIVERS / dest / entry["fileName"]
|
||||
return zip_path.exists()
|
||||
|
||||
|
||||
def main():
|
||||
print("=== PXE Hardware Model Sync ===")
|
||||
print()
|
||||
|
||||
# Step 1: Build unified catalog from all images
|
||||
print("Reading driver catalogs...")
|
||||
catalog = OrderedDict()
|
||||
|
||||
image_dirs = sorted(
|
||||
[d for d in WINPEAPPS.iterdir() if d.is_dir() and not d.name.startswith("_")]
|
||||
)
|
||||
|
||||
for img_dir in image_dirs:
|
||||
hw_file = img_dir / "Deploy" / "Control" / "HardwareDriver.json"
|
||||
if not hw_file.exists():
|
||||
continue
|
||||
with open(hw_file) as f:
|
||||
entries = json.load(f)
|
||||
print(" Read {} entries from {}".format(len(entries), img_dir.name))
|
||||
for entry in entries:
|
||||
norm = normalize_entry(entry)
|
||||
key = (norm["family"], norm["fileName"])
|
||||
if key in catalog:
|
||||
catalog[key]["osId"] = merge_os_ids(
|
||||
catalog[key]["osId"], norm["osId"]
|
||||
)
|
||||
# Prefer longer/more complete model names
|
||||
if len(norm["modelswminame"]) > len(catalog[key]["modelswminame"]):
|
||||
catalog[key]["modelswminame"] = norm["modelswminame"]
|
||||
if len(norm["modelsfriendlyname"]) > len(
|
||||
catalog[key]["modelsfriendlyname"]
|
||||
):
|
||||
catalog[key]["modelsfriendlyname"] = norm["modelsfriendlyname"]
|
||||
else:
|
||||
catalog[key] = norm
|
||||
|
||||
unified = list(catalog.values())
|
||||
print()
|
||||
print("Unified catalog: {} unique driver entries".format(len(unified)))
|
||||
|
||||
# Step 2: Check which drivers actually exist on disk
|
||||
missing = []
|
||||
found = 0
|
||||
for entry in unified:
|
||||
if check_driver_exists(entry):
|
||||
found += 1
|
||||
else:
|
||||
missing.append(
|
||||
" {}: {}".format(entry["family"], entry["fileName"])
|
||||
)
|
||||
|
||||
print(" {} drivers found on disk".format(found))
|
||||
if missing:
|
||||
print(" WARNING: {} driver zips NOT found on disk:".format(len(missing)))
|
||||
for m in missing[:15]:
|
||||
print(m)
|
||||
if len(missing) > 15:
|
||||
print(" ... and {} more".format(len(missing) - 15))
|
||||
print(" (Entries still included - PESetup may download them)")
|
||||
|
||||
# Step 3: Build unified model selection from all driver entries
|
||||
models = []
|
||||
seen = set()
|
||||
for entry in unified:
|
||||
friendly_names = [
|
||||
n.strip()
|
||||
for n in entry["modelsfriendlyname"].split(",")
|
||||
if n.strip()
|
||||
]
|
||||
family = entry["family"]
|
||||
for name in friendly_names:
|
||||
key = (name, family)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
models.append({"Model": name, "Id": family})
|
||||
models.sort(key=lambda x: x["Model"])
|
||||
print()
|
||||
print("Unified model selection: {} models".format(len(models)))
|
||||
|
||||
# Step 4: Update each image
|
||||
print()
|
||||
print("Updating images...")
|
||||
for img_dir in image_dirs:
|
||||
hw_file = img_dir / "Deploy" / "Control" / "HardwareDriver.json"
|
||||
us_file = img_dir / "Tools" / "user_selections.json"
|
||||
if not hw_file.exists() or not us_file.exists():
|
||||
continue
|
||||
|
||||
# Write unified HardwareDriver.json
|
||||
with open(hw_file, "w") as f:
|
||||
json.dump(unified, f, indent=2)
|
||||
f.write("\n")
|
||||
|
||||
# Update user_selections.json (preserve OperatingSystemSelection etc.)
|
||||
with open(us_file) as f:
|
||||
user_sel = json.load(f)
|
||||
old_count = len(user_sel[0].get("HardwareModelSelection", []))
|
||||
user_sel[0]["HardwareModelSelection"] = models
|
||||
with open(us_file, "w") as f:
|
||||
json.dump(user_sel, f, indent=2)
|
||||
f.write("\n")
|
||||
|
||||
print(
|
||||
" {}: {} -> {} models, {} driver entries".format(
|
||||
img_dir.name, old_count, len(models), len(unified)
|
||||
)
|
||||
)
|
||||
|
||||
print()
|
||||
print("Done!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user