ge-aerospace/pxe-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

webapp import: checksum-aware sync (skip unchanged, update only changed)

Browse Source 
Image import previously full-copied everything and rmtree'd existing target dirs on every run. Replaced the shutil copy/move/copytree/rmtree with deploy.sync_tree(), which shells out to rsync -a --checksum: files whose content already matches the target are skipped, only new or changed files are written, and existing target files not in the source are left untouched (merge, not mirror). move=True uses --remove-source-files (frees the SMB upload dir) and prunes emptied source dirs. Applies to the Deploy import, the _shared redirections, and the root-level items. Big re-imports now only rewrite what actually changed.
This commit is contained in:
cproudlock
2026-07-23 10:45:58 -04:00
parent 8fbae24b4d
commit 0cb6b26c27
2 changed files with 42 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
webapp/app.py
View File

@@ -113,9 +113,9 @@ def images_import():
src_items = os.listdir(source)
# Move files from network upload to save disk space; copy from USB.
# deploy.sync_tree uses rsync --checksum: unchanged files are
# skipped, only new/changed files are written (no full replace).
use_move = source == config.UPLOAD_DIR or source.startswith(config.UPLOAD_DIR + "/")
_transfer = shutil.move if use_move else shutil.copy2
_transfer_tree = shutil.move if use_move else shutil.copytree
top_dirs = {d for d in src_items if os.path.isdir(os.path.join(source, d))}
full_layout = "Deploy" in top_dirs
@@ -134,8 +134,7 @@ def images_import():
elif os.path.isdir(src_item) and item in shared_root:
prefix_key = target.split("-")[0] + "-"
shared_dest = os.path.join(config.SHARED_DIR, f"{prefix_key}{item}")
os.makedirs(shared_dest, exist_ok=True)
deploy._merge_tree(src_item, shared_dest, move=use_move)
deploy.sync_tree(src_item, shared_dest, move=use_move)
dst_item = os.path.join(root, item)
if os.path.islink(dst_item):
os.remove(dst_item)
@@ -143,12 +142,9 @@ def images_import():
shutil.rmtree(dst_item)
os.symlink(shared_dest, dst_item)
elif os.path.isdir(src_item):
dst_item = os.path.join(root, item)
if os.path.exists(dst_item):
shutil.rmtree(dst_item)
_transfer_tree(src_item, dst_item)
deploy.sync_tree(src_item, os.path.join(root, item), move=use_move)
else:
_transfer(src_item, os.path.join(root, item))
deploy.sync_tree(src_item, os.path.join(root, item), move=use_move)
else:
deploy.import_deploy(source, dest, target, move=use_move)