Compare commits

..

35 Commits

Author SHA1 Message Date
Nicholas Dudfield
83f6bc64e1 fix: restore ninja -v flag for compile command visibility
Removed in b24e4647b under naive assumption we were "past debugging phase".

Reality: debugging is never done, verbose output is invaluable, costs nothing.
Keep -v flag permanently.
2025-10-31 13:09:53 +07:00
Nicholas Dudfield
be6fad9692 fix: revert to PID-based temp files (was working before)
Reverts the unnecessary mktemp change from 638cb0afe that broke cache saving.

What happened:
- Original delta code used $$ (PID) for temp files: DELTA_TARBALL="/tmp/...-$$.tar.zst"
- This creates a STRING, not a file - zstd creates the file when writing
- When removing deltas (638cb0afe), I unnecessarily changed to mktemp for "better practice"
- mktemp CREATES an empty file - zstd refuses to overwrite it
- Result: "already exists; not overwritten" error

Why it seemed to work:
- Immutability check skipped save for existing caches
- Upload code path never executed during testing
- Bug only appeared when actually trying to create new cache

The fix:
- Revert to PID-based naming ($$) that was working
- Don't fix what isn't broken

Applies to both save and restore actions for consistency.
2025-10-31 12:10:13 +07:00
Nicholas Dudfield
b24e4647ba fix: configure ccache after cache restore to prevent stale config
Same ordering bug as Conan profile (Session 8) - ccache config was being
created in workflow BEFORE cache restore, causing cached ccache.conf to
overwrite fresh configuration.

Changes:
- Build action: Add ccache config inputs (max_size, hash_dir, compiler_check)
- Build action: Configure ccache AFTER cache restore (overwrites cached config)
- Build action: Add "Show ccache config before build" step (debugging aid)
- Build action: Remove debug steps (past debugging phase)
- Build action: Remove ninja -v flag (past debugging phase)
- Nix workflow: Remove "Configure ccache" step (now handled in build action)
- macOS workflow: Remove "Configure ccache" step (now handled in build action)
- macOS workflow: Add missing stdlib and AWS credentials to build step
- Delete unused xahau-configure-ccache action (logic moved to build action)

Flow now matches Conan pattern:
1. Restore cache (includes potentially stale config)
2. Configure ccache (overwrites with fresh config: 2G max, hash_dir=true, compiler_check=content)
3. Show config (verification)
4. Build

This ensures fresh ccache configuration for each job, preventing issues
from cached config files with different settings.
2025-10-31 11:03:12 +07:00
Nicholas Dudfield
638cb0afe5 refactor: remove OverlayFS delta caching entirely
THE GREAT CULLING: Remove all OverlayFS and delta caching logic.

After extensive investigation and testing, we determined that OverlayFS
file-level layering is fundamentally incompatible with ccache's access
patterns:

- ccache opens files with O_RDWR → kernel must provide writable file handle
- OverlayFS must copy files to upper layer immediately (can't wait)
- Even with metacopy=on, metadata-only files still appear in upper layer
- Result: ~366MB deltas instead of tiny incremental diffs

The fundamental constraint: cannot have all three of:
1. Read-only lower layer (for base sharing)
2. Writable file handles (for O_RDWR)
3. Minimal deltas (for efficient caching)

Changes:
- Removed all OverlayFS mounting/unmounting logic
- Removed workspace and registry tracking
- Removed delta creation and restoration
- Removed use-deltas parameter
- Simplified to direct tar/extract workflow

Before: 726 lines across cache actions
After:  321 lines (-55% reduction)

Benefits:
-  Simpler architecture (direct tar/extract)
-  More maintainable (less code, less complexity)
-  More reliable (fewer moving parts)
-  Same performance (base-only was already used)
-  Clear path forward (restic/borg for future optimization)

Current state works great:
- Build times: 20-30 min → 2-5 min (80% improvement)
- Cache sizes: ~323-609 MB per branch (with zst compression)
- S3 costs: acceptable for current volume

If bandwidth costs become problematic, migrate to restic/borg for
chunk-level deduplication (completely different architecture).
2025-10-31 10:30:31 +07:00
Nicholas Dudfield
bd384e6bc1 Revert "feat: enable metacopy=on to test metadata-only copy-up"
This reverts commit 4c546e5d91.
2025-10-31 09:51:27 +07:00
Nicholas Dudfield
4c546e5d91 feat: enable metacopy=on to test metadata-only copy-up
Mount OverlayFS with metacopy=on option (kernel 4.2+, supported on ubuntu-22.04).
This prevents full file copy-up when files are opened with O_RDWR but not modified.

Expected behavior:
- ccache opens cache files with write access
- OverlayFS creates metadata-only entry in upper layer
- Full copy-up only happens if data is actually written
- Should dramatically reduce delta sizes from ~324 MB to ~KB

Re-enabled use-deltas for ccache to test this optimization.
Conan remains base-only (hash-based keys mean exact match most of the time).

If successful, deltas should be tiny for cache hit scenarios.
2025-10-31 09:36:09 +07:00
Nicholas Dudfield
28727b3f86 perf: disable delta caching (use base-only mode)
OverlayFS operates at FILE LEVEL:
- File modified by 8 bytes → Copy entire 500 KB file to delta
- ccache updates LRU metadata (8 bytes) on every cache hit
- 667 cache hits = 667 files copied = 324 MB delta (nearly full cache)

What we need is BYTE LEVEL deltas:
- File modified by 8 bytes → Delta is 8 bytes
- 100% cache hit scenario: 10 KB delta (not 324 MB)

OverlayFS is the wrong abstraction for ccache. Binary diff tools (rsync,
xdelta3, borg) would work better, but require significant rework.

For now: Disable deltas, use base-only caching
- Base cache restore still provides massive speed boost (2-5 min vs 20-30)
- Simpler, more reliable
- Saves S3 bandwidth (no 324 MB delta uploads)

Future: Research binary diff strategies (see Gemini prompt in session notes)
2025-10-31 08:11:47 +07:00
Nicholas Dudfield
a4f96a435a fix: don't override ccache's default cache_dir
PROBLEM: Setting cache_dir='~/.ccache' explicitly stores the
LITERAL tilde in ccache.conf, which might not expand properly.

SOLUTION: Don't set cache_dir at all - let ccache use its default
- Linux default: ~/.ccache
- ccache will expand the tilde correctly when using default
- Only configure max_size, hash_dir, compiler_check
- Remove CCACHE_DIR env var (not needed with default)

This should fix the issue where ccache runs but creates no cache files.
2025-10-30 17:58:08 +07:00
Nicholas Dudfield
d0f63cc2d1 debug: add ccache directory inspection after build
Add comprehensive debug output to inspect ~/.ccache contents:
- Full recursive directory listing
- Disk space check (rule out disk full)
- ccache config verification
- Directory sizes
- List actual files (distinguish config from cache data)

This will help diagnose why ccache is being invoked on every
compilation but not creating any cache files (0.0 GB).
2025-10-30 17:53:33 +07:00
Nicholas Dudfield
2433bfe277 debug: add ninja verbose output to see compile commands
Temporarily adding -v flag to ninja builds to see actual compile
commands. This will show whether ccache is actually being invoked
or if the CMAKE_C_COMPILER_LAUNCHER setting isn't taking effect.

We should see either:
  ccache /usr/bin/g++-13 -c file.cpp  (working)
  /usr/bin/g++-13 -c file.cpp         (not working)

This is temporary for debugging - remove once ccache issue resolved.
2025-10-30 17:04:00 +07:00
Nicholas Dudfield
ef40a7f351 refactor: move Conan profile creation after cache restore
PROBLEM: Profile was created before cache restore, then immediately
overwritten by cached profile. This meant we were using a potentially
stale cached profile instead of fresh configuration.

SOLUTION: Move profile creation into dependencies action after restore
- Dependencies action now takes compiler params as inputs
- Profile created AFTER cache restore (overwrites any cached profile)
- Ensures fresh profile with correct compiler settings for each job

CHANGES:
- Dependencies action: Add inputs (os, arch, compiler, compiler_version, cc, cxx)
- Dependencies action: Add 'Configure Conan' step after cache restore
- Dependencies action: Support both Linux and macOS profile generation
- Nix workflow: Remove 'Configure Conan' step, pass compiler params
- macOS workflow: Detect compiler version, pass to dependencies action

Benefits:
 Profile always fresh and correct for current matrix config
 Cache still includes .conan.db and other important files
 Self-contained dependencies action (easier to understand)
 Works for both Linux (with explicit cc/cxx) and macOS (auto-detect)
2025-10-30 16:41:43 +07:00
Nicholas Dudfield
a4a4126bdc fix: indent heredoc content for valid YAML syntax
The heredoc content inside the run block needs proper indentation
to be valid YAML. Without indentation, the YAML parser treats the
lines as top-level keys, causing syntax errors.

The generated CMake file will have leading spaces, but CMake handles
this fine (whitespace-tolerant parser).
2025-10-30 16:17:40 +07:00
Nicholas Dudfield
0559b6c418 fix: enable ccache for main app via wrapper toolchain
PROBLEM: ccache was never being used (all builds compiling from scratch)
- CMake command-line args (-DCMAKE_C_COMPILER_LAUNCHER=ccache) were
  being overridden by Conan's conan_toolchain.cmake
- This toolchain file loads AFTER command-line args and resets settings
- Result: Empty ccache entries (~200 bytes) in both old and new cache

SOLUTION: Wrapper toolchain that overlays ccache on Conan's toolchain
- Create wrapper_toolchain.cmake that includes Conan's toolchain first
- Then sets CMAKE_C_COMPILER_LAUNCHER and CMAKE_CXX_COMPILER_LAUNCHER
- This happens AFTER Conan's toolchain loads, so settings stick
- Only affects main app build (NOT Conan dependency builds)

CHANGES:
- Build action: Create wrapper toolchain when ccache enabled
- Build action: Use CMAKE_CURRENT_LIST_DIR for correct relative path
- Build action: Remove broken CCACHE_ARGS logic (was being overridden)
- Build action: Use ${TOOLCHAIN_FILE} variable instead of hardcoded path

This approach:
 Conan dependency builds: Clean (no ccache overhead)
 Main xahaud build: Uses ccache via wrapper
 Separation: Build action controls ccache, not Conan profile
 Simple: No profile regeneration, just one wrapper file
2025-10-30 16:06:45 +07:00
Nicholas Dudfield
f8d1a6f2b4 fix: normalize paths in cache actions to fix bootstrap detection
Path comparison was failing when registry had expanded paths
(/home/runner/.ccache) but input had unexpanded paths (~/.ccache),
causing bootstrap mode to not be detected.

Now both restore and save actions consistently expand tildes to
absolute paths before writing to or reading from the mount registry.
2025-10-30 13:25:55 +07:00
Nicholas Dudfield
c46ede7c8f chore: bump CACHE_VERSION to 3 for fresh bootstrap
Triggering clean rebuild after S3 cache cleared and ccache simplified to single directory.
2025-10-30 11:57:02 +07:00
Nicholas Dudfield
0e2bc365ea refactor: simplify ccache to single directory with restore-keys fallback
Removed split ccache configuration (~/.ccache-main + ~/.ccache-current).

The split had an edge case: building a feature branch before main branch
cache is primed results in double ccache disk usage:
- Feature branch builds first → populates ~/.ccache-current (2G max_size)
- Main branch builds later → populates ~/.ccache-main (2G max_size)
- Next feature branch build → restores both caches (4G total)

This doubles ccache disk usage unnecessarily on GitHub-hosted runners.

New single-directory approach:
- Single ~/.ccache directory for all branches
- Branch-specific keys with restore-keys fallback to main branch
- Cache actions handle cross-branch sharing via partial-match mode
- Simpler and prevents double disk usage edge case

Changes:
- xahau-configure-ccache: Single cache_dir input, removed branch logic
- xahau-ga-build: Single restore/save step, uses restore-keys for fallback
- xahau-ga-nix: Removed is_main_branch parameter

Disk usage: 2G max (vs 4G worst case with split)
2025-10-30 11:24:14 +07:00
Nicholas Dudfield
446bc76b69 fix: remove broken cache save conditions for Conan
The check-conanfile-changes logic prevented cache saves on feature
branches unless conanfile changed, but this was broken because:
1. Cache keys don't include branch name (shared across branches)
2. S3 immutability makes this unnecessary (first-write-wins)
3. It prevented bootstrap saves on feature branches

Also removed unused safe-branch step (dead code from copy-paste)
2025-10-30 10:35:37 +07:00
Nicholas Dudfield
a0c38a4fb3 fix: disable deltas for Conan cache (base-only mode)
Conan dependencies are relatively static, so use base-only mode
instead of delta caching to avoid unnecessary complexity
2025-10-30 10:03:48 +07:00
Nicholas Dudfield
631650f7eb feat(wip): add nd-experiment-overlayfs-2025-10-29 to nix push 2025-10-30 09:56:33 +07:00
Nicholas Dudfield
0b31d8e534 feat: replace actions/cache with custom S3+OverlayFS cache
- Updated xahau-ga-dependencies to use xahau-actions-cache-restore/save
- Updated xahau-ga-build to use custom cache for ccache (4 operations)
- Added AWS credential inputs to both actions
- Main workflow now passes AWS secrets to cache actions
- Removed legacy ~/.conan path (Conan 2 uses ~/.conan2 only)
- All cache keys remain unchanged for compatibility
2025-10-30 09:55:06 +07:00
Nicholas Dudfield
ecf03f4afe test: expect state 1 [ci-clear-cache] 2025-10-30 09:50:57 +07:00
Nicholas Dudfield
b801c2837d test: expect state 3 2025-10-30 09:46:17 +07:00
Nicholas Dudfield
1474e808cb test: expect state 2 2025-10-30 09:43:26 +07:00
Nicholas Dudfield
457e633a81 feat(test): add workflow_dispatch inputs for state machine testing
Instead of requiring commits with message tags, allow manual workflow
triggering with inputs:
- state_assertion: Expected state for validation
- start_state: Force specific starting state
- clear_cache: Clear cache before running

Benefits:
- No need for empty commits to trigger tests
- Easy manual testing via GitHub Actions UI
- Still supports commit message tags for push events
- Workflow inputs take priority over commit tags
2025-10-30 09:40:26 +07:00
Nicholas Dudfield
7ea99caa19 fix(cache): trim whitespace in delta count + use s3api for tagging
Two fixes:
1. Restore: Trim whitespace from DELTA_COUNT to fix integer comparison
2. Save: Use 's3api put-object' instead of 's3 cp' to support --tagging

The aws s3 cp command doesn't support --tagging parameter.
Switched to s3api put-object which supports tagging directly.
Tags are needed for lifecycle policies (30-day eviction).
2025-10-30 09:29:02 +07:00
Nicholas Dudfield
3e5c15c172 fix(cache): handle empty cache gracefully in [ci-clear-cache]
When [ci-clear-cache] is used but no cache exists yet, grep returns
exit code 1 which causes script failure with set -e.

Add || echo "0" to handle case where no deltas exist to delete.
2025-10-30 09:25:21 +07:00
Nicholas Dudfield
52b4fb503c feat(cache): implement [ci-clear-cache] tag + auto-detecting state machine test
Cache Clearing Feature:
- Add [ci-clear-cache] commit message tag detection in restore action
- Deletes base + all deltas when tag present
- Implicit access via github.event.head_commit.message env var
- No workflow changes needed (action handles automatically)
- Commit message only (not PR title) - one-time action

State Machine Test Workflow:
- Auto-detects state by counting state files (state0.txt, state1.txt, etc.)
- Optional [state:N] assertions validate detected == expected
- [start-state:N] forces specific state for scenario testing
- Dual validation: local cache state AND S3 objects
- 4 validation checkpoints: S3 before, local after restore, after build, S3 after save
- Self-documenting: prints next steps after each run
- Supports [ci-clear-cache] integration

Usage:
  # Auto-advance (normal)
  git commit -m 'continue testing'

  # With assertion
  git commit -m 'test delta [state:2]'

  # Clear and restart
  git commit -m 'fresh start [ci-clear-cache]'

  # Jump to scenario
  git commit -m 'test from state 3 [start-state:3]'
2025-10-30 09:19:12 +07:00
Nicholas Dudfield
98123fa934 feat(cache): implement inline delta cleanup (keep only 1 per key)
- Add automatic cleanup after each delta upload
- Query all deltas for key, sort by LastModified
- Keep only latest (just uploaded), delete all older ones
- Matches restore logic (only uses latest delta)
- Minimal storage: 1 delta per key (~2GB) vs unbounded growth
- Simpler than keeping N: restore never needs older deltas
- Concurrency-safe (idempotent batch deletes)
- Eliminates need for separate cleanup workflow

Rationale: Since restore only ever uses the single latest delta,
keeping historical deltas adds complexity without benefit. This
matches GitHub Actions semantics (one 'latest' per key).
2025-10-29 14:51:28 +07:00
Nicholas Dudfield
ce7b1c4f1d feat: add custom S3+OverlayFS cache actions with configurable delta support
Implements drop-in replacement for actions/cache using S3 backend and OverlayFS for delta caching:

- xahau-actions-cache-restore: Downloads immutable base + optional latest delta
- xahau-actions-cache-save: Saves immutable bases (bootstrap/partial-match) or timestamped deltas (exact-match)

Key features:
- Immutable bases: One static base per key (first-write-wins, GitHub Actions semantics)
- Timestamped deltas: Always-timestamped to eliminate concurrency issues
- Configurable use-deltas parameter (default true):
  - true: For symbolic keys (branch-based) - massive bandwidth savings via incremental deltas
  - false: For content-based keys (hash-based) - base-only mode, no delta complexity
- Three cache modes: bootstrap, partial-match (restore-keys), exact-match
- OverlayFS integration: Automatic delta extraction via upperdir, whiteout file support
- S3 lifecycle ready: Bases tagged 'type=base', deltas tagged 'type=delta-archive'

Decision rule for use-deltas:
- Content-based discriminator (hashFiles, commit SHA) → use-deltas: false
- Symbolic discriminator (branch name, tag, PR) → use-deltas: true

Also disables existing workflows temporarily during development.
2025-10-29 13:07:40 +07:00
Nicholas Dudfield
e062dcae58 feat(wip): comment out unused secret encryption 2025-10-29 09:05:17 +07:00
Nicholas Dudfield
a9d284fec1 feat(wip): use new key names 2025-10-29 09:02:24 +07:00
Nicholas Dudfield
065d0c3e07 feat(wip): remove currently unused workflows 2025-10-29 08:57:21 +07:00
Nicholas Dudfield
4fda40b709 test: add S3 upload to overlayfs delta test
- Upload delta tarball to S3 bucket
- Test file: hello-world-first-test.tar.gz
- Uses new secret names: XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_KEY_ID/ACCESS_KEY
- Verifies upload with aws s3 ls
- Complete end-to-end test: OverlayFS → tarball → S3
2025-10-29 08:49:05 +07:00
Nicholas Dudfield
6014356d91 test: add encrypted secrets test to overlayfs workflow
- Generate random encryption key and store in GitHub Secrets via gh CLI
- Encrypt test message with GPG and commit to repo
- Decrypt in workflow using key from secrets and echo result
- Demonstrates encrypted secrets approach for SSH keys
2025-10-29 08:04:28 +07:00
Nicholas Dudfield
d790f97430 feat(wip): experiment overlayfs 2025-10-29 07:52:06 +07:00
14 changed files with 486 additions and 191 deletions

View File

@@ -1,5 +1,4 @@
name: 'Xahau Cache Restore (S3)'
bump: 1
description: 'Drop-in replacement for actions/cache/restore using S3 storage'
inputs:
@@ -75,93 +74,14 @@ runs:
echo "S3 bucket: s3://${S3_BUCKET}"
echo ""
# Normalize target path (expand tilde)
# Normalize target path (expand tilde and resolve to absolute path)
if [[ "${TARGET_PATH}" == ~* ]]; then
TARGET_PATH="${HOME}${TARGET_PATH:1}"
fi
# Canonicalize path (Linux only - macOS realpath doesn't support -m)
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
TARGET_PATH=$(realpath -m "${TARGET_PATH}")
fi
TARGET_PATH=$(realpath -m "${TARGET_PATH}")
echo "Normalized target path: ${TARGET_PATH}"
echo ""
# Debug: Show commit message
echo "=========================================="
echo "DEBUG: Cache clear tag detection"
echo "=========================================="
echo "Raw commit message:"
echo "${XAHAU_GA_COMMIT_MSG}"
echo ""
echo "Searching for: [ci-ga-clear-cache] or [ci-ga-clear-cache:*]"
echo ""
# Check for [ci-ga-clear-cache] tag in commit message (with optional search terms)
# Examples:
# [ci-ga-clear-cache] - Clear this job's cache
# [ci-ga-clear-cache:ccache] - Clear only if key contains "ccache"
# [ci-ga-clear-cache:gcc Debug] - Clear only if key contains both "gcc" AND "Debug"
# Extract search terms if present (e.g., "ccache" from "[ci-ga-clear-cache:ccache]")
SEARCH_TERMS=$(echo "${XAHAU_GA_COMMIT_MSG}" | grep -o '\[ci-ga-clear-cache:[^]]*\]' | sed 's/\[ci-ga-clear-cache://;s/\]//' || echo "")
SHOULD_CLEAR=false
if [ -n "${SEARCH_TERMS}" ]; then
# Search terms provided - check if THIS cache key matches ALL terms (AND logic)
echo "🔍 [ci-ga-clear-cache:${SEARCH_TERMS}] detected"
echo "Checking if cache key matches search terms..."
echo " Cache key: ${CACHE_KEY}"
echo " Search terms: ${SEARCH_TERMS}"
echo ""
MATCHES=true
for term in ${SEARCH_TERMS}; do
if ! echo "${CACHE_KEY}" | grep -q "${term}"; then
MATCHES=false
echo " ✗ Key does not contain '${term}'"
break
else
echo " ✓ Key contains '${term}'"
fi
done
if [ "${MATCHES}" = "true" ]; then
echo ""
echo "✅ Cache key matches all search terms - will clear cache"
SHOULD_CLEAR=true
else
echo ""
echo "⏭️ Cache key doesn't match search terms - skipping cache clear"
fi
elif echo "${XAHAU_GA_COMMIT_MSG}" | grep -q '\[ci-ga-clear-cache\]'; then
# No search terms - always clear this job's cache
echo "🗑️ [ci-ga-clear-cache] detected in commit message"
echo "Clearing cache for key: ${CACHE_KEY}"
SHOULD_CLEAR=true
fi
if [ "${SHOULD_CLEAR}" = "true" ]; then
echo ""
# Delete base layer
S3_BASE_KEY="s3://${S3_BUCKET}/${CACHE_KEY}-base.tar.zst"
if aws s3 ls "${S3_BASE_KEY}" --region "${S3_REGION}" >/dev/null 2>&1; then
echo "Deleting base layer: ${S3_BASE_KEY}"
aws s3 rm "${S3_BASE_KEY}" --region "${S3_REGION}" 2>/dev/null || true
echo "✓ Base layer deleted"
else
echo " No base layer found to delete"
fi
echo ""
echo "✅ Cache cleared successfully"
echo "Build will proceed from scratch (bootstrap mode)"
echo ""
fi
# Function to try restoring a cache key
try_restore_key() {
local key=$1
@@ -254,7 +174,7 @@ runs:
echo ""
echo "Downloading cache..."
aws s3 cp "${S3_KEY}" "${TEMP_TARBALL}" --region "${S3_REGION}" --no-progress
aws s3 cp "${S3_KEY}" "${TEMP_TARBALL}" --region "${S3_REGION}"
TARBALL_SIZE=$(du -h "${TEMP_TARBALL}" | cut -f1)
echo "✓ Downloaded: ${TARBALL_SIZE}"

View File

@@ -95,7 +95,7 @@ runs:
--body "${BASE_TARBALL}" \
--tagging 'type=base' \
--region "${S3_REGION}" \
>/dev/null 2>&1
>/dev/null
echo "✓ Uploaded: ${S3_BASE_KEY}"

View File

@@ -80,7 +80,7 @@ runs:
- name: Restore ccache directory
if: inputs.ccache_enabled == 'true'
id: ccache-restore
uses: ./.github/actions/xahau-ga-cache-restore
uses: ./.github/actions/xahau-actions-cache-restore
with:
path: ~/.ccache
key: ${{ runner.os }}-ccache-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ inputs.configuration }}-${{ steps.safe-branch.outputs.name }}
@@ -208,15 +208,7 @@ runs:
shell: bash
run: |
cd ${{ inputs.build_dir }}
# Check for verbose build flag in commit message
VERBOSE_FLAG=""
if echo "${XAHAU_GA_COMMIT_MSG}" | grep -q '\[ci-ga-cmake-verbose\]'; then
echo "🔊 [ci-ga-cmake-verbose] detected - enabling verbose output"
VERBOSE_FLAG="-- -v"
fi
cmake --build . --config ${{ inputs.configuration }} --parallel $(nproc) ${VERBOSE_FLAG}
cmake --build . --config ${{ inputs.configuration }} --parallel $(nproc) -- -v
- name: Show ccache statistics
if: inputs.ccache_enabled == 'true'
@@ -224,8 +216,8 @@ runs:
run: ccache -s
- name: Save ccache directory
if: success() && inputs.ccache_enabled == 'true'
uses: ./.github/actions/xahau-ga-cache-save
if: always() && inputs.ccache_enabled == 'true'
uses: ./.github/actions/xahau-actions-cache-save
with:
path: ~/.ccache
key: ${{ runner.os }}-ccache-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ inputs.configuration }}-${{ steps.safe-branch.outputs.name }}

View File

@@ -73,7 +73,7 @@ runs:
- name: Restore Conan cache
if: inputs.cache_enabled == 'true'
id: cache-restore-conan
uses: ./.github/actions/xahau-ga-cache-restore
uses: ./.github/actions/xahau-actions-cache-restore
with:
path: ~/.conan2
# Note: compiler-id format is compiler-version-stdlib[-gccversion]
@@ -160,8 +160,8 @@ runs:
..
- name: Save Conan cache
if: success() && inputs.cache_enabled == 'true' && steps.cache-restore-conan.outputs.cache-hit != 'true'
uses: ./.github/actions/xahau-ga-cache-save
if: always() && inputs.cache_enabled == 'true' && steps.cache-restore-conan.outputs.cache-hit != 'true'
uses: ./.github/actions/xahau-actions-cache-save
with:
path: ~/.conan2
key: ${{ runner.os }}-conan-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ hashFiles('**/conanfile.txt', '**/conanfile.py') }}-${{ inputs.configuration }}

View File

@@ -1,73 +0,0 @@
name: 'Get Commit Message'
description: 'Gets commit message for both push and pull_request events and sets XAHAU_GA_COMMIT_MSG env var'
inputs:
event-name:
description: 'The event name (push or pull_request)'
required: true
head-commit-message:
description: 'The head commit message (for push events)'
required: false
default: ''
pr-head-sha:
description: 'The PR head SHA (for pull_request events)'
required: false
default: ''
runs:
using: 'composite'
steps:
- name: Get commit message and set environment variable
shell: python
run: |
import os
import subprocess
import secrets
event_name = "${{ inputs.event-name }}"
pr_head_sha = "${{ inputs.pr-head-sha }}"
print("==========================================")
print("Setting XAHAU_GA_COMMIT_MSG environment variable")
print("==========================================")
print(f"Event: {event_name}")
if event_name == 'push':
# For push events, use the input directly
message = """${{ inputs.head-commit-message }}"""
print("Source: workflow input (github.event.head_commit.message)")
elif event_name == 'pull_request':
# For PR events, fetch the specific SHA
print(f"Source: git show {pr_head_sha} (fetching PR head commit)")
# Fetch the PR head commit
subprocess.run(
['git', 'fetch', 'origin', pr_head_sha],
check=True
)
# Get commit message from the fetched SHA
result = subprocess.run(
['git', 'show', '-s', '--format=%B', pr_head_sha],
capture_output=True,
text=True,
check=True
)
message = result.stdout.strip()
else:
message = ""
print(f"Warning: Unknown event type: {event_name}")
print(f"Commit message (first 100 chars): {message[:100]}")
# Write to GITHUB_ENV using heredoc with random delimiter (prevents injection attacks)
# See: https://securitylab.github.com/resources/github-actions-untrusted-input/
delimiter = f"EOF_{secrets.token_hex(16)}"
with open(os.environ['GITHUB_ENV'], 'a') as f:
f.write(f'XAHAU_GA_COMMIT_MSG<<{delimiter}\n')
f.write(message)
f.write(f'\n{delimiter}\n')
print(f"✓ XAHAU_GA_COMMIT_MSG set (available to all subsequent steps)")
print("==========================================")

View File

@@ -0,0 +1,290 @@
name: Test Cache Actions (State Machine)
on:
push:
branches: ["nd-experiment-overlayfs-*"]
workflow_dispatch:
inputs:
state_assertion:
description: 'Expected state (optional, e.g. "2" to assert state 2)'
required: false
type: string
default: '1'
start_state:
description: 'Force specific starting state (optional, e.g. "3" to start at state 3)'
required: false
type: string
clear_cache:
description: 'Clear cache before running'
required: false
type: boolean
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test-cache-state-machine:
runs-on: ubuntu-latest
env:
CACHE_KEY: test-state-machine-${{ github.ref_name }}
CACHE_DIR: /tmp/test-cache
S3_BUCKET: xahaud-github-actions-cache-niq
S3_REGION: us-east-1
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Parse Inputs (workflow_dispatch or commit message)
id: parse-inputs
run: |
# Priority 1: workflow_dispatch inputs (manual trigger)
STATE_ASSERTION="${{ inputs.state_assertion }}"
START_STATE="${{ inputs.start_state }}"
SHOULD_CLEAR="${{ inputs.clear_cache }}"
# Priority 2: commit message tags (push event)
if [ "${{ github.event_name }}" = "push" ]; then
COMMIT_MSG="${{ github.event.head_commit.message }}"
# Parse [state:N] assertion tag (optional, if not provided as input)
if [ -z "${STATE_ASSERTION}" ] && echo "${COMMIT_MSG}" | grep -qE '\[state:[0-9]+\]'; then
STATE_ASSERTION=$(echo "${COMMIT_MSG}" | grep -oE '\[state:[0-9]+\]' | grep -oE '[0-9]+')
echo "State assertion found in commit: ${STATE_ASSERTION}"
fi
# Parse [start-state:N] force tag (optional, if not provided as input)
if [ -z "${START_STATE}" ] && echo "${COMMIT_MSG}" | grep -qE '\[start-state:[0-9]+\]'; then
START_STATE=$(echo "${COMMIT_MSG}" | grep -oE '\[start-state:[0-9]+\]' | grep -oE '[0-9]+')
echo "Start state found in commit: ${START_STATE}"
fi
# Parse [ci-clear-cache] tag (if not provided as input)
if [ "${SHOULD_CLEAR}" != "true" ] && echo "${COMMIT_MSG}" | grep -q '\[ci-clear-cache\]'; then
SHOULD_CLEAR=true
echo "Cache clear requested in commit"
fi
fi
# Output final values
echo "state_assertion=${STATE_ASSERTION}" >> "$GITHUB_OUTPUT"
echo "start_state=${START_STATE}" >> "$GITHUB_OUTPUT"
echo "should_clear=${SHOULD_CLEAR}" >> "$GITHUB_OUTPUT"
# Log what we're using
echo ""
echo "Configuration:"
[ -n "${STATE_ASSERTION}" ] && echo " State assertion: ${STATE_ASSERTION}"
[ -n "${START_STATE}" ] && echo " Start state: ${START_STATE}"
echo " Clear cache: ${SHOULD_CLEAR}"
- name: Check S3 State (Before Restore)
env:
AWS_ACCESS_KEY_ID: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_ACCESS_KEY }}
run: |
echo "=========================================="
echo "S3 State Check (Before Restore)"
echo "=========================================="
echo "Cache key: ${CACHE_KEY}"
echo ""
# Check if base exists
BASE_EXISTS=false
if aws s3 ls "s3://${S3_BUCKET}/${CACHE_KEY}-base.tar.zst" --region "${S3_REGION}" >/dev/null 2>&1; then
BASE_EXISTS=true
fi
echo "Base exists: ${BASE_EXISTS}"
# Count deltas
DELTA_COUNT=$(aws s3 ls "s3://${S3_BUCKET}/" --region "${S3_REGION}" | grep "${CACHE_KEY}-delta-" | wc -l || echo "0")
echo "Delta count: ${DELTA_COUNT}"
- name: Restore Cache
uses: ./.github/actions/xahau-actions-cache-restore
with:
path: ${{ env.CACHE_DIR }}
key: ${{ env.CACHE_KEY }}
s3-bucket: ${{ env.S3_BUCKET }}
s3-region: ${{ env.S3_REGION }}
use-deltas: 'true'
aws-access-key-id: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_KEY_ID }}
aws-secret-access-key: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_ACCESS_KEY }}
- name: Auto-Detect State and Validate
id: state
env:
STATE_ASSERTION: ${{ steps.parse-inputs.outputs.state_assertion }}
START_STATE: ${{ steps.parse-inputs.outputs.start_state }}
run: |
echo "=========================================="
echo "State Detection and Validation"
echo "=========================================="
# Create cache directory if it doesn't exist
mkdir -p "${CACHE_DIR}"
# Handle [start-state:N] - force specific state
if [ -n "${START_STATE}" ]; then
echo "🎯 [start-state:${START_STATE}] detected - forcing state setup"
# Clear cache and create state files 0 through START_STATE
rm -f ${CACHE_DIR}/state*.txt 2>/dev/null || true
for i in $(seq 0 ${START_STATE}); do
echo "State ${i} - Forced at $(date)" > "${CACHE_DIR}/state${i}.txt"
echo "Commit: ${{ github.sha }}" >> "${CACHE_DIR}/state${i}.txt"
done
DETECTED_STATE=${START_STATE}
echo "✓ Forced to state ${DETECTED_STATE}"
else
# Auto-detect state by counting state files
STATE_FILES=$(ls ${CACHE_DIR}/state*.txt 2>/dev/null | wc -l)
DETECTED_STATE=${STATE_FILES}
echo "Auto-detected state: ${DETECTED_STATE} (${STATE_FILES} state files)"
fi
# Show cache contents
echo ""
echo "Cache contents:"
if [ -d "${CACHE_DIR}" ] && [ "$(ls -A ${CACHE_DIR})" ]; then
ls -la "${CACHE_DIR}"
else
echo "(empty)"
fi
# Validate [state:N] assertion if provided
if [ -n "${STATE_ASSERTION}" ]; then
echo ""
echo "Validating assertion: [state:${STATE_ASSERTION}]"
if [ "${DETECTED_STATE}" -ne "${STATE_ASSERTION}" ]; then
echo "❌ ERROR: State mismatch!"
echo " Expected (from [state:N]): ${STATE_ASSERTION}"
echo " Detected (from cache): ${DETECTED_STATE}"
exit 1
fi
echo "✓ Assertion passed: detected == expected (${DETECTED_STATE})"
fi
# Output detected state for next steps
echo "detected_state=${DETECTED_STATE}" >> "$GITHUB_OUTPUT"
echo ""
echo "=========================================="
- name: Simulate Build (State Transition)
env:
DETECTED_STATE: ${{ steps.state.outputs.detected_state }}
run: |
echo "=========================================="
echo "Simulating Build (State Transition)"
echo "=========================================="
# Calculate next state
NEXT_STATE=$((DETECTED_STATE + 1))
echo "Transitioning: State ${DETECTED_STATE} → State ${NEXT_STATE}"
echo ""
# Create state file for next state
STATE_FILE="${CACHE_DIR}/state${NEXT_STATE}.txt"
echo "State ${NEXT_STATE} - Created at $(date)" > "${STATE_FILE}"
echo "Commit: ${{ github.sha }}" >> "${STATE_FILE}"
echo "Message: ${{ github.event.head_commit.message }}" >> "${STATE_FILE}"
echo "✓ Created ${STATE_FILE}"
# Show final cache state
echo ""
echo "Final cache contents:"
ls -la "${CACHE_DIR}"
echo ""
echo "State files:"
cat ${CACHE_DIR}/state*.txt
- name: Save Cache
uses: ./.github/actions/xahau-actions-cache-save
with:
path: ${{ env.CACHE_DIR }}
key: ${{ env.CACHE_KEY }}
s3-bucket: ${{ env.S3_BUCKET }}
s3-region: ${{ env.S3_REGION }}
use-deltas: 'true'
aws-access-key-id: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_KEY_ID }}
aws-secret-access-key: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_ACCESS_KEY }}
- name: Validate S3 State (After Save)
env:
AWS_ACCESS_KEY_ID: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_ACCESS_KEY }}
DETECTED_STATE: ${{ steps.state.outputs.detected_state }}
run: |
echo "=========================================="
echo "S3 State Validation (After Save)"
echo "=========================================="
# Calculate next state (what we just saved)
NEXT_STATE=$((DETECTED_STATE + 1))
echo "Saved state: ${NEXT_STATE}"
echo ""
# Check if base exists
if aws s3 ls "s3://${S3_BUCKET}/${CACHE_KEY}-base.tar.zst" --region "${S3_REGION}" >/dev/null 2>&1; then
BASE_SIZE=$(aws s3 ls "s3://${S3_BUCKET}/${CACHE_KEY}-base.tar.zst" --region "${S3_REGION}" | awk '{print $3}')
echo "✓ Base exists: ${CACHE_KEY}-base.tar.zst (${BASE_SIZE} bytes)"
else
echo "❌ ERROR: Base should exist after save"
exit 1
fi
# List deltas
echo ""
echo "Delta layers:"
DELTAS=$(aws s3 ls "s3://${S3_BUCKET}/" --region "${S3_REGION}" | grep "${CACHE_KEY}-delta-" || echo "")
if [ -n "${DELTAS}" ]; then
echo "${DELTAS}"
DELTA_COUNT=$(echo "${DELTAS}" | wc -l)
else
echo "(none)"
DELTA_COUNT=0
fi
# Validate S3 state
echo ""
if [ "${DETECTED_STATE}" -eq 0 ]; then
# Saved state 1 from bootstrap (state 0 → 1)
if [ "${DELTA_COUNT}" -ne 0 ]; then
echo "⚠️ WARNING: Bootstrap (state 1) should have 0 deltas, found ${DELTA_COUNT}"
else
echo "✓ State 1 saved: base exists, 0 deltas"
fi
else
# Saved delta (state N+1)
if [ "${DELTA_COUNT}" -ne 1 ]; then
echo "⚠️ WARNING: State ${NEXT_STATE} expects 1 delta (inline cleanup), found ${DELTA_COUNT}"
echo "This might be OK if multiple builds ran concurrently"
else
echo "✓ State ${NEXT_STATE} saved: base + 1 delta (old deltas cleaned)"
fi
fi
echo ""
echo "=========================================="
echo "✅ State ${DETECTED_STATE} → ${NEXT_STATE} Complete!"
echo "=========================================="
echo ""
echo "Next commit will auto-detect state ${NEXT_STATE}"
echo ""
echo "Options:"
echo " # Normal (auto-advance)"
echo " git commit -m 'continue testing'"
echo ""
echo " # With assertion (validate state)"
echo " git commit -m 'test delta [state:${NEXT_STATE}]'"
echo ""
echo " # Clear cache and restart"
echo " git commit -m 'fresh start [ci-clear-cache]'"
echo ""
echo " # Jump to specific state"
echo " git commit -m 'jump to state 3 [start-state:3]'"

View File

@@ -0,0 +1,182 @@
name: Test OverlayFS Delta Extraction
on:
push:
branches: ["*"]
workflow_dispatch:
jobs:
test-overlayfs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
# - name: Test encrypted secrets (decrypt test message)
# run: |
# echo "========================================"
# echo "TESTING ENCRYPTED SECRETS"
# echo "========================================"
# echo ""
# echo "Decrypting test message from .github/secrets/test-message.gpg"
# echo "Using encryption key from GitHub Secrets..."
# echo ""
#
# # Decrypt using key from GitHub Secrets
# echo "${{ secrets.TEST_ENCRYPTION_KEY }}" | \
# gpg --batch --yes --passphrase-fd 0 \
# --decrypt .github/secrets/test-message.gpg
#
# echo ""
# echo "========================================"
# echo "If you see the success message above,"
# echo "then encrypted secrets work! 🎉"
# echo "========================================"
# echo ""
- name: Setup OverlayFS layers
run: |
echo "=== Creating directory structure ==="
mkdir -p /tmp/test/{base,delta,upper,work,merged}
echo "=== Creating base layer files ==="
echo "base file 1" > /tmp/test/base/file1.txt
echo "base file 2" > /tmp/test/base/file2.txt
echo "base file 3" > /tmp/test/base/file3.txt
mkdir -p /tmp/test/base/subdir
echo "base subdir file" > /tmp/test/base/subdir/file.txt
echo "=== Base layer contents ==="
find /tmp/test/base -type f -exec sh -c 'echo "{}:"; cat "{}"' \;
echo "=== Mounting OverlayFS ==="
sudo mount -t overlay overlay \
-o lowerdir=/tmp/test/base,upperdir=/tmp/test/upper,workdir=/tmp/test/work \
/tmp/test/merged
echo "=== Mounted successfully ==="
mount | grep overlay
- name: Verify merged view shows base files
run: |
echo "=== Contents of /merged (should show base files) ==="
ls -R /tmp/test/merged
find /tmp/test/merged -type f -exec sh -c 'echo "{}:"; cat "{}"' \;
- name: Make changes via merged layer
run: |
echo "=== Making changes via /merged ==="
# Overwrite existing file
echo "MODIFIED file 2" > /tmp/test/merged/file2.txt
echo "Modified file2.txt"
# Create new file
echo "NEW file 4" > /tmp/test/merged/file4.txt
echo "Created new file4.txt"
# Create new directory with file
mkdir -p /tmp/test/merged/newdir
echo "NEW file in new dir" > /tmp/test/merged/newdir/newfile.txt
echo "Created newdir/newfile.txt"
# Add file to existing directory
echo "NEW file in existing subdir" > /tmp/test/merged/subdir/newfile.txt
echo "Created subdir/newfile.txt"
echo "=== Changes complete ==="
- name: Show the delta (upperdir)
run: |
echo "========================================"
echo "THE DELTA (only changes in /upper):"
echo "========================================"
if [ -z "$(ls -A /tmp/test/upper)" ]; then
echo "Upper directory is empty - no changes detected"
else
echo "Upper directory structure:"
ls -R /tmp/test/upper
echo ""
echo "Upper directory files with content:"
find /tmp/test/upper -type f -exec sh -c 'echo "---"; echo "FILE: {}"; cat "{}"; echo ""' \;
echo "========================================"
echo "SIZE OF DELTA:"
du -sh /tmp/test/upper
echo "========================================"
fi
- name: Compare base vs upper vs merged
run: |
echo "========================================"
echo "COMPARISON:"
echo "========================================"
echo "BASE layer (original, untouched):"
ls -la /tmp/test/base/
echo ""
echo "UPPER layer (DELTA - only changes):"
ls -la /tmp/test/upper/
echo ""
echo "MERGED layer (unified view = base + upper):"
ls -la /tmp/test/merged/
echo ""
echo "========================================"
echo "PROOF: Upper dir contains ONLY the delta!"
echo "========================================"
- name: Simulate tarball creation (what we'd upload)
run: |
echo "=== Creating tarball of delta ==="
tar -czf /tmp/delta.tar.gz -C /tmp/test/upper .
echo "Delta tarball size:"
ls -lh /tmp/delta.tar.gz
echo ""
echo "Delta tarball contents:"
tar -tzf /tmp/delta.tar.gz
echo ""
echo "========================================"
echo "This is what we'd upload to S3/rsync!"
echo "Only ~few KB instead of entire cache!"
echo "========================================"
- name: Upload delta to S3 (actual test!)
env:
AWS_ACCESS_KEY_ID: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_AWS_ACCESS_KEY }}
run: |
echo "========================================"
echo "UPLOADING TO S3"
echo "========================================"
# Upload the delta tarball
aws s3 cp /tmp/delta.tar.gz \
s3://xahaud-github-actions-cache-niq/hello-world-first-test.tar.gz \
--region us-east-1
echo ""
echo "✅ Successfully uploaded to S3!"
echo "File: s3://xahaud-github-actions-cache-niq/hello-world-first-test.tar.gz"
echo ""
# Verify it exists
echo "Verifying upload..."
aws s3 ls s3://xahaud-github-actions-cache-niq/hello-world-first-test.tar.gz --region us-east-1
echo ""
echo "========================================"
echo "S3 upload test complete! 🚀"
echo "========================================"
- name: Cleanup
if: always()
run: |
echo "=== Unmounting OverlayFS ==="
sudo umount /tmp/test/merged || true

View File

@@ -30,14 +30,6 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Get commit message
id: get-commit-message
uses: ./.github/actions/xahau-ga-get-commit-message
with:
event-name: ${{ github.event_name }}
head-commit-message: ${{ github.event.head_commit.message }}
pr-head-sha: ${{ github.event.pull_request.head.sha }}
- name: Install Conan
run: |
brew install conan

View File

@@ -162,14 +162,6 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Get commit message
id: get-commit-message
uses: ./.github/actions/xahau-ga-get-commit-message
with:
event-name: ${{ github.event_name }}
head-commit-message: ${{ github.event.head_commit.message }}
pr-head-sha: ${{ github.event.pull_request.head.sha }}
- name: Install build dependencies
run: |
sudo apt-get update

View File

@@ -72,15 +72,15 @@ It generates many files of [results](results):
desired as described above. In a perfect repo, this file will be
empty.
This file is committed to the repo, and is used by the [levelization
Github workflow](../../.github/workflows/levelization.yml) to validate
Github workflow](../../.github/workflows/levelization.yml.disabled) to validate
that nothing changed.
* [`ordering.txt`](results/ordering.txt): A list showing relationships
between modules where there are no loops as they actually exist, as
opposed to how they are desired as described above.
This file is committed to the repo, and is used by the [levelization
Github workflow](../../.github/workflows/levelization.yml) to validate
Github workflow](../../.github/workflows/levelization.yml.disabled) to validate
that nothing changed.
* [`levelization.yml`](../../.github/workflows/levelization.yml)
* [`levelization.yml`](../../.github/workflows/levelization.yml.disabled)
Github Actions workflow to test that levelization loops haven't
changed. Unfortunately, if changes are detected, it can't tell if
they are improvements or not, so if you have resolved any issues or