Commit Graph

65 Commits

Author SHA1 Message Date
Nicholas Dudfield
1474e808cb test: expect state 2 2025-10-30 09:43:26 +07:00
Nicholas Dudfield
457e633a81 feat(test): add workflow_dispatch inputs for state machine testing
Instead of requiring commits with message tags, allow manual workflow
triggering with inputs:
- state_assertion: Expected state for validation
- start_state: Force specific starting state
- clear_cache: Clear cache before running

Benefits:
- No need for empty commits to trigger tests
- Easy manual testing via GitHub Actions UI
- Still supports commit message tags for push events
- Workflow inputs take priority over commit tags
2025-10-30 09:40:26 +07:00
Nicholas Dudfield
52b4fb503c feat(cache): implement [ci-clear-cache] tag + auto-detecting state machine test
Cache Clearing Feature:
- Add [ci-clear-cache] commit message tag detection in restore action
- Deletes base + all deltas when tag present
- Implicit access via github.event.head_commit.message env var
- No workflow changes needed (action handles automatically)
- Commit message only (not PR title) - one-time action

State Machine Test Workflow:
- Auto-detects state by counting state files (state0.txt, state1.txt, etc.)
- Optional [state:N] assertions validate detected == expected
- [start-state:N] forces specific state for scenario testing
- Dual validation: local cache state AND S3 objects
- 4 validation checkpoints: S3 before, local after restore, after build, S3 after save
- Self-documenting: prints next steps after each run
- Supports [ci-clear-cache] integration

Usage:
  # Auto-advance (normal)
  git commit -m 'continue testing'

  # With assertion
  git commit -m 'test delta [state:2]'

  # Clear and restart
  git commit -m 'fresh start [ci-clear-cache]'

  # Jump to scenario
  git commit -m 'test from state 3 [start-state:3]'
2025-10-30 09:19:12 +07:00
Nicholas Dudfield
ce7b1c4f1d feat: add custom S3+OverlayFS cache actions with configurable delta support
Implements drop-in replacement for actions/cache using S3 backend and OverlayFS for delta caching:

- xahau-actions-cache-restore: Downloads immutable base + optional latest delta
- xahau-actions-cache-save: Saves immutable bases (bootstrap/partial-match) or timestamped deltas (exact-match)

Key features:
- Immutable bases: One static base per key (first-write-wins, GitHub Actions semantics)
- Timestamped deltas: Always-timestamped to eliminate concurrency issues
- Configurable use-deltas parameter (default true):
  - true: For symbolic keys (branch-based) - massive bandwidth savings via incremental deltas
  - false: For content-based keys (hash-based) - base-only mode, no delta complexity
- Three cache modes: bootstrap, partial-match (restore-keys), exact-match
- OverlayFS integration: Automatic delta extraction via upperdir, whiteout file support
- S3 lifecycle ready: Bases tagged 'type=base', deltas tagged 'type=delta-archive'

Decision rule for use-deltas:
- Content-based discriminator (hashFiles, commit SHA) → use-deltas: false
- Symbolic discriminator (branch name, tag, PR) → use-deltas: true

Also disables existing workflows temporarily during development.
2025-10-29 13:07:40 +07:00
Nicholas Dudfield
e062dcae58 feat(wip): comment out unused secret encryption 2025-10-29 09:05:17 +07:00
Nicholas Dudfield
a9d284fec1 feat(wip): use new key names 2025-10-29 09:02:24 +07:00
Nicholas Dudfield
065d0c3e07 feat(wip): remove currently unused workflows 2025-10-29 08:57:21 +07:00
Nicholas Dudfield
4fda40b709 test: add S3 upload to overlayfs delta test
- Upload delta tarball to S3 bucket
- Test file: hello-world-first-test.tar.gz
- Uses new secret names: XAHAUD_GITHUB_ACTIONS_CACHE_NIQ_KEY_ID/ACCESS_KEY
- Verifies upload with aws s3 ls
- Complete end-to-end test: OverlayFS → tarball → S3
2025-10-29 08:49:05 +07:00
Nicholas Dudfield
6014356d91 test: add encrypted secrets test to overlayfs workflow
- Generate random encryption key and store in GitHub Secrets via gh CLI
- Encrypt test message with GPG and commit to repo
- Decrypt in workflow using key from secrets and echo result
- Demonstrates encrypted secrets approach for SSH keys
2025-10-29 08:04:28 +07:00
Nicholas Dudfield
d790f97430 feat(wip): experiment overlayfs 2025-10-29 07:52:06 +07:00
tequ
96222baf5e Add hook header generators and CI verification workflow (#597) 2025-10-22 15:25:38 +10:00
Niq Dudfield
83f09fd8ab ci: add clang to build matrix [ci-nix-full-matrix] (#569) 2025-10-15 11:26:31 +10:00
Niq Dudfield
fa1b93bfd8 build: migrate to conan 2 (#585)
Migrates the build system from Conan 1 to Conan 2
2025-10-10 14:57:46 +11:00
tequ
8f7ebf0377 Optimize github action cache (#544)
* optimize github action cache

* fix

* refactor: improve github actions cache optimization (#3)

- move ccache configuration logic to dedicated action
- rename conanfile-changed to should-save-conan-cache for clarity

---------

Co-authored-by: Niq Dudfield <ndudfield@gmail.com>
2025-09-08 15:53:40 +10:00
Niq Dudfield
7a790246fb fix: upgrade CI to GCC 13 and fix compilation issues, fixes #557 (#559) 2025-08-14 17:41:49 +10:00
tequ
2fc912d54d Make release build use conan deps where possible and hbb 4.0.1 (#516)
Co-authored-by: Denis Angell <dangell@transia.co>
Co-authored-by: Niq Dudfield <ndudfield@gmail.com>
2025-08-14 12:59:57 +10:00
Niq Dudfield
1233694b6c chore: add suspicious_patterns to .scripts/pre-hook and not-suspicious filter (#525)
* chore: add suspicious_patterns to .scripts/pre-hook and not-suspicious filter

* rm: kill annoying checkpatterns job

* chore: cleanup

---------

Co-authored-by: RichardAH <richard.holland@starstone.co.nz>
2025-07-01 20:58:06 +10:00
Denis Angell
a5ea86fdfc Add Conan Building For Development (#432) 2025-05-14 14:00:20 +10:00
tequ
989532702d Update clang-format workflow (#490) 2025-04-17 16:16:59 +10:00
Niq Dudfield
ef77b02d7f CI Release Builder (#455) 2025-03-11 13:19:28 +01:00
Wietse Wind
1fb1a99ea2 Update build-in-docker.yml 2025-02-05 08:23:49 +01:00
Wietse Wind
fa71bda29c Artifact v4 continue on error 2025-02-01 08:58:13 +01:00
Wietse Wind
412593d7bc Update artifact 2025-02-01 08:57:48 +01:00
Wietse Wind
12d8342c34 Update artifact 2025-02-01 08:57:25 +01:00
Denis Angell
79d83bd424 fix240911 (#363) 2024-09-11 13:43:03 +10:00
Wietse Wind
849a4435e0 CI Split jobs with prev job dependency & CI on jshooks (#320)
* CI on `jshooks` branch

* CI Split jobs with prev job dependency

* No multi branch worker in parallel

---------

Co-authored-by: Denis Angell <dangell@transia.co>
2024-05-29 13:45:59 +02:00
Wietse Wind
247e9d98bf CI on jshooks branch (#317) 2024-05-24 10:10:40 +10:00
Wietse Wind
caa8b382d8 🤦 2024-02-22 23:28:19 +01:00
Wietse Wind
82e04073be Revert checkout v3 2024-02-14 15:21:43 +01:00
Wietse Wind
e1b78f9682 Do clean 2024-02-14 15:20:24 +01:00
Wietse Wind
901d1d4e8d Update checkout CI to v4 2024-02-14 15:17:45 +01:00
Wietse Wind
66ee96d456 Build on release after all 2023-12-29 15:43:33 +01:00
Wietse Wind
b476aea55b Do not auto build on release 2023-12-29 15:39:48 +01:00
Denis Angell
b77b0e70e3 Add Script to check for suspicious patterns (#199)
* Create check_keys.sh

* add to workflow
2023-11-16 16:23:55 +01:00
Denis Angell
559b504c7d Update build-in-docker.yml (#196) 2023-11-09 19:44:28 +01:00
Denis Angell
43cb255337 Update Workflow (#193) 2023-11-09 19:01:50 +01:00
Denis Angell
2a18ec563d Fix HBB workflow (#191)
* Update build-in-docker.yml

* Update build-in-docker.yml

* Update build-in-docker.yml

* Update build-core.sh

* update workflow

* Update build-core.sh

* Update build-core.sh

* Update build-core.sh

* Update build-core.sh

* update workflow

* fix workflow

* Update xahaud.binary.dockerfile

* fixup

* fixup

* fixup

* fixup

* fixup

* fixup

* Update build-in-docker.yml
2023-11-09 12:20:51 +01:00
Denis Angell
04ceb5af9a update workflow (#175) 2023-11-01 17:04:13 +01:00
Denis Angell
70bd7c2ce7 Reintroduce Clang-Format & Levelization (#171)
* clang-format

* levelization

* clang-format

* update workflow (#172)

* update workflow

* Update build-in-docker.yml

* fix from `clang-format`

* Update Enum.h
2023-11-01 14:12:24 +01:00
Wietse Wind
dc29234ff5 Disable publisher: http://build.xahau.tech/ 2023-04-12 15:28:57 +02:00
Richard Holland
311d0b8bf1 Revert "Use the Conan package manager (#4367)"
This reverts commit c3a9f3dbf3.
2023-04-06 09:46:00 +00:00
Richard Holland
a5ca117ff6 Merge remote-tracking branch 'ripple/develop' into dev 2023-04-06 09:36:24 +00:00
RichardAH
466c7dba9f Update build-in-docker.yml
make auto build run on release branch
2023-03-11 07:59:38 +00:00
Elliot Lee
c2b03fecca Remove recipe for RocksDB and add recipe for Snappy (#4431)
- Copies the recipe for Snappy from Conan Center, but removes three
  lines that explicitly link the standard library, which prevents
  builders from statically linking it.
- Removes the recipe for RocksDB now that an official recipe for version
  6.27.3 is in Conan Center.

Developers will likely need to remove cached versions of both RocksDB
and Snappy:

```
conan remove -f rocksdb
conan remove -f snappy
```

---------

Co-authored-by: John Freeman <jfreeman08@gmail.com>
2023-02-23 12:41:18 -08:00
John Freeman
60c276d90b Fix Conan version constraint in workflows (#4430)
Conan 2.0 is not compatible with the current workflows.
2023-02-22 19:54:51 -08:00
Ed Hennis
aebf2ac990 Resolve a couple of Github Action CI annoyances: (#4413)
* Set "fail-fast: false" so that multiple jobs in one workflow can
  finish independently. By default, if one job fails, other running jobs
  will be aborted, even if the other jobs are working fine and are
  almost done. This leads to wasted time and resources if the failure
  is, for example, OS specific, or due to a flaky unit test, and the
  failed job needs to be re-run, because all the jobs end up re-running.
* Put conditions back into the windows.yml job (manual, and for
  a specific branch name and that job). This prevents Github Actions
  from sending "No jobs were run" failure emails on every commit.
2023-02-15 11:34:32 -08:00
Wietse Wind
c33f5b46a8 zipzip - zip 2023-02-09 14:49:34 +01:00
Wietse Wind
7fee0eb171 Fixes zip-zip 2023-02-09 13:29:46 +01:00
Wietse Wind
ba86191406 Publish built binary post builder, pre tests 2023-02-03 00:38:07 +01:00
Wietse Wind
bebf34311b Separate build & test job 2023-01-31 23:27:24 +01:00