Pools

Pools. A pool is the set of computers that runs your work, and you choose the pool for each job. On a hosted pool, TALOS Defence operates the computers in AWS Sydney, and you pay from prepaid credit. On a customer-controlled pool, the computers are in your own AWS account, and you pay for a licence and support, and you pay AWS for the instances.

1. The pool object

GET /v1/pools lists the pools your organisation can see, and GET /v1/pools/{id} gives one, by id or by name.

FieldWhat it holds
id, nameThe id of the pool, and the name of the pool, such as hosted-cpu-syd.
kindhosted, operated by TALOS Defence, or customer_controlled, in the customer's own environment.
stateavailable, registered (a customer-controlled pool that waits for a licence and keys) or retired.
platformsThe image platforms that the pool runs, such as linux/arm64.
gpunone, h100 or trn2.
regionThe AWS region of the pool. Each certificate from the pool shows the region.
placementon_demand (a fresh instance for each attempt), slot (inside a booked slot) or local (the customer's own runtime).
release_jobstrue when the pool runs releases.
identityThe identity that every return shows: provider, site, owner_nationality, jurisdiction and assurance (the assurance statement of the pool).
organisationThe owning organisation, for a customer-controlled pool.
deploymentFor a customer-controlled pool: provider (aws), account and region.
keysetFor a customer-controlled pool: the digest and issuer of the key set that signs the returns of the pool.
licenceFor a customer-controlled pool: the id, serial, state and end of the licence.
revocation_policyFor a customer-controlled pool: max_age_days and on_stale (Section 3.7).

Every certificate and every return shows the identity of the pool. A reader of a return sees where the job executed and under which jurisdiction, without asking Hangar.

2. The hosted pools

PoolWhat it runsPlacement
hosted-cpu-sydJobs of a release and CPU sessions, for linux/arm64, on AWS Graviton in Sydney.on_demand: each attempt uses one fresh instance for that attempt alone. Hangar terminates the instance when the attempt ends.
hosted-h100-sydBooked GPU jobs on one NVIDIA H100, for linux/amd64 with CUDA.slot: inside the hours that you booked on a night.

TALOS Defence Pty Ltd, which is 100% Australian-owned and controlled, operates both pools in AWS Asia Pacific (Sydney). The published key set of TALOS Defence signs the returns and certificates of both pools. You can verify them offline.

3. The customer-controlled form

The customer-controlled form runs the same releases inside the customer's own environment: the same package, release object, envelope and return, on capacity that the customer owns. TALOS Defence offers this form from 2027. Pool registration, licences, the local records contract and the offline bundles in this section are part of the Hangar control plane now. The runtime in the customer's environment is hangar-api --profile customer. The customer runtime explains how the runtime starts, checks the licence, imports releases and signs returns.

3.1 What the customer controls

The customer controlsIn practice
The environmentThe customer's own AWS account in ap-southeast-2: a network with no inbound path from the internet, the local control plane behind the customer's internal load balancer, a PostgreSQL database, and storage buckets that are versioned and locked against deletion.
The capacityThe customer's own On-Demand or Spot instances, on the customer's own AWS bill.
The identityThe roles in the customer's account. The roles trust principals in that account alone.
The keysThe records-signing root at the customer's own key site, the day keys that the customer attests, and every data protection key.
AdmissionLocal approval of each release and each widening, by a named person in the customer's organisation.
The dataEvery input, output, configuration value and secret, in the customer's own account.

3.2 Key custody

TALOS Defence keeps two keys for this form, and only two. The release signing key signs runtime releases and package acceptances. The entitlement key signs licences. The customer holds every other key: the root that signs the records, trees and returns of the customer, the day keys under the root, and the keys that protect the data of the customer. The device identity of each instance comes from the hardware of the instance.

The owner decides what runs, and TALOS Defence keeps only the proof of what TALOS Defence shipped and licensed. A return from a customer-controlled pool shows the organisation of the customer as the issuer. The keys of the customer sign the return. Anyone can verify the return offline against the published key set of the customer and the published release key set of TALOS Defence.

PUT /v1/pools/{id}/keyset records the key set of the customer for a registered pool. Hangar accepts a key set that its own root signs and that names the organisation of the customer as the issuer. Hangar refuses these key sets with keyset_issuer_refused: the record key set of TALOS Defence, and any set that shares a root with the record, release or entitlement key sets of TALOS Defence. After a pool has a key set, Hangar accepts a new key set only as a rotation: a root of the old set must sign the new set. Otherwise Hangar answers keyset_invalid.

3.3 Who operates what

The contract takes one of three forms. In every form, the keys and the data stay with the customer.

A: customer-operatedB: operated by TALOS DefenceC: operated by a partner
Installs and updatesThe customer's peopleTALOS Defence, in the customer's accountThe vendor or an Australian managed service provider, in the customer's account
Access to the accountThe customer's ownA role the customer issues per session: time-bound, approved by the customer, logged in the customer's audit trail, and denied every data bucket and every key operationAs B
Releases and documentationTALOS DefenceTALOS DefenceTALOS Defence, through the partner
SupportFrom diagnostics exports and the customer's descriptionAs A, with operation in the session the customer approvesAs B
Keys and dataThe customerThe customerThe customer

3.4 Registering a pool and its licence

hangar pool register --name harbour-aws --aws-account 123456789012
hangar pool licence harbour-aws

An owner registers a customer-controlled pool with POST /v1/pools. The request gives the name, the provider aws, the 12-digit AWS account id, the region ap-southeast-2, the platforms and, optionally, the revocation policy. The pool is registered until TALOS Defence issues the licence and the customer records the key set.

The licence is an entitlement file of the format hangar-entitlement-v1. The file is canonical CBOR, signed by the entitlement key of TALOS Defence. The licence lists:

The price of the licence depends on organisational scale, permitted deployments, supported capability and lifecycle commitments. GET /v1/pools/{id}/licence gives the fields of the licence. GET /v1/pools/{id}/licence/file gives the signed file.

The local runtime checks the licence offline, at start and once a day. The runtime reports nothing to TALOS Defence. No usage record leaves the customer's account for licensing, and a renewal is a new file. A licence lists its deployments, so the runtime refuses at start a licence for another account. A changed byte fails the signature.

When the term ends, the runtime keeps running the releases that are installed, shows the lapse on every return, and takes no new release import. After a further 90 days of grace, the runtime takes no new tenant.

3.5 The local records contract

A customer-controlled pool keeps the records, tree heads and returns of the pool in the customer's account. The customer chooses what to share. POST /v1/pools/{id}/returns imports one return that the customer's runtime signed. Hangar can then list the return beside hosted returns. Hangar verifies the return against the key set of the pool and the release key set of TALOS Defence, and stores the return with the verifier report. If the return does not verify, Hangar refuses the import with return_import_invalid. The import contains only the return.

Every return from a customer-controlled pool shows the licence of the runtime and the age of the revocation list that the runtime used. The verifier requires both. The import judges both against the licence of the pool that was in force when the last attempt of the job started, and against the revocation policy of the pool. The newest licence whose term had begun is the licence in force. The licence state must be the state that the term gives: valid, or lapsed, before or after the grace. The age of the list must meet the revocation policy of the pool. The import also refuses a return in three cases: the pool identity on the return is not the full identity of the pool; the last attempt starts after the return was issued; or the release was revoked before the list that the return says the runtime held. The import answer and the kept record show both results. A pool with no issued licence imports nothing (licence_not_found).

A vendor usually supplies the release that a customer-controlled pool runs. Hangar serves the release bundle (Section 3.6) to the vendor, and to an organisation that holds an issued licence for a customer-controlled pool. The release must be accepted and not revoked.

3.6 The release bundle and the site snapshot

A disconnected pool needs two documents. Each item in the two documents verifies offline on its own.

DocumentRouteWhat it holds
Release bundleGET /v1/releases/{id}/bundleThe signed release with the acceptance, the signed revocation list, the release key set of TALOS Defence, and the package manifest, byte for byte. The signed release lists the digest of the manifest.
Site snapshotGET /v1/pools/{id}/snapshotThe licence of the pool, the signed revocation list, the release key set and the entitlement key set.

The vendor ships the image layers, as an OCI layout on the customer's media. The runtime checks every layer against the digests in the signed release.

3.7 The disconnected operating policy

With no outside connection, everything in the customer profile works: jobs, the compile, returns, approvals and the local log. Each imported item allows one more thing:

ItemArrives asWhat it allows
A package releaseA release bundle and the vendor's OCI layout, on the customer's mediaJobs of that release, after the customer's local approval.
A runtime updateA signed runtime release, as an offline bundleAn update, after the customer's acceptance jobs and approval (Section 3.8).
The release revocation listInside each bundle and snapshotNew jobs while the list is younger than the policy's maximum age.
The licenceThe licence file, or a site snapshotRelease imports and new tenants for the licence's term.
The key setsInside each bundle and snapshotOffline checks of releases, acceptances and licences.

The revocation policy sets the maximum age of the list, from 1 to 365 days (30 by default). It also sets what happens after that age:

Time comes from the time source of the customer. Each return shows the error bound of the clock, measured against the time source. The gateway connects to the destinations that the network of the customer connects to, and refuses every other destination.

3.8 Updates and rollback

A runtime update installs only when the customer accepts the update:

  1. The customer imports a signed runtime release, from the channel that the licence lists or as an offline bundle.
  2. The local control plane installs the new release beside the running release.
  3. The control plane runs the conformance suite that ships with the release, and the acceptance jobs of the customer. The acceptance jobs are a list that the customer keeps of package jobs with their expected output checks.
  4. A named person in the customer's organisation approves.
  5. The control plane switches.

Every step is a record. A failed check stops the update and leaves the running release in place. A rollback returns to the previous release within the rollback window. The database migrations of each release are additive for one release, so the previous release still works on the new schema. Package releases follow the same path. The customer imports a release that TALOS Defence accepted and approves the release locally. The widening rule (Releases and jobs, Section 5) applies, and the customer is the approver.

3.9 The diagnostics export

When the customer wants support, the customer exports a diagnostics bundle. The customer previews every field before export. The customer signs the bundle with the day key of the customer and sends the bundle through the customer's own channel. The bundle contains:

3.10 Support boundaries

PartySupports
TALOS DefenceThe runtime release as released, on the supported profiles, until the end-of-life date of 5 November 2031. Support works from diagnostics exports and the description of the customer.
The vendorThe package and the application of the vendor.
The customerThe account, identity, network and keys of the customer, and the changes that the customer makes to the installation. TALOS Defence supports a modified installation again after the installation report passes.

Support works from diagnostics exports and descriptions. It never asks for payloads, secrets or screen sharing.

3.11 The runtime release

The release root of TALOS Defence signs each runtime release. The manifest of the runtime release has the format hangar-runtime-release-v1 and lists:

FieldWhat it holds
versionThe runtime version.
control planeThe digest of the local control plane built with the customer profile.
worker and gatewayThe digests of the worker image and the gateway image, each built twice and compared, with the SBOM digest and the boot measurements of each image.
installationThe digest of the installation template of each environment.
migrationsThe database migrations, in order, by digest.
package runtime rangeThe package runtime versions this release runs.
rollbackThe previous release that the customer can roll back to.

The installation verifier reads the deployed account through the read-only role of the customer. It writes a signed installation report. The report covers every artefact digest against the manifest, every role and the trust policy of the role, the network paths, the bucket settings, the key-site link, the licence and the boot measurements. hangar verify checks the report offline.

4. The portability report

tools/portability_report.py compares two returns of one job: one release, the same inputs and narrowing, for one tenant, on two pools. The tool verifies each return with hangar verify --json, then lists:

Only a return that hangar verify passes counts. Any other answer, or no hangar on the search path, fails the report. --no-verify compares the fields alone, and the report then never says that the returns are portable.

A package that declares deterministic promises equal output bytes for equal inputs and configuration, so a different digest fails the report. A package that declares envelope promises results within the declared contract, and the report lists any different digest.

tools/portability_report.py hosted.cert harbour.cert \
    --keys-b harbour_keyset.cbor --release-keys release_keyset.cbor

5. Errors

CodeStatusWhen
pool_not_found404No pool of that name or id.
pool_registration_invalid422The registration lacks a name, the provider aws or a 12-digit account id.
pool_runs_locally409A job on a customer-controlled pool starts from the runtime of the pool.
keyset_invalid422The key set is not valid, or is not signed by its own root.
keyset_issuer_refused422The key set does not name your organisation as issuer.
licence_not_found404No licence is issued for the pool.
licence_invalid422The licence is not signed by TALOS Defence's entitlement key, or names another deployment.
return_import_invalid422The return does not verify against the pool's key set and the release key set.
revocation_list_stale409The pool's revocation list is older than its policy allows.
not_in_this_runtime404The call is served by the hosted Hangar service at api.gethangar.au.
runtime_only404The call is served by a customer runtime.
profile_not_supported422Hangar does not install the deployment profile. Use the AWS installation profile.

Read this page as Markdown