First task

Work starts in your repository through your coding harness. Track the job under Outcomes and its settled receipt under Usage.

Deliveries are checked against the client’s agreed tests. Verification is currently operated by Ormas.

1 — Create an account#

Go to ormas.ai/signup and sign in with Google, GitHub, or an email magic link. US-only. No card is required to look.

A new account starts on Start here. Add credit in Billing to open Outcomes; you do not need an operator invitation. Any signed-in account with credit and an Ormas key can submit to a public repository. There is no selected-pilot allowlist for public work. See Billing for the current minimum credit purchase and service fee at checkout. Admission stays open after your first credit purchase, but running work still needs sufficient credit.

2 — Mint an Ormas key#

Open /app/keys while signed in. This direct link works before adding credit. Create an Ormas key (tb_live_…) and copy it once. It is not a model-provider key.

3 — Connect your client#

Choose your agent below. Ask that agent to run the command on your computer, or paste it into Terminal. Setup installs Ormas, downloads a private Python runtime when needed, checks the agent server, and configures the chosen agent. You do not need to install Python, activate an environment, set environment variables or edit configuration files yourself.

For Codex (desktop app, CLI or IDE):

bash -c '(
  installer="$(mktemp)" || exit 1
  trap '\''rm -f "$installer"'\'' EXIT
  curl --fail --silent --show-error --location --proto '\''=https'\'' --proto-redir '\''=https'\'' \
    --connect-timeout 15 --max-time 180 \
    https://ormas.ai/install-client.sh -o "$installer" || {
      echo "Ormas installer download failed. Check your connection to https://ormas.ai and retry." >&2
      exit 1
    }
  bash "$installer" --agent codex
)'

For Claude Code, installed on this computer:

bash -c '(
  installer="$(mktemp)" || exit 1
  trap '\''rm -f "$installer"'\'' EXIT
  curl --fail --silent --show-error --location --proto '\''=https'\'' --proto-redir '\''=https'\'' \
    --connect-timeout 15 --max-time 180 \
    https://ormas.ai/install-client.sh -o "$installer" || {
      echo "Ormas installer download failed. Check your connection to https://ormas.ai and retry." >&2
      exit 1
    }
  bash "$installer" --agent claude
)'

The connection uses a fixed executable path. It works even if your agent does not inherit your terminal's PATH. Codex uses its shared user settings; Claude Code uses its private user-scoped MCP connection across projects. After setup and login, restart your agent to load Ormas. Setup checks that the local agent server initializes and exposes its tools; it does not claim that your already-running agent has loaded the new connection.

On Windows, run setup and your coding agent inside WSL2 Ubuntu. A native Windows desktop agent connection is not supported by this installer. For a new WSL installation use wsl --install -d Ubuntu-24.04, then open its Ubuntu terminal. Supported: macOS 14 or later (Apple Silicon or Intel), and Linux x86_64 with glibc 2.28 or later, including Ubuntu 22.04/24.04. Alpine/musl and native Windows are not supported.

Make ormas available on PATH#

Setup handles this automatically for Bash, zsh and fish. It adds the short ormas command to supported shell startup files, preserves their existing contents, backs up modified files privately, and avoids duplicate entries on repeat installs. Open a new terminal to use that command. Your agent connection does not depend on this step.

The full-path login and doctor commands below work immediately in the current terminal, including when shell setup could not finish. If an existing launcher is reported, it is preserved; use those full-path commands until you resolve the conflict.

Installation troubleshooting#

Setup failures name the step and show a specific next action. Private logs are under ~/.local/state/ormas-client/install-0.0.4-r2.*/install.log. You can share the release, operating system, failed step and relevant error lines when asking for help; review logs first. No key is needed for installation.

Advanced and manual setup#

The installer source, versioned installer, dependency lock, runtime manifest and wheel manifest identify the release. The script and every dependency are checked against pinned hashes; source builds are disabled. A Python environment isolates packages, not operating-system permissions.

To install without configuring an agent:

bash -c '(
  installer="$(mktemp)" || exit 1
  trap '\''rm -f "$installer"'\'' EXIT
  curl --fail --silent --show-error --location --proto '\''=https'\'' --proto-redir '\''=https'\'' \
    --connect-timeout 15 --max-time 180 \
    https://ormas.ai/install-client.sh -o "$installer" || {
      echo "Ormas installer download failed. Check your connection to https://ormas.ai and retry." >&2
      exit 1
    }
  bash "$installer"
)'

The script also accepts --no-modify-path to leave shell files untouched, --managed-python to always use the private runtime, and --python /absolute/path/to/python3.12 to select an existing runtime. An explicit unusable interpreter produces an error rather than being silently replaced. Backups of modified shell/agent settings live in that run's private log directory; unrelated agent settings and existing client environments are preserved.

The account login step remains explicit: setup does not copy keys into agent configuration or start paid work. Future remote sign-in would need a separate authenticated service design; the local connection supports preparation against files on your computer today.

Log in and check the connection#

"$HOME/.ormas-client/bin/ormas" login
"$HOME/.ormas-client/bin/ormas" doctor

Paste your Ormas key when login prompts for it, never into a command argument or chat. Login stores the key locally; doctor performs a non-billable authentication check against https://api.ormas.ai. Local storage success alone does not prove authentication. You do not need to supply model-provider keys.

The default preparation mode is local. It runs the base check in the declared container environment, using a non-root Linux user and a native Linux/amd64 Docker engine, with access to the public dependency registries. On macOS (Apple Silicon or Intel), or any machine without that engine, ask your agent to prepare with preflight_mode: "deferred". No Docker is needed. The miner checks the unfixed base before any paid work. If the check already passes on the base, the job ends as no-delivery at $0, so commit a failing acceptance check first. The acceptance check itself must fail at a plain assert; that rule is in the prepare step below. Linux/amd64 with Docker may keep the default local preflight for immediate feedback. On Windows, use WSL2 with Ubuntu and follow these steps in the Ubuntu terminal. Native Windows is not supported.

For Codex, run in your shell:

codex mcp add ormas -- ~/.ormas-client/bin/python -I -m tensorbox_spec.mcp_server.client_server
codex mcp get ormas

For Claude Code, use claude mcp add --scope user --transport stdio ormas -- ~/.ormas-client/bin/python -I -m tensorbox_spec.mcp_server.client_server, then claude mcp list. For GrokBuild, run:

grok mcp add ormas -- ~/.ormas-client/bin/python -I -m tensorbox_spec.mcp_server.client_server
grok mcp list

Restart your coding agent and confirm that it exposes ormas_prepare_task, the project tools, and ormas_outcomes_job. If editing Codex configuration by hand, use an absolute executable path and put command and args on separate TOML lines. See the Codex MCP documentation.

4 — Create a project and send one task#

Check Billing for existing credit before buying more. In your coding agent, describe the change, the behavior that proves it, and the maximum price you authorize. The agent handles the calls below; you do not write an isolation wrapper or paste a work packet.

1. Reuse or create a public project. Ask the agent to list existing projects for this repository. If none exists, call ormas_create_outcomes_project(name="my-repo", repo_url="https://github.com/OWNER/REPO.git", third_party_miners=True) and retain the project_id. This explicitly opts in to third-party miners and qualified checkers. The Standard service level makes no zero-data-retention promise.

   Private repositories. Private repositories at the Standard service level connect through Connect repository in Setup using the Ormas SN76 GitHub App. Create the project with third-party miners turned on (third_party_miners: true), then follow the App connection steps below. Prepare each task with execution_environment.repository_access set to {"visibility": "private", "sharing": "shared-code-v1"}. The assigned miner, the checker and their AI providers can read the code; see Where your code goes. On macOS, or without a native Linux/amd64 Docker engine, use preflight_mode: "deferred" (see above).

2. Connect the repository once. In Setup, select the project and press Connect repository. GitHub asks you to install the Ormas SN76 app; choose Only select repositories and pick this repository. If an organization owner must approve the install, press Connect repository again after approval. GitHub then asks you to authorize Ormas SN76, and Setup shows Repository connected. Ormas uses a short-lived token for that one repository only when it publishes the result branch, and checks at each submission that the person who connected it can still write to it. Disconnect repository in Setup removes access.

   Alternative for public repositories only: a saved token. Run ormas project github-token PROJECT_ID locally, replacing PROJECT_ID with the returned ID. Enter a fine-grained GitHub token at the hidden prompt. Restrict it to the chosen repository with Contents: read and write. GitHub grants that permission at repository scope; Ormas publishes only the agreed result branch. Workflow-file changes can require additional GitHub permission and must satisfy the supported task policy. See GitHub token setup and reference creation permissions. Never pass the token as an argument or paste it into chat. Ormas stores it encrypted; public miners and checkers clone anonymously and receive no write credential. Setup shows whether access is saved; submission checks current access. Reuse it rather than creating new keys for each task.

3. Prepare the agreed check. Offered task sizes are small, medium and large; large runs as a single attempt against the attempt authorization you set (attempt_authorization_usd). Your agent authors or reuses a frozen acceptance check in a supported environment, commits its dependency locks, and pushes the base. The check observes the requested behavior independently of candidate code. Every failing acceptance case must fail at a plain assert (an AssertionError). An exception (for example KeyError), an import error, or extra output mixed into the test result (a library warning printed into the result) is refused before any paid work, with $0 charged, as your task fault: it did not fail at an assertion. Keep warnings and prints out of the result. The check must also fail on the unfixed base. If it already passes, the job ends as no-delivery at $0. Missing tools and collection errors are not a failing assertion. Ordinary unit tests can also be regressions, but candidate-controlled test reports are not the independent acceptance boundary. For Python, follow the acceptance check and lock recipe below. The agent passes the original test command and explicit allowed and immutable paths to ormas_prepare_task; Ormas freezes the environment and isolates execution. You do not choose a model; the miner decides how to do the work and receipts show the outcome, price and verification, not a model. A preparation-only rehearsal stops here: no job or provider call has been submitted.

4. Authorize the maximum price. Your agent calls:

   textCopy code

   ormas_outcomes_job(action="enqueue", payload={
       "project_id": "<returned project_id>",
       "reserve_usd": <your chosen maximum price in USD>,
       # Optional: public repositories only; omit for automatic selection.
       "award_policy": {"policy": "manual", "ack_by_s": 60, "window_s": 300, "fallback": "hold"},
   }, work_packet=<the inner packet from successful preparation>)
   

   The exact inner packet is the top-level work_packet argument. Qualified miners offer a firm price, or an estimate with a hard limit. Your ceiling at submission is a maximum, never the price. An offer within your ceiling is accepted automatically by default; for public repositories, set award_policy to manual at enqueue to choose the offer yourself. A ceiling does not guarantee capacity. Admission requires a qualified miner and enough qualified checkers for the agreed policy before work starts. If every miner slot is busy, the job waits as queued instead of being refused. Each client can have up to 5 jobs in flight (queued and running). An unclaimed job ends after 60 minutes as no-delivery with $0 charged. You may set queue_wait_s (1–3600 seconds) in the enqueue payload for a shorter wait, never longer. Status shows queued and queue_expires_at while waiting. Funds for queued jobs are held against your balance and any daily cap at submission.

5. Follow the same job to settlement. Save the returned job_id and call ormas_outcomes_job(action="status", job_id="<returned job_id>"). A job whose miner never delivers settles as no-delivery at $0 on that ordinary status read. You do not need to call reconcile after a restart. Reconcile still exists and is safe. Resubmitting after a no-delivery receipt is safe. After a restart or an uncertain network response, read status on this job before you treat it as lost. Worker completion can still be Awaiting validator checks. Follow acceptance and the receipt; a finished worker or queued job is not accepted delivery.

Choose your offer#

Offer selection is available for bounded tasks in public repositories. Private repositories have no offer window today; omit award_policy or enqueue returns award_policy_requires_public_bounded_packet.

Set payload.award_policy.policy to:

• lowest_expected_charge: automatically choose the lowest predicted final charge.
• lowest_expected_total_cost (default): automatically choose the lowest expected cost of getting an accepted result, also weighing the chance of failed delivery and price exposure up to the limit.
• manual: keep the offers open so you can choose one yourself.

Open the job under Outcomes to compare each offer's firm Price, or Estimate and Limit, alongside its Expected charge and the gateway's recommendation. Predictions are not bills. In manual mode, press Choose, review the terms, then Confirm acceptance. If you see This offer changed, cancel and review the refreshed offers before choosing again. An accepted award cannot be changed.

Automatic selection uses a 60-second window by default and can decide earlier when all eligible miners have answered. Set ack_by_s to an integer from 15 to 60 seconds. Manual selection uses window_s: 600 seconds by default, at least ack_by_s and at most 600. The enqueue example gives you five minutes. When a manual window closes, fallback applies: lowest_expected_charge, lowest_expected_total_cost (default), or hold, which requeues without choosing an offer.

Your agent can use these MCP calls while the job is offering, its window is open and no offer has been awarded:

ormas_outcomes_job(action="accept_offer", job_id="<job_id>", payload={"bid_id": "<bid_id>"})
ormas_outcomes_job(action="decide_now", job_id="<job_id>")
ormas_outcomes_job(action="update_award_policy", job_id="<job_id>", payload={"award_policy": {"policy": "manual", "window_s": 300, "fallback": "hold"}})

accept_offer commits your manual choice. decide_now ends the window early using the current automatic policy or manual fallback. update_award_policy changes only the supplied policy fields; switch to manual before the automatic award if you want to choose. Changing policy does not restart the window's clock.

On verified delivery, you are charged the firm price or the miner's settled price, never above its limit. An estimate or expected charge is not the bill. There is no per-task markup; read the settled receipt for the actual charge.

Preparation fields for client authors#

The public preparer takes verify_cwd pointing to the clean local checkout at the full pushed repo_base_sha, the ordinary verify_command, allowed_paths, and immutable_paths. Supply execution_environment with profile_id, languages, and lock_paths from the catalog. For Python pytest, use {"profile_id":"linux-python-pytest-v1","languages":["python"],"lock_paths":["requirements.lock"]}. The named test files are frozen automatically; acceptance_paths can include their support files. Locks and acceptance files must be immutable. Use one supported executable and arguments, without shell chaining or redirects. A relative interpreter still needs its declared toolchain. Inspect ok, packet, and packet_sha256; pass the successful packet unchanged. Unknown requirements are refused rather than silently substituted.

Python acceptance check and lock#

The acceptance check runs in its own container. That container holds only your frozen acceptance files and requirements.lock, so the test cannot import your project code, and pytest plugins do not load automatically. Instead, the test runs a command against the candidate checkout through ormas_acceptance.run and asserts on what comes back. Ormas supplies that module when the check runs. The command starts at the repository root, and its stdout and stderr come back together as output, so print only the value you assert on.

tests/ormas_acceptance/test_total_with_tax.py:

import pytest
from ormas_acceptance import run  # supplied by Ormas when the check runs


@pytest.mark.parametrize("subtotal, rate, expected", [
    ("10.00", "0.0825", "10.83"),
    ("19.99", "0.0825", "21.64"),
    ("0", "0.0825", "0.00"),
])
def test_total_is_rounded_to_cents(subtotal, rate, expected):
    code = ("import sys; from shop.pricing import total_with_tax; "
            "print(total_with_tax(sys.argv[1], sys.argv[2]))")
    observed = run(["python3", "-c", code, subtotal, rate])
    assert observed["returncode"] == 0, observed["output"]
    assert observed["output"].strip() == expected

If the candidate raises an exception, the first assert fails on its return code. Every failing case therefore fails at a plain assert. Prepare it with verify_command="python3 -m pytest tests/ormas_acceptance/test_total_with_tax.py -q -p no:cacheprovider --tb=short", allowed_paths=["shop/pricing.py"], and immutable_paths=["tests/ormas_acceptance", "requirements.lock"].

The lock lists every package the check needs, including pytest and all of its dependencies. Each line is name==version --hash=sha256:<hash> for a wheel that installs on Linux x86_64 with CPython 3.12. Installation is offline from those wheels only. Hashed locks from pip-compile or uv pip compile split each package's hashes across several lines and are refused. From the repository root, list your direct test dependencies in requirements.txt (for example pytest, plus any packages your code imports), then run this in bash or zsh on macOS or Linux. The platform list covers every Linux wheel the environment accepts:

WHEELS=$(mktemp -d)
PLATFORMS=()
for v in {36..5}; do PLATFORMS+=(--platform "manylinux_2_${v}_x86_64"); done
PLATFORMS+=(--platform manylinux2014_x86_64 --platform manylinux2010_x86_64 --platform manylinux1_x86_64)
~/.ormas-client/bin/python -m pip download --only-binary=:all: "${PLATFORMS[@]}" \
  --python-version 3.12 --implementation cp --abi cp312 -d "$WHEELS" -r requirements.txt
~/.ormas-client/bin/python - "$WHEELS" > requirements.lock <<'EOF'
import hashlib, pathlib, sys
for wheel in sorted(pathlib.Path(sys.argv[1]).glob("*.whl"), key=lambda p: p.name.lower()):
    name, version = wheel.name.split("-")[:2]
    print(f"{name}=={version} --hash=sha256:{hashlib.sha256(wheel.read_bytes()).hexdigest()}")
EOF

Commit requirements.txt, requirements.lock and the acceptance test before you prepare; preparation refuses a checkout with uncommitted or untracked files.

Pip applies each package's operating-system conditions for the machine it runs on. On macOS, a dependency that differs by operating system (for example keyring) can be missing from the lock, or pip download can stop on a macOS-only package. In that case, run the same commands on Linux x86_64, such as WSL2 with Ubuntu. If a package has no Linux x86_64 wheel for CPython 3.12, this environment cannot install it.

To confirm the base fails at an assertion before you submit, run the check locally against a stand-in for ormas_acceptance. Keep the stand-in outside the repository:

mkdir -p ~/.ormas-acceptance-local
cat > ~/.ormas-acceptance-local/ormas_acceptance.py <<'EOF'
import subprocess


def run(argv, timeout_s=30):
    done = subprocess.run(argv, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, timeout=timeout_s)
    return {"returncode": done.returncode, "output": done.stdout}
EOF
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=~/.ormas-acceptance-local python3 -m pytest tests/ormas_acceptance -q -p no:cacheprovider

The stand-in runs the command on your machine and needs pytest installed locally. PYTHONDONTWRITEBYTECODE=1 keeps the rehearsal from leaving untracked cache files that would block preparation. Every case should fail with AssertionError on the base. Ormas runs the real check in isolation.

5 — Read the receipt#

Open the project under Outcomes for the result and validator-check count. Accepted delivery is charged at the firm price or the miner’s settled price, never above its limit. A failed delivery costs nothing. Limit offers show Estimate, Limit and Charged separately; an estimate is not the charge. Rejected or expired acceptance is no-delivery; a checker setup error does not prove the miner's patch is wrong. Pending acceptance is not a settled charge. Usage lists settled receipts and 24h / 7d / 30d totals. Review the branch and verification evidence before merging it into your repository.

6 — Send the next change#

Reuse the project, Ormas login and saved GitHub access. Describe the next change and confirm its success check and budget. Your agent prepares the new base and task, then follows the same job and receipt flow. Expired or revoked GitHub access needs reconnecting once through the hidden prompt.