CLI

The CLI needs Node.js 20 or later.

npm install --save-dev @timetotest/cli@beta
npx ttt init

ttt init creates .ttt/config.json and an example test, tests/example.e2e.ts. It does not overwrite existing files. ttt collect then lists the cases and steps that would run, without calling our servers.

In the project's Settings, click Create key under CLI keys. Name it after where it is used and choose when it expires: 30 days, 90 days or 1 year. Copy the key from Key created. It is shown once. Only project admins can create keys.

The key identifies the project, so you do not pass a project ID.

export TTT_API_KEY="ttt_pk_..."

Keep the key in your environment or CI secrets. The CLI never prints it or writes it to disk.

Save test files as tests/*.e2e.ts. By default the CLI collects every *.e2e.ts, *.e2e.js, *.e2e.mts and *.e2e.mjs file in the project.

import { test, expect } from 'e2e';

test.describe('checkout', () => {
  test('a member pays with a saved card', async ({ app, agent, screen }) => {
    await app.open('/cart');
    await agent.act('Sign in with the staging member account');
    await agent.act('Pay with the saved Visa card');
    await expect(screen.getByText('Order confirmed')).toBeVisible();
    await agent.assert('The order number is shown');
  });
});

The CLI reads test files without running them. agent.act becomes a step and agent.assert or expect becomes a result the agent checks. Loops, helper calls and interpolated strings are rejected with the file and line.

Supported: test, test.describe, test.skip, beforeEach, afterEach, app.open, agent.act, agent.assert, agent.waitFor, and screen.getByRole, getByText or getByTestId with toBeVisible, toBeHidden, toHaveText, toContainText or toHaveValue. The agent judges these assertions; they are not run through a locator engine.

The format follows tester.army test files, which has more examples.

npx ttt collect
npx ttt run --url https://staging.example.com
npx ttt run tests/checkout.e2e.ts --url https://staging.example.com
npx ttt run --resume

collect lists the cases that would run, without calling our servers. run submits them and waits for the result. --resume reconnects to the last batch.

Add a step after the deploy job. This example assumes the deploy job outputs the staging URL, commit SHA and pull request number.

- name: Test staging
  run: npx ttt run --url "$STAGING_URL" --sha "$DEPLOYED_SHA" --pr "$PR_NUMBER"
  env:
    TTT_API_KEY: ${{ secrets.TTT_API_KEY }}
    STAGING_URL: ${{ needs.deploy.outputs.staging_url }}
    DEPLOYED_SHA: ${{ needs.deploy.outputs.commit_sha }}
    PR_NUMBER: ${{ needs.deploy.outputs.pr_number }}

Omit --pr when there is no pull request. Upload .ttt/ as an artifact even when the job fails. With the project's GitHub App connected, results also appear on the pull request.

Failed cases print what was expected, what happened, steps to reproduce and evidence. Reports are written to .ttt/report.json, .ttt/junit.xml and .ttt/summary.md. Commit .ttt/config.json and nothing else in .ttt/.