# Testing with Hoot Lens

`@hootlens/playwright` lets a coding agent (or any test) check its own change with Hoot Lens. A Playwright test runs the page, Hoot Lens records the run as a controlled visit, and the test reads back the same narrative and snag counts a person sees in the dashboard. It is a testing tool as well as an analytics tool.

The package lives in `packages/playwright`. Until it is on npm, use it from this repository.

## When to use it

- After a UI fix, to confirm the change produced no pile-ups, no-shows or misfires on the flow it touched.
- To watch what a test actually did: the replay link shows every click, scroll and page change of the run.
- To give an agent evidence to read. The narrative is plain text ("Landed on /checkout", "Pile-up on 'Pay' (3 taps in 0.6 s)") and each step names the recording and the moment to seek to.

It is not a replacement for assertions. Keep ordinary Playwright `expect` checks for behavior, and add `expectNoSnags()` for how the interface felt to use.

## How a test visit is marked

Hoot Lens already has controlled validation visits. A visit is controlled when its `variant` label is `controlled-validation` or starts with `controlled-validation-`. The tracker takes the label from `['variant', ...]` in the `window.hootlens` command queue (or the `variant` option), and the collector reads it from each batch's segment. The fixture sets it for you, as `controlled-validation-<test title>-<six hex characters>`, so each run is unique and can be found again.

Controlled visits are kept, but never counted. They are left out of traffic, findings, heatmaps, goals and plan usage, and recording lists hide them unless asked (`controlled=only` or `include`). Test runs cannot change a customer's numbers. See [Recordings](API.md) for the exact rules. The `controlled` option only accepts `true`: a test run cannot be made to count.

## Setup

1. Install it next to Playwright: `npm install --save-dev @hootlens/playwright @playwright/test`.
2. Pick a project. A dedicated staging project keeps test recordings away from the production list, but any project works because the visits are controlled.
3. Allow the address the test serves from. The collector refuses origins the project does not list. For a local dev server, the project owner opens **Installation** for the project and chooses **Allow localhost for testing**. It adds one exact `http://localhost:PORT` or `http://127.0.0.1:PORT` origin and nothing else, so start the server on that port every time. A deployed preview needs its exact `https://` origin added to the project's allowed sites. Without this step the fixture throws a `HootlensError` with code `origin_not_registered` that says the same thing and names the origin it was refused for.
4. To read narratives, create a personal access token with the **read** scope in the dashboard and expose it as `HOOTLENS_PAT`. Without a token the test still records and gets the replay links, but cannot check snags.
5. Add the options to `playwright.config.ts`.

```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
import type { HootlensTestOptions } from '@hootlens/playwright';

export default defineConfig<HootlensTestOptions>({
  reporter: [['list'], ['html'], ['@hootlens/playwright/reporter']],
  use: {
    baseURL: 'http://localhost:5173',
    hootlensOptions: {
      projectId: process.env.HOOTLENS_PROJECT_ID,
      apiUrl: process.env.HOOTLENS_API_URL,        // optional; the hosted API by default
      token: process.env.HOOTLENS_PAT,             // read scope
      controlled: true,
    },
  },
});
```

The option is named `hootlensOptions` because the `hootlens` name belongs to the fixture. Each option also reads its environment variable when unset: `HOOTLENS_PROJECT_ID`, `HOOTLENS_API_URL`, `HOOTLENS_PAT`, `HOOTLENS_DASHBOARD_URL` and `HOOTLENS_TRACKER_SOURCE`.

## A complete example

```ts
// tests/checkout.spec.ts
import { test, expect } from '@hootlens/playwright';

test('paying with a card', async ({ page, hootlens }) => {
  await hootlens.start();                 // before the first page.goto(); labels the visit with the test title
  await page.goto('/checkout');

  await page.getByLabel('Card number').fill('4242 4242 4242 4242');
  await page.getByRole('button', { name: 'Pay' }).click();
  await expect(page.getByText('Thank you')).toBeVisible();

  const narrative = await hootlens.narrative();
  if (narrative.available) {
    console.log(narrative.text);          // the story of the run, one line per step
    expect(narrative.snags.pileUps).toBe(0);
  }
  await hootlens.expectNoSnags();         // fails with the narrative and the replay link when there are snags
});
```

The fixture also runs after every test that called `start()`. It flushes the tracker, adds the replay address to the test's annotations (the HTML report shows it as a link), attaches `hootlens-narrative.txt` and `hootlens-recording.json`, and the reporter prints the links at the end of the run. A problem at that point is recorded as a `hootlens-error` annotation and never fails an otherwise passing test.

## The fixture

| Call | What it does |
| --- | --- |
| `hootlens.start({ projectId?, label? })` | Marks every page the test loads as a controlled visit and returns the handle. A page without the Hoot Lens tag gets the tracker (the bundle runs inside a Playwright init script, so a page's script rules cannot block it, though its uploads still need the page's `connect-src` to allow the API). A page that already has the tag is only marked controlled. It also works on a page that is already loaded, and then confirms delivery. `label` defaults to the test title. |
| `hootlens.flush()` | Calls `window.Hootlens.flush()` and waits until the collector has answered, using `diagnostics().lastUpload`. Throws a `HootlensError` that names the fix when the collector refuses the page. |
| `hootlens.recording()` | Returns `{ projectId, sessionId, recordingId?, recordingIds, variant, dashboardUrl }`. The session ID is read from the page's `hootlens:session:<projectId>` storage entry. The recording IDs need a token (the page views are listed by the visit's variant). `shareUrl` is never set: share links are made by signed-in people in the dashboard, not by tokens. |
| `hootlens.narrative({ token? })` | Flushes, then reads `GET /v1/projects/:projectId/sessions/:sessionId/narrative`. Returns `{ available: true, summary, text, steps, facts, snags: { pileUps, noShows, misfires, total, steps }, recording }`. Without a token it returns `{ available: false, note, recording }` and the note says why. |
| `hootlens.expectNoSnags({ allow? })` | Throws when the visit had more pile-ups, no-shows or misfires than `allow` (default none). The message holds the narrative and the replay link. Without a token it throws too, because nothing was checked. |

Other options: `inject` (`auto`, `always` or `never`), `consent` (default `true`, so a project that requires consent still records the test visit), `trackerSource` (an https URL or a file path for the tracker bundle, `https://hootlens.com/t.js` by default) and `timeoutMs`.

Pages that load the tracker through the npm package instead of the tag or script cannot be marked from outside. Use `inject: 'never'` and set the variant in the app's own test build (`variant: 'controlled-validation-e2e'`).

## Reading a failure

Every error is a `HootlensError` with a `code`. The ones an agent meets most often:

- `origin_not_registered`: the project does not allow this page's origin. Use **Allow localhost for testing** for a local server, or add the origin.
- `project_not_found`: the project ID or `apiUrl` is wrong, or the API is not running.
- `path_not_targeted`: the project's page rules exclude the page. Test a page the project records.
- `api_unauthorized` and `api_forbidden`: the token is missing the **read** scope, expired, or belongs to another project.
- `snags_found`: `expectNoSnags()` found snags. The message is the narrative, with the SNAG steps marked.
- `not_recording`: the browser sends Global Privacy Control or Do Not Track, the project samples fewer than 100% of visits, or the tracker is waiting for consent. Hoot Lens never records those visits.

## CI

Store a read-scoped key as a secret. A read key cannot read replay events or change anything.

```yaml
- run: npx playwright test
  env:
    HOOTLENS_PROJECT_ID: ${{ vars.HOOTLENS_PROJECT_ID }}
    HOOTLENS_PAT: ${{ secrets.HOOTLENS_PAT }}
```

In CI the dev server runs on a known port, so allow that exact address once (the localhost step above, or the preview origin). The run's replay links appear in the HTML report and in the log through the reporter. Without `HOOTLENS_PAT` the run still records and links, and only `narrative()` and `expectNoSnags()` stop being able to check.

## Privacy

- Test visits are controlled visits. They are never counted in traffic, findings, heatmaps or plan usage, and the dashboard shows them only on request.
- Recording still follows the project's privacy policy (full, balanced or strict, plus the project's selectors). Password, payment and one-time-code values, `[data-hoot-private]` regions, URL fragments and URLs with credentials are never captured in any mode. Use synthetic data in tests all the same: the card number above is a test number.
- The fixture grants consent for the test browser by default because the person running a test is the visitor. Global Privacy Control and Do Not Track still stop capture.
- The test title becomes part of the visit's variant label, so it is stored with the recording. Do not put secrets in test titles.
- The token is a server-side secret. Keep `HOOTLENS_PAT` in CI secrets or a local environment file, never in a page or a visitor-facing script.
- The narrative holds no page text beyond click labels that already passed the project's privacy policy, and none at all under the strict preset.

## What this does not do

- It does not validate any other framework or runner. It is a Playwright Test fixture.
- It does not publish or share replays: share links need a signed-in person.
- Narratives and snags describe observed behavior with evidence. A clean narrative means no pile-ups, no-shows or misfires were observed in that run, not that the page is good.