How do I test a landing page change without an A/B tool?

What you need

Hoot Lens installed on the page, at least one goal that counts as a conversion, and enough visits on each side of the change. Goals count from the day they are added, so add them before you ship the change. Each side needs at least 30 visits for a verdict, and two weeks is a good aim.

Steps

1. In Settings, Goals, add the goals that count as a conversion on this page: a booking button, a phone link, a thank-you page.
2. Ship one change at a time. Two changes in the same window cannot be told apart.
3. When the change is live, choose Log a change in the workspace. Name what changed, pick the page group, and optionally note the release. Hoot Lens also logs a marker by itself when a page's own content changes, and starts a new page version for it.
4. Wait for visits. Until each side has 30 visits the verdict reads "Too few visits to tell (need N more)".
5. Open "Did it help?" on the change. It compares the days before and after for that page group, with the same device and language filters on both sides, and leaves out the day of the change because daily counts cannot separate its visits.
6. Read the verdict as written. "Likely real" means chance alone would rarely produce the difference. It does not say the change caused it.
7. Look at the heatmap for the new page version next to the old one on the same device class. Heatmaps for different page versions are kept apart, so a redesign never muddies the version before it.

If you really are running two versions at the same time

Label each version so the data stays separate. Push the variant from your experiment code, with a label you allowlist yourself, for example (window.hootlens = window.hootlens || []).push(['variant', 'hero-b']). Labels use letters, digits, _ and -. A heatmap is one page group, release, revision and device class, with the variant as a breakdown inside it, and recordings and funnels can be filtered by variant. Hoot Lens reports what each variant's visitors did. It does not assign visitors to variants and does not calculate a winner, so use your own split and read the results with the same caution as above. To mark a deliberate page change inside one release, push a page version label such as ['pageVersion', 'pricing-v3'], which starts a new revision so before and after stay separate.

Ask your coding agent to do it

An agent that ships the change can log it itself, with the Hoot Lens connector at https://mcp.hootlens.com/mcp and a connection that includes the annotate permission (see how to give your coding agent real user behavior data). These are the exact tool calls.

list_goals { "projectId": "PROJECT_ID" }
log_change { "projectId": "PROJECT_ID", "title": "Shortened the hero headline", "pageGroup": "/", "release": "RELEASE_OR_COMMIT" }
list_changes { "projectId": "PROJECT_ID", "pageGroup": "/" }
change_impact { "projectId": "PROJECT_ID", "changeId": "CHANGE_ID", "days": 14, "deviceClass": "mobile" }

Call log_change right after the change goes live. Call change_impact later, once both sides have visits. change_impact returns a one-sentence summary and the numbers: visits on each side, each goal's conversion rate with a 95% range and its verdict, snags per 100 visits, and the elements whose share of clicks changed most. A report should quote the verdict in its own words with the visit counts, and say it is a before and after comparison.

How much to trust it

The verdict is a pooled two-proportion test on sessions that converted. It cannot see anything outside the page: an email campaign or a holiday in the window shifts conversions without any change on the page. Keep campaigns steady during the window where you can, compare the same device class, and treat a single comparison as evidence to weigh.

Related

• Why aren't users clicking my button?
• Developer FAQ: how do I know if a change helped?