When an AI test agent retries a browser step and still fails, the problem is rarely just the final exception. The useful question is usually, what changed between the first attempt, the retries, and the terminal failure? Without that history, the incident review becomes guesswork: was the selector wrong, did the page re-render, did the prompt steer the agent toward a brittle path, or did the application itself change state unexpectedly?

This checklist focuses on the logs and artifacts that make browser-step failures explainable. It is written for QA engineers, SREs, DevOps teams, and SDETs who need postmortem-quality evidence, not just a pass or fail status. The goal is simple, capture enough context that a person can replay the reasoning, reconstruct the browser state, and decide whether the fix belongs in the test, the agent, or the product.

A retry that fails three times is not a single failure, it is a small sequence of observations. Log it like a sequence, not like one collapsed error.

The minimum useful logging model

A browser agent that retries should emit logs at four levels:

  1. Run level, the test execution as a whole
  2. Step level, the user action or assertion being attempted
  3. Retry level, each attempt and the reason for the retry
  4. Artifact level, screenshots, DOM snapshots, network traces, and other evidence

If you only keep the terminal error, you lose the path. If you only keep screenshots, you lose the reasoning. If you only keep prompt text, you lose the browser state. The point of observability is correlation across all four.

For teams already using traditional test automation, this is the same principle behind good CI/CD troubleshooting and structured application logs, except the “application” here is a combination of the browser, the application under test, and the agent policy that chooses actions. The broader concepts are familiar from software testing, test automation, and continuous integration, but agentic retries add a new layer of ambiguity that deserves explicit logging.

Checklist, what to log before the first retry

1) The step identity

Every browser step needs a stable identifier, even if the agent chose it dynamically. Log:

  • Test name or scenario name
  • Step ID or step index
  • Human-readable step description
  • Timestamp for first attempt
  • Environment name, branch, build number, commit SHA
  • Browser name and version
  • Platform and viewport size

Why this matters: a failure on step 4 of “checkout happy path” is not enough. You need to know whether step 4 is “click Continue,” “wait for address validation,” or “select saved card.” The step identity is the anchor for all downstream artifacts.

2) The prompt version and policy context

If an agent is driven by a prompt, an instruction template, or a policy bundle, log the exact version used for the run.

Capture:

  • Prompt template name and version hash
  • System instructions version
  • Tool policy or action constraints
  • Model name and model version, if applicable
  • Temperature or other sampling settings, if relevant
  • Any route-specific or app-specific overrides

This is one of the most common blind spots in agentic testing. A test can fail because the application changed, but it can also fail because a prompt update changed the agent’s interpretation of ambiguous UI text. Without the prompt version, you cannot separate those possibilities.

3) The starting browser state

Log the state that existed before the step began:

  • Current URL and referrer
  • Open tabs or windows count
  • Local storage and session storage keys, if safe to record
  • Authentication state, for example anonymous, signed in, expired session
  • Feature flag context, if the test environment injects it
  • Cookies metadata, but avoid raw secrets
  • Any prior modal, toast, or overlay visible on screen

A browser agent often fails because it starts from a slightly different state than expected. Maybe a modal was already open, maybe a retry left the page half-navigated, or maybe a cached redirect landed on a different route. Logging the starting state makes those issues visible.

4) The exact action plan the agent intended

Before the agent clicks or types, log the intended action in plain language and, where possible, the structured representation.

Examples:

  • “Click button with accessible name Continue”
  • “Fill email field with user@example.com”
  • “Wait for network idle, then assert order total”
  • “Find element matching role=button, name=Submit”

If the agent has a chain of thought or internal reasoning, do not log sensitive reasoning verbatim. But do log the actionable plan. That is enough for postmortem review.

Checklist, what to log on each retry

5) Retry counter and retry reason

Each retry should have its own record with:

  • Retry number
  • Trigger condition, for example timeout, detached element, stale DOM, assertion mismatch, navigation failure
  • Retry delay or backoff
  • Any automatic fallback, such as alternate selector or refreshed locator strategy

This matters because “retry” can mean several different behaviors. A retry after a timeout is not the same as a retry after an element disappears mid-click. If your system uses selector fallbacks, treat those as explicit retry reasons, not hidden behavior.

6) Selector history

If the agent searched for elements, log the selector path it tried. This is one of the highest-value fields for debugging browser automation.

Capture:

  • Primary locator used
  • Alternative locators attempted
  • Selector strategy order, for example role, text, CSS, XPath
  • Element attributes observed, such as aria-label, text content, data-testid
  • Whether the element was found, visible, enabled, or covered by another element

A selector history can explain why a step failed even when the screenshot looks fine. For example, the page may show a “Save” button, but the agent queried an inaccessible name, or the button was present but obscured by a sticky footer. If you do not log the selector history, you only know the agent was “unable to click,” which is too vague to act on.

7) DOM snapshot or accessibility tree snapshot

A screenshot is useful, but a DOM or accessibility snapshot is often more useful for machine diagnosis.

Capture at least one of:

  • Serialized DOM subtree around the target
  • Accessibility tree fragment
  • Element bounding box and visibility state
  • Computed text for the nearest target node

Use caution with full-page DOM dumps if the page contains secrets or highly sensitive content. Redact or scope the snapshot to the relevant region when possible. The point is to see what the browser actually exposed, not to archive the entire application indiscriminately.

8) Screenshot before and after each retry

For visual failures, keep at least:

  • Screenshot before the failed attempt
  • Screenshot after the failed attempt
  • Screenshot after any recovery action, such as refresh or navigation

Store screenshots with a naming scheme that includes run ID, step ID, and retry number. The “before” image often reveals state leakage, and the “after” image often reveals whether the action partially succeeded.

If you only save the screenshot at terminal failure, you usually save the least informative image in the sequence.

9) Network evidence around the retry window

Browser-step failures often look like UI issues, but they are actually backend timing issues. Log network activity around the retry window:

  • Failed requests and status codes
  • Response times for relevant routes
  • Redirect chains
  • API calls that changed state
  • Asset load failures that may break layout or lazy rendering

You do not need a full HAR file for every step, but you do need enough evidence to see whether the page was waiting on a request, rendering stale data, or being redirected unexpectedly.

10) Console errors and browser warnings

Record browser console output for the duration of the step and retries:

  • JavaScript exceptions
  • CSP violations
  • Resource load errors
  • Degradation warnings from the application
  • Deprecation warnings when they affect behavior

A UI that appears broken often has a console error that explains why the agent could not interact with it. This is especially important when the app uses reactive rendering, where a JavaScript exception can leave the page visually incomplete while still looking “mostly loaded.”

Checklist, what to log when the retry still fails

11) Final failure classification

Do not stop at “failed.” Classify the failure.

Possible categories:

  • Selector not found
  • Element not visible
  • Element covered by overlay
  • Navigation timeout
  • Assertion mismatch
  • Stale element or detached node
  • Authentication expired
  • App error state
  • Ambiguous UI, agent chose wrong target
  • Unknown, but with artifacts attached

A failure taxonomy is not just reporting cosmetics. It determines whether the issue belongs to the test, the agent policy, or the product. This is especially important in agentic QA workflows, where a retry may mask the underlying root cause if all failures collapse into one generic timeout bucket.

12) The full attempt timeline

Log a timeline that includes every attempt and state transition.

Example structure:

  • 10:03:12.104, step started
  • 10:03:12.220, selector search began
  • 10:03:12.611, primary selector failed
  • 10:03:13.002, fallback selector found candidate
  • 10:03:13.144, click attempted
  • 10:03:14.450, navigation timeout
  • 10:03:15.000, retry 2 started

A timeline is often the fastest way to discover whether the agent spent most of its time waiting, searching, or interacting.

13) State deltas between retries

What changed between attempt 1 and attempt 2?

Log differences in:

  • URL
  • DOM content around target
  • Cookies or session state
  • Network responses
  • Visible overlays or modals
  • Focused element
  • Scroll position

The failure mode may be state-dependent. For example, the first click opens a tooltip that blocks the second click, or the first attempt triggers validation that changes the form. If you keep only isolated attempt logs, you miss the transition that caused the issue.

It is not enough to say “screenshot captured.” Store direct links or identifiers for every artifact:

  • Screenshot file path or object key
  • DOM snapshot ID
  • HAR file path
  • Video clip path
  • Console log file
  • Trace bundle path

The person triaging the failure should not need to guess where the files live. If your CI system stores artifacts in object storage, include a stable run manifest that lists all files in one place.

A practical logging schema

A structured JSON log makes this much easier to query than free text. Here is a compact example of a retry record.

{ “run_id”: “r-18422”, “step_id”: “checkout-04”, “step_name”: “Click Continue”, “attempt”: 2, “retry_reason”: “element_covered”, “prompt_version”: “prompt-v7.3.1”, “browser”: “chromium-126”, “viewport”: “1440x900”, “url”: “https://app.example.com/checkout”, “selector_history”: [ { “strategy”: “role”, “value”: “button[name=’Continue’]”, “result”: “covered” }, { “strategy”: “text”, “value”: “Continue”, “result”: “covered” } ], “artifacts”: { “before_screenshot”: “s3://runs/r-18422/checkout-04/a2-before.png”, “after_screenshot”: “s3://runs/r-18422/checkout-04/a2-after.png”, “dom_snapshot”: “s3://runs/r-18422/checkout-04/a2-dom.json”, “console_log”: “s3://runs/r-18422/checkout-04/console.json” }, “outcome”: “failed” }

This shape is not mandatory, but the fields are worth preserving. The key idea is that a triager should be able to answer, in one pass, what the agent tried, what it saw, what it changed, and what it attached.

Practical failure modes this checklist exposes

Ambiguous locators

If the agent keeps selecting the wrong control, selector history will show whether the accessible name was ambiguous, stale, or absent. That points the fix toward test design, not retries.

UI overlays and sticky elements

Screenshots and bounding box data can reveal that the target was physically present but blocked. This is common with cookie banners, chat widgets, and sticky footers.

Race conditions after navigation

Network traces and timeline data show whether the page was still loading when the retry fired. A test that retries too quickly may repeatedly hit a page that is not ready.

Prompt drift

If the same step begins failing after a prompt update, the prompt version and action plan make that visible. Without those fields, the agent appears to have “randomly” regressed.

State leakage between retries

URL, session, and DOM delta logging can show that the first attempt changed the app state in a way the second attempt did not expect. This is a common issue in long-running flows like checkout, onboarding, or account recovery.

How much logging is enough

The right answer is usually, enough to reproduce the failure path without making the logs so heavy that nobody uses them.

A good rule is:

  • Always log structured step and retry metadata
  • Always capture at least one screenshot per attempt
  • Capture DOM, console, and network evidence for failures, or for steps that commonly flake
  • Scope deep traces to the retry window, not the whole run
  • Redact secrets and sensitive payloads before storing artifacts

The tradeoff is storage and noise versus diagnosis speed. Full tracing on every step can overwhelm artifact stores and make reviews tedious. Sparse logging saves space but creates mystery failures. Most teams end up with tiered observability, lightweight logs by default, richer capture on retries or on designated critical flows.

A lightweight CI pattern that helps

If your tests run in CI, make the artifact bundle part of the job output and link it in the build summary.

name: ui-tests
on: [push, pull_request]
jobs:
  browser-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: browser-trace
          path: artifacts/

This does not solve observability by itself, but it ensures the data survives the run and is available to whoever is debugging the failure.

Where agentic platforms can help triage

A practical agentic QA platform should make these artifacts easy to inspect rather than hiding them inside opaque retries. For example, Endtest, an agentic AI test automation platform,’s AI Test Creation Agent generates editable, platform-native test steps from natural language, and that kind of human-readable step structure is easier to review alongside run artifacts than a large amount of generated framework code. In triage, the useful question is not whether the agent was clever, it is whether the run left enough evidence to explain the failure.

If your team uses a platform like Endtest, look for the artifacts that a run can capture for review, such as step-by-step execution history, screenshots, logs, and other evidence attached to the run. Those artifacts are most valuable when they are associated with the exact step and retry attempt that failed, not just the test as a whole.

Final checklist for postmortems

Before closing a failed agent retry incident, verify that you have these items:

  • Run ID, step ID, environment, browser, and viewport
  • Prompt version and policy context
  • Starting browser state
  • Intended action plan
  • Retry number and retry reason
  • Selector history with alternative locators
  • DOM or accessibility snapshot near the target
  • Screenshot before and after each retry
  • Network evidence around the retry window
  • Console errors and warnings
  • Final failure classification
  • Full attempt timeline
  • State deltas between retries
  • Artifact links that are stable and easy to open

If one of those is missing, ask whether the next incident will be explainable without it. That is usually the best test.

Bottom line

The most useful answer to what to log when an AI test agent retries a browser step is not “everything.” It is the smallest set of structured evidence that makes the run understandable after the fact. For retry traces, that means step identity, prompt version, selector history, artifacts, and a timeline that shows how the browser state changed between attempts.

When retries still fail, the logs should let a person answer three questions quickly: what did the agent try, what did the browser show, and what changed between attempts? If the answer is not recoverable from the artifacts, the observability layer is still too thin.