Why a Browser Agent Cannot See Elements in the Accessibility Tree
A browser agent can miss an element because the accessibility tree is not a complete description of the rendered page. It is derived from HTML semantics, ARIA and browser rules. Custom controls, role-less elements, canvas content, layered interfaces and incorrectly implemented accessibility markup can leave the agent with an incomplete or misleading view.
For the category overview, see the category's perception layers.
What the accessibility tree represents
The accessibility tree is a browser-generated representation designed for assistive technology. It can provide roles, names, values, states and relationships for elements that expose usable semantics.
For browser automation, that is valuable. A button represented as a button with a clear accessible name is easier for a model to understand than an anonymous DOM node.
The problem begins when an agent treats the accessibility tree as the page itself.
Why elements go missing
The control has no useful semantics
A clickable div may look and behave like a button while exposing no button role or accessible name. A human sees a control. The accessibility tree may expose an unnamed generic element or omit the useful interaction entirely.
ARIA is incomplete or incorrect
ARIA can improve a control's semantics, but incorrect roles and labels can also misdescribe it. The agent may see the wrong name, the wrong state or a relationship that does not match what is rendered.
The page uses canvas or other visual rendering
Canvas content does not automatically become a set of semantic controls. The page may contain a rich visual interface while the accessibility tree exposes little that an agent can use.
The element is inside an iframe or shadow root
Modern applications divide interfaces across document boundaries and component trees. A browser tool that inspects only one surface may not reach the control the user sees.
Visibility and accessibility are different questions
An element can exist in the accessibility tree while being covered by a modal, outside the viewport or visually unavailable. Another element can be visible and meaningful to the user while exposing weak accessibility semantics.
An agent needs both meaning and rendered state.
How to diagnose the problem
When an agent says an element does not exist, check four layers:
- Rendered page: Is the element visibly present now?
- DOM: Does an element exist, and does it have text or interaction handlers?
- Accessibility tree: Does it expose a useful role, name and state?
- Layout and occlusion: Is it actually reachable, or is another layer covering it?
This separates a perception problem from a timing or action problem.
Do not solve every miss with a selector
Adding a hard-coded CSS selector may repair one version of one page. It does not solve the underlying problem for an agent that must work on unknown pages, dynamic layouts or interfaces that change between sessions.
A robust browser representation should help the model find an element by meaning and current rendered state, then provide a stable actionable reference for that moment.
What structured browser perception adds
E2LLM's SiFR representation combines page structure with rendered information such as text, position, salience, relationships and occlusion. The model can reason about what is present on screen without depending exclusively on the accessibility tree or a screenshot.
That helps with cases such as:
- a visible control built from generic elements;
- a modal covering an otherwise valid target;
- multiple controls with similar labels in different regions;
- a page that re-renders and replaces old references;
- an interface where layout carries meaning that roles do not.
It does not make every page automatically actionable. Browser-internal pages, closed tabs, expired sessions and unsupported rendering surfaces still create real boundaries.
A recovery pattern for agents
When the expected element is missing:
- Recapture the current rendered page.
- Search by visible text and meaning, not only by role.
- Inspect the relevant region and its relationships.
- Check for a modal, overlay, iframe or state change.
- Act only after the current target is confirmed.
- Read the resulting change before continuing.
The important step is re-observation. Repeating the same action against an old representation only repeats the uncertainty.
Accessibility still matters
This is not an argument against accessible HTML. Correct semantics help people, assistive technology, testing tools and agents. The accessibility tree should be one strong source of meaning.
It should not be the only source an agent can use to understand a rendered interface.