UiPath Documentation
activities
latest
false
UI Automation activities

v26.10

v26.10 release notes for UI Automation activities, versions 26.10.x.

v26.10.2

Release date: August 10, 2026

UI Automation support for coding agents

Improvements for coding agents in UI Automation include:

Selector and targeting improvements:

  • Implemented semantic selectors for coding agents, enabling element identification using natural-language descriptions and visual analysis when DOM-based methods are unavailable.
  • Added coding agent support for SAP GUI for Java applications.
  • Added target configuration validation for coding agents, helping ensure targeting definitions are accurate before use.
  • Updated fuzzy targets to disable check:text by default for Type Into, Set Text, Get Text, and Select Item activities, improving element targeting reliability.

Command-line improvements:

  • Expanded the uip rpa uia interact sap command set with expand-tree, select-dates-in-calendar, and click-picture-on-screen interactions.
  • Added a new uip rpa uia target-anchorable preview-text command to preview the text a Get Text activity would return for a target anchorable definition.
  • Enhanced the uip rpa uia target-anchorable resolve-defaults command to support element name and description attributes for more precise identification.
  • Improved the command-line interface (CLI) experience on macOS with system permission inspection and a new uip rpa uia permissions command to verify or request necessary permissions (Accessibility and Screen recording).

UI Automation Modern

  • Adopted the new JS-enabled code editor widget for the Script Code property of Inject Js Script activity. The widget is available on Windows projects only, starting with Studio Desktop 26.0.198. Studio Web support is not yet implemented.
  • Expanded UI Automation activity tracing to capture consumption details for Semantic Activities and ScreenPlay. Execution tracing is disabled by default and it can be enabled through the UI Automation Modern > Robot Logging > Enable activity tracing project setting. Trace data is uploaded to Orchestrator for failed jobs only.

Selection Helper UI Tree

  • Added an Enable Computer Vision button directly in the UI Tree panel. The CV elements tab is now always visible when an application is in scope and elements are being picked. When Computer Vision is disabled, the panel shows an actionable state with an Enable Computer Vision button that toggles CV on, captures a screenshot, and builds the CV element tree automatically.

Bug fixes

  • Fixed an issue where a proxy authentication dialog ("Sign in to access this site") would appear during element indication with Modern UI Automation activities, preventing users from indicating UI elements behind the dialog.

v26.10.1

Release date: July 21, 2026

Selection Helper

  • Added support for configuring offset points directly from the Selection Helper, via a new button on the target expansion panel. When updating an offset point, the associated target is automatically selected and highlighted to provide immediate visual feedback.
  • Updated the preferred UI framework selector to appear as a compact menu button in the action bar, with strategy abbreviations and icons (Auto/AA/UIA).
  • Improved Selection Helper performance, reducing change-detection and rendering overhead during hover, element selection, and layout interactions.

UI Automation support for coding agents

  • Added Computer Vision (CV) targeting support, enabling coding agents to configure CV-based targeting methods for UI elements.
  • Enhanced agent-based element discovery with image and semantic analysis capabilities. When a browser or application DOM is unavailable, agents can now identify UI elements using visual information and natural-language descriptions.
  • Improved visual diagnostics by generating annotated screenshots that highlight identified elements, plus new screenshot-size caps to control image and token costs.
  • Added support for validating target and anchor configurations through UI Automation tooling, helping ensure targeting definitions are accurate before use.

Improvements

  • Extended the UI Automation highlighting API with a Fill option that renders a solid-colored rectangle over a target element, enabling workflows to hide sensitive information or highlight multiple screen regions before capturing screenshots.

v26.10.0

Release date: June 24, 2026

Agentic UIAutomation

Semantic selectors

Semantic selectors now support two find methods: Document Object Model (DOM) and Image. DOM mode is the default for web applications and matches elements using information extracted from the application DOM. Image mode is a new find method, and the default for non-web applications, that uses the visible screen image and the image grounding capabilities of modern multimodal models to understand the visible content like a human would. It supports any kind of descriptions, visual descriptions included (where the DOM method would generally fail).

You can choose the model used for each find method from the Model drop-down in Project Settings > UI Automation > Semantic Selector. DOM mode supports Gemini 2.5 Flash and PathMini; Image mode uses PathMini.

For more information, refer to Find methods.

Semantic activities

You can now power UI Automation Semantic activities with your own large language model subscription instead of the UiPath-managed subscription. You can configure your model under UI Automation > Semantic Activities in the LLM configurations tab of the AI Trust Layer, using one of the supported models. For the setup steps, refer to Semantic activities bring-your-own-model setup.

You can now set project-level defaults for Semantic activities — the default model for each activity and the extraction mode for Extract UI Data (Preview) — under Project Settings > UI Automation Modern > Semantic Activities. These defaults apply both during Studio design-time experiences, such as selection and test runs, and at runtime. For the project-level defaults, refer to UI Automation Modern Project Settings.

Updating the UI Automation Activities package changes the default model for Semantic activities (excluding Close Popup) to PathMini. For the setup steps, refer to Semantic activities bring-your-own-model setup. For the project-level defaults, refer to UI Automation Modern Project Settings.

ScreenPlay
  • SecureString and ScreenPlayImage support in coded workflows — ScreenPlay coded workflows now handle two additional argument types: SecureString, for passing sensitive string values without exposing them in plain text, and ScreenPlayImage, for including a screenshot directly in the ScreenPlay prompt to give the model additional visual context. Previously, coded workflows had no native handling for these types; this change brings them to parity with low-code workflows.
  • Distributed tracing SpanId — ScreenPlay now adds an X-UIPATH-SEMANTIC-SPAN-ID header to the HTTP calls it makes to backend services, such as the Semantic Proxy and LLM Gateway. The span identifier lets a single ScreenPlay execution and its requests be correlated end-to-end across services, improving observability and troubleshooting.
  • Application instance attach mode — ScreenPlay now supports the Application instance window attach mode on the Use Application/Browser activity. Inner activities search across the entire application instance — all parent and child windows, alerts, and popups — instead of a single window, and ScreenPlay captures and records traces correctly across all of them, in addition to the existing Single window support.
  • Refreshed HTML trace file — The ScreenPlay HTML trace file has a refreshed visual design. The trace viewer output now has an updated layout and styling that make execution traces easier to read and navigate, aligned with current UiPath design standards.
Extract UI Data (Preview)

Extract UI Data is a new semantic, AI-powered activity for extracting structured business data from any application screen. You define the custom data structures you want to capture and retrieve them as business objects, working across UI technologies rather than being limited to a single layout type. The activity defaults to capturing webform data when a form is present, and also supports extracting custom fields through custom prompts.

The activity now handles "not found" results consistently. Previously, it threw an exception when no answer could be found for any query, but returned empty strings when only a subset of answers was missing. Now, when a value cannot be found, it is returned as null, giving a single, predictable way to detect and handle missing data across all extraction scenarios.

Extract UI Data is a preview activity — an experimental lab for schema-driven, semantic UI extraction. It is not intended to reach General Availability as a standalone activity; this functionality will be absorbed into ScreenPlay's data scraping capabilities. Users adopting Extract UI Data during preview should plan for an eventual migration to ScreenPlay.

Healing Agent

When Healing Agent cannot recover an activity, the Robot logs now include the reason for the failure. The reason appears in the Robot logs in Orchestrator, so you can see why a recovery did not succeed without inspecting internal logs. For example, a log entry might read "Healing agent could not recover the activity. Self-healing is disabled."

UI Automation support for coding agents

The 26.10.0 release brings UI automation to AI coding agents. You can now ask an agent — such as Claude Code, Cursor, GitHub Copilot — to build and maintain UI automation in plain language, and it will drive real applications, generate reliable selectors, and register them in your project's Object Repository on your behalf.

This support ships as a set of agent skills packaged alongside the activities and backed by the uip rpa uia command-line interface. The skills work across the same surfaces as the activities themselves — desktop, web, and SAP applications.

Three skills are included:

  • Configure a target (uia-configure-target) — Point the agent at a screen and the elements you care about ("set up the Submit button on the checkout page"). It captures the live application, locates each element, generates a reliable selector, registers the screen and elements in the Object Repository, and verifies each one against a screenshot before handing back the reference IDs your workflow can use.
  • Interact with a live app (uia-interact) — Ask the agent to interact with a running application — click, type, read values, fill forms, navigate menus, extract tables, or take screenshots. Useful for exploring an app before you automate it, or for confirming that a change behaves as expected.
  • Improve a selector (uia-improve-selector) — Fix, harden, or recover a selector from runtime data. See Selector intelligence below.

The skills ship in the uipath-rpa skills repository.

Selector intelligence (uia-improve-selector)

Selector intelligence is the reasoning engine behind the uia-improve-selector skill. It diagnoses why a selector failed and rewrites it — without hand-editing XML.

Selectors break when an application changes — an attribute is renamed, an auto-generated ID shifts, a layout moves. Selector intelligence lets a coding agent diagnose and rewrite a selector so your automation keeps running, without you hand-editing fragile selector XML.

It works in two modes:

  • Recover — the selector has stopped working ("element not found", a runtime failure). The agent rewrites it against the current state of the application so it matches the element again.
  • Improve — the selector still works but is fragile, because it leans on position, an auto-generated ID, or a value that reflects changing content. The agent produces a more robust version that targets the same element.

Use Improve proactively when you review a newly generated selector and notice it relies on a generated ID or positional index. Use Recover reactively when an unattended job fails and you have the runtime data.

What makes it reliable:

  • Driven by runtime data. It works from a live application or from data captured at the moment of failure — so a selector that broke on an unattended robot can be recovered later, on a different machine, even when the app is no longer running.
  • Activity-aware. The agent knows, for example, that a Get Text target should not lean on the text it is meant to read, and that a Check target should not depend on whether the box is currently ticked. Each rewrite follows the rules that keep a selector stable for its specific use.
  • Validated, not guessed. Every candidate selector is checked and scored before anything is written back. If the first attempt doesn't hold up, the agent refines it automatically and tries again, then keeps the strongest result.
  • Explained. When a selector is recovered, you get a one-line root cause (why it broke) and the strategy used to make it durable — so you can trust the change at a glance.

Chromium Automation

We've added Chromium Automation, a new browser automation method that controls Chrome and Edge directly through the Chrome DevTools Protocol (CDP). UiPath launches and manages the browser session internally — no browser extension to install and no WebDriver binary to download or keep in sync with your browser version.

Two new options are available in the Browser automation mode dropdown on the Use Application/Browser activity, and in Project Settings > UI Automation Modern > Application/Browser:

Chromium Automation works with Chrome, Edge, and all Chromium-based browsers. Firefox and Safari are not supported by this method — for those, continue using Browser Extension or WebDriver.

Oracle Automation

UI Automation now supports automating Oracle Fusion Applications across both the Oracle ADF (Application Development Framework) and Oracle Redwood (Oracle JET) interfaces. A new identification approach surfaces stable selector attributes for each technology, so automations stay reliable despite the dynamic nature of these interfaces, and the full range of UI Automation activities can be used against Oracle Fusion screens.

For more information, refer to Oracle Fusion Automation.

macOS UI Automation

  • Enhanced support for macOS menu-based controls in Select Item activities. You can now use Select Item with AXPopUpButton and AXMenuButton controls, enabling reliable interaction with dropdown and menu-based UI elements that require opening a popup to access their items.
  • Added support for triple-click actions. This capability is available on:
    • Windows and macOS using Hardware Events
    • Windows and macOS using ChromiumAPI
  • Added support for mouse drag actions on macOS.

UI Automation Modern

  • Enabled the Capture Elements feature in Object Repository in cross-platform projects.

  • The Set Runtime Browser activity can now resolve the default system browser at runtime in cross-platform projects. A new Use default system browser option allows selecting between the system default browser or a specific browser:

    • Windows projects: Chrome, Edge, Firefox
    • Cross-platform projects: Chrome, Edge, Safari

    The activity’s display name now reflects the selected browser (literal value, Default system browser, or a truncated expression).

  • Added the Accessibility Check activity which enables users to perform Accessibility Testing in Test Manager instances within Test Cloud. This activity loads a web browser page, and performs accessibility checks as per the WCAG 2.1 (or higher) standard. Examples of accessibility checks include DOM and visual accessibility, contrast ratio, alt tags, tab order, improper ARIA roles.

Advanced Selector Editor

  • Added zoom controls to the Advanced Selector Editor and Selection Helper, allowing you to zoom in or out between 80% and 110% using +, -, and Reset options.

  • Added case-sensitive support for attributes in the Advanced Selector Editor.

  • Added an Edit target option to Use Application/Browser, exposing:

    • Window selector
    • Application properties (such as Browser URL, browser arguments, application path, application arguments, and working directory)
    • The option to Update the window selector in child activities. This allows any changes made to the screen’s Window Selector to be automatically applied to all child elements.
  • The Advanced Selector Editor can now be used for editing Window selectors in addition to UI element Selectors.

  • In the UI Tree panel of UI Explorer, added a toggle to show or hide properties with empty values in the Selector compatible and Other(s) sections, helping display only relevant information.

  • Added a Filter properties option to the UI Tree panel, enabling filtering of properties for the selected node by name in both the Selector compatible and Other(s) sections.

Improvements

  • Optimized Active Accessibility selectors to improve Extract Table Data performance for large tables.

Bug fixes

  • For link type nodes, the text was not retrieved and the tree path was generated using the internal SAP id. With this fix, the proper node text is used. This affected the SAP Expand Tree activity.

  • Fixed an issue where selectors could not be identified when interacting with HTML views in the CAMBIO Cosmic application (a Java-based healthcare application using JxBrowser).

  • Fixed an issue where the cross-platform Type Into activity became unresponsive on Automation Cloud Robot VMs.

  • Fixed native capture for glyphs and Japanese characters.

  • Fixed an issue where Java extensions continued to be reported as installed after uninstalling them.

Corrections, added August 11, 2026

The following ScreenPlay entries were omitted from the original v26.10.0 announcement and apply to the v26.10.0 release.

Expanded model selection for ScreenPlay

The Model drop-down on the ScreenPlay activity now offers a wider range of Screen Agent models, adding GPT-5.4, GPT-5.4 mini, GPT-5.5, Gemini 3.0 Flash Preview, and Path Mini. The added models are more capable and faster than the ones previously available, and are the largest contributor to the reduced execution latency in this release compared to the 25.10 GA release. Path Mini is a Qwen-based model fine-tuned by UiPath for screen understanding and UI interaction, hosted and served by UiPath on its own cloud infrastructure, and is the fastest option in the selection.

The full selection is:

Basic tier:

  • Screen Agent (using Path Mini)
  • Screen Agent (using Gemini 2.5 Flash)
  • Screen Agent (using Gemini 3.0 Flash Preview)
  • Screen Agent (using GPT-4.1 mini)
  • Screen Agent (using GPT-5 mini)
  • Screen Agent (using GPT-5.4 mini)

Standard tier:

  • Screen Agent (using GPT-4.1)
  • Screen Agent (using GPT-5)
  • Screen Agent (using GPT-5.4)
  • Screen Agent (using GPT-5.5)
  • OpenAI - Operator
  • Anthropic - Computer Use

For the characteristics of each model, refer to ScreenPlay.

New default model for ScreenPlay

The default model for the ScreenPlay activity changed from Screen Agent (using Gemini 2.5 Flash) to Screen Agent (using GPT-5.4). GPT-5.4 is a Standard tier model, whereas Gemini 2.5 Flash is a Basic tier one. Existing automations keep the model they were configured with; the new default applies to newly added ScreenPlay activities.

ScreenPlay execution beyond the indicated scope

ScreenPlay is no longer confined to the application or window passed as its scope. It can now work across multiple browser tabs and multiple applications within the same task.

ScreenPlay execution fixes

This release includes fixes to ScreenPlay execution reliability.

Was this page helpful?

Connect

Need help? Support

Want to learn? UiPath Academy

Have questions? UiPath Forum

Stay updated