- Overview
- UI Automation
- About the UI Automation activity package
- Applications and technologies automated with UI Automation
- Project compatibility
- UI-ANA-016 - Pull Open Browser URL
- UI-ANA-017 - ContinueOnError True
- UI-ANA-018 - List OCR/Image Activities
- UI-DBP-006 - Container Usage
- UI-DBP-013 - Excel Automation Misuse
- UI-DBP-030 - Forbidden Variables Usage In Selectors
- UI-DBP-031 - Activity verification
- UI-PRR-001 - Simulate Click
- UI-PRR-002 - Simulate Type
- UI-PRR-003 - Open Application Misuse
- UI-PRR-004 - Hardcoded Delays
- UI-REL-001 - Large Idx in Selectors
- UI-SEC-004 - Selector Email Data
- UI-SEC-010 - App/Url Restrictions
- UI-USG-011 - Non Allowed Attributes
- UX-SEC-010 - App/Url Restrictions
- UX-DBP-029 - Insecure Password Use
- UI-PST-001 - Audit Log Level in Project Settings
- UiPath Browser Migration Tool
- Clipping region
- Computer Vision Recorder
- Activities index
- Activate
- Anchor Base
- Attach Browser
- Attach Window
- Block User Input
- Callout
- Check
- Click
- Click Image
- Click Image Trigger
- Click OCR Text
- Click Text
- Click Trigger
- Close Application
- Close Tab
- Close Window
- Context Aware Anchor
- Copy Selected Text
- Element Attribute Change Trigger
- Element Exists
- Element Scope
- Element State Change Trigger
- Export UI Tree
- Extract Structured Data
- Find Children
- Find Element
- Find Image
- Find Image Matches
- Find OCR Text Position
- Find Relative Element
- Find Text Position
- Get Active Window
- Get Ancestor
- Get Attribute
- Get Event Info
- Get From Clipboard
- Get Full Text
- Get OCR Text
- Get Password
- Get Position
- Get Source Element
- Get Text
- Get Visible Text
- Go Back
- Go Forward
- Go Home
- Google Cloud Vision OCR
- Hide Window
- Highlight
- Hotkey Trigger
- Hover
- Hover Image
- Hover OCR Text
- Hover Text
- Image Exists
- Indicate On Screen
- Inject .NET Code
- Inject Js Script
- Invoke ActiveX Method
- Key Press Trigger
- Load Image
- Maximize Window
- Microsoft Azure Computer Vision OCR
- Microsoft OCR
- Microsoft Project Oxford Online OCR
- Minimize Window
- Monitor Events
- Mouse Trigger
- Move Window
- Navigate To
- OCR Text Exists
- On Element Appear
- On Element Vanish
- On Image Appear
- On Image Vanish
- Open Application
- Open Browser
- Refresh Browser
- Replay User Event
- Restore Window
- Save Image
- Select Item
- Select Multiple Items
- Send Hotkey
- Set Clipping Region
- Set Focus
- Set Text
- Set To Clipboard
- Set Web Attribute
- Show Window
- Start Process
- System Trigger
- Take Screenshot
- Tesseract OCR
- Text Exists
- Tooltip
- Type Into
- Type Secure Text
- Use Foreground
- Wait Attribute
- Wait Element Vanish
- Wait Image Vanish
- Accessibility Check
- Application Event Trigger
- Block User Input
- Check/Uncheck
- Check App State
- Check Element
- Click
- Click Event Trigger
- Drag and Drop
- Element Scope
- Extract Table Data
- Find Elements
- For Each UI Element
- Get Browser Data
- Get Clipboard
- Get Text
- Get URL
- Go to URL
- Highlight
- Hover
- Inject Js Script
- Keyboard Shortcuts
- Keypress Event Trigger
- Mouse Scroll
- Navigate Browser
- Save Image
- Select Item
- Set Browser Data
- Set Clipboard
- Set CV Server
- Set Runtime Browser
- Set Focus
- Set Project Setting
- Set Text
- Take Screenshot
- Type Into
- Unblock User Input
- Window Operation
- Perform browser search and retrieve results using UI Automation APIs
- Web Browsing
- Find Images
- Click Images
- Trigger and Monitor Events
- Create and Override Files
- HTML Pages: Extract and Manipulate Information
- Window Manipulation
- Automated List Selection
- Find and Manipulate Window Elements
- Manage Text Automation
- Load and Process Images
- Manage Mouse Activated Actions
- Automate Application Runtime
- Automated Run of a Local Application
- Browser Navigation
- Web Automation
- Trigger Scope Example
- Enable UI Automation support in DevExpress
- Computer Vision Local Server
- Mobile Automation
- Release notes
- About the mobile device automation architecture
- Project compatibility
- Get Log Types
- Get Logs
- Get Page Source
- Get Device Orientation
- Get Session Identifier
- Install App
- Manage Current App
- Manage Other App
- Open DeepLink
- Open URL
- Mobile Device Connection
- Directional Swipe
- Draw Pattern
- Positional Swipe
- Press Hardware Button
- Set Device Orientation
- Take Screenshot
- Take Screenshot Part
- Element Exists
- Execute Command
- Get Attribute
- Get Selected Item
- Get Text
- Set Selected Item
- Set Text
- Swipe
- Tap
- Type Text
- Terminal
- Release notes
- About the Terminal activity package
- Project compatibility
- Best practices
- Find Text
- Get Color at Position
- Get Cursor Position
- Get Field
- Get Field at Position
- Get Screen Area
- Get Text
- Get Text at Position
- Move Cursor
- Move Cursor to Text
- Send Control Key
- Send Keys
- Send Keys Secure
- Set Field
- Set Field at Position
- Terminal Session
- Wait Field Text
- Wait Screen Ready
- Wait Screen Text
- Wait Text at Position
- Terminal coded automation APIs
v26.10 release notes for UI Automation activities, versions 26.10.x.
v26.10.2
Release date: August 10, 2026
UI Automation support for coding agents
Improvements for coding agents in UI Automation include:
Selector and targeting improvements:
- Implemented semantic selectors for coding agents, enabling element identification using natural-language descriptions and visual analysis when DOM-based methods are unavailable.
- Added coding agent support for SAP GUI for Java applications.
- Added target configuration validation for coding agents, helping ensure targeting definitions are accurate before use.
- Updated fuzzy targets to disable
check:textby default for Type Into, Set Text, Get Text, and Select Item activities, improving element targeting reliability.
Command-line improvements:
- Expanded the
uip rpa uia interact sapcommand set withexpand-tree,select-dates-in-calendar, andclick-picture-on-screeninteractions. - Added a new
uip rpa uia target-anchorable preview-textcommand to preview the text a Get Text activity would return for a target anchorable definition. - Enhanced the
uip rpa uia target-anchorable resolve-defaultscommand to support elementnameanddescriptionattributes for more precise identification. - Improved the command-line interface (CLI) experience on macOS with system permission inspection and a new
uip rpa uia permissionscommand to verify or request necessary permissions (Accessibility and Screen recording).
UI Automation Modern
- Adopted the new JS-enabled code editor widget for the Script Code property of Inject Js Script activity. The widget is available on Windows projects only, starting with Studio Desktop 26.0.198. Studio Web support is not yet implemented.
- Expanded UI Automation activity tracing to capture consumption details for Semantic Activities and ScreenPlay. Execution tracing is disabled by default and it can be enabled through the UI Automation Modern > Robot Logging > Enable activity tracing project setting. Trace data is uploaded to Orchestrator for failed jobs only.
Selection Helper UI Tree
- Added an Enable Computer Vision button directly in the UI Tree panel. The CV elements tab is now always visible when an application is in scope and elements are being picked. When Computer Vision is disabled, the panel shows an actionable state with an Enable Computer Vision button that toggles CV on, captures a screenshot, and builds the CV element tree automatically.
Bug fixes
- Fixed an issue where a proxy authentication dialog ("Sign in to access this site") would appear during element indication with Modern UI Automation activities, preventing users from indicating UI elements behind the dialog.
v26.10.1
Release date: July 21, 2026
Selection Helper
- Added support for configuring offset points directly from the Selection Helper, via a new button on the target expansion panel. When updating an offset point, the associated target is automatically selected and highlighted to provide immediate visual feedback.
- Updated the preferred UI framework selector to appear as a compact menu button in the action bar, with strategy abbreviations and icons (Auto/AA/UIA).
- Improved Selection Helper performance, reducing change-detection and rendering overhead during hover, element selection, and layout interactions.
UI Automation support for coding agents
- Added Computer Vision (CV) targeting support, enabling coding agents to configure CV-based targeting methods for UI elements.
- Enhanced agent-based element discovery with image and semantic analysis capabilities. When a browser or application DOM is unavailable, agents can now identify UI elements using visual information and natural-language descriptions.
- Improved visual diagnostics by generating annotated screenshots that highlight identified elements, plus new screenshot-size caps to control image and token costs.
- Added support for validating target and anchor configurations through UI Automation tooling, helping ensure targeting definitions are accurate before use.
Improvements
- Extended the UI Automation highlighting API with a Fill option that renders a solid-colored rectangle over a target element, enabling workflows to hide sensitive information or highlight multiple screen regions before capturing screenshots.
v26.10.0
Release date: June 24, 2026
Agentic UIAutomation
Semantic selectors
Semantic selectors now support two find methods: Document Object Model (DOM) and Image. DOM mode is the default for web applications and matches elements using information extracted from the application DOM. Image mode is a new find method, and the default for non-web applications, that uses the visible screen image and the image grounding capabilities of modern multimodal models to understand the visible content like a human would. It supports any kind of descriptions, visual descriptions included (where the DOM method would generally fail).
You can choose the model used for each find method from the Model drop-down in Project Settings > UI Automation > Semantic Selector. DOM mode supports Gemini 2.5 Flash and PathMini; Image mode uses PathMini.
For more information, refer to Find methods.
Semantic activities
You can now power UI Automation Semantic activities with your own large language model subscription instead of the UiPath-managed subscription. You can configure your model under UI Automation > Semantic Activities in the LLM configurations tab of the AI Trust Layer, using one of the supported models. For the setup steps, refer to Semantic activities bring-your-own-model setup.
You can now set project-level defaults for Semantic activities — the default model for each activity and the extraction mode for Extract UI Data (Preview) — under Project Settings > UI Automation Modern > Semantic Activities. These defaults apply both during Studio design-time experiences, such as selection and test runs, and at runtime. For the project-level defaults, refer to UI Automation Modern Project Settings.
Updating the UI Automation Activities package changes the default model for Semantic activities (excluding Close Popup) to PathMini. For the setup steps, refer to Semantic activities bring-your-own-model setup. For the project-level defaults, refer to UI Automation Modern Project Settings.
ScreenPlay
- SecureString and ScreenPlayImage support in coded workflows — ScreenPlay coded workflows now handle two additional argument types:
SecureString, for passing sensitive string values without exposing them in plain text, andScreenPlayImage, for including a screenshot directly in the ScreenPlay prompt to give the model additional visual context. Previously, coded workflows had no native handling for these types; this change brings them to parity with low-code workflows. - Distributed tracing SpanId — ScreenPlay now adds an
X-UIPATH-SEMANTIC-SPAN-IDheader to the HTTP calls it makes to backend services, such as the Semantic Proxy and LLM Gateway. The span identifier lets a single ScreenPlay execution and its requests be correlated end-to-end across services, improving observability and troubleshooting. - Application instance attach mode — ScreenPlay now supports the Application instance window attach mode on the Use Application/Browser activity. Inner activities search across the entire application instance — all parent and child windows, alerts, and popups — instead of a single window, and ScreenPlay captures and records traces correctly across all of them, in addition to the existing Single window support.
- Refreshed HTML trace file — The ScreenPlay HTML trace file has a refreshed visual design. The trace viewer output now has an updated layout and styling that make execution traces easier to read and navigate, aligned with current UiPath design standards.
Extract UI Data (Preview)
Extract UI Data is a new semantic, AI-powered activity for extracting structured business data from any application screen. You define the custom data structures you want to capture and retrieve them as business objects, working across UI technologies rather than being limited to a single layout type. The activity defaults to capturing webform data when a form is present, and also supports extracting custom fields through custom prompts.
The activity now handles "not found" results consistently. Previously, it threw an exception when no answer could be found for any query, but returned empty strings when only a subset of answers was missing. Now, when a value cannot be found, it is returned as null, giving a single, predictable way to detect and handle missing data across all extraction scenarios.
Extract UI Data is a preview activity — an experimental lab for schema-driven, semantic UI extraction. It is not intended to reach General Availability as a standalone activity; this functionality will be absorbed into ScreenPlay's data scraping capabilities. Users adopting Extract UI Data during preview should plan for an eventual migration to ScreenPlay.
Healing Agent
When Healing Agent cannot recover an activity, the Robot logs now include the reason for the failure. The reason appears in the Robot logs in Orchestrator, so you can see why a recovery did not succeed without inspecting internal logs. For example, a log entry might read "Healing agent could not recover the activity. Self-healing is disabled."
UI Automation support for coding agents
The 26.10.0 release brings UI automation to AI coding agents. You can now ask an agent — such as Claude Code, Cursor, GitHub Copilot — to build and maintain UI automation in plain language, and it will drive real applications, generate reliable selectors, and register them in your project's Object Repository on your behalf.
This support ships as a set of agent skills packaged alongside the activities and backed by the uip rpa uia command-line interface. The skills work across the same surfaces as the activities themselves — desktop, web, and SAP applications.
Three skills are included:
- Configure a target (
uia-configure-target) — Point the agent at a screen and the elements you care about ("set up the Submit button on the checkout page"). It captures the live application, locates each element, generates a reliable selector, registers the screen and elements in the Object Repository, and verifies each one against a screenshot before handing back the reference IDs your workflow can use. - Interact with a live app (
uia-interact) — Ask the agent to interact with a running application — click, type, read values, fill forms, navigate menus, extract tables, or take screenshots. Useful for exploring an app before you automate it, or for confirming that a change behaves as expected. - Improve a selector (
uia-improve-selector) — Fix, harden, or recover a selector from runtime data. See Selector intelligence below.
The skills ship in the uipath-rpa skills repository.
Selector intelligence (uia-improve-selector)
Selector intelligence is the reasoning engine behind the uia-improve-selector skill. It diagnoses why a selector failed and rewrites it — without hand-editing XML.
Selectors break when an application changes — an attribute is renamed, an auto-generated ID shifts, a layout moves. Selector intelligence lets a coding agent diagnose and rewrite a selector so your automation keeps running, without you hand-editing fragile selector XML.
It works in two modes:
- Recover — the selector has stopped working ("element not found", a runtime failure). The agent rewrites it against the current state of the application so it matches the element again.
- Improve — the selector still works but is fragile, because it leans on position, an auto-generated ID, or a value that reflects changing content. The agent produces a more robust version that targets the same element.
Use Improve proactively when you review a newly generated selector and notice it relies on a generated ID or positional index. Use Recover reactively when an unattended job fails and you have the runtime data.
What makes it reliable:
- Driven by runtime data. It works from a live application or from data captured at the moment of failure — so a selector that broke on an unattended robot can be recovered later, on a different machine, even when the app is no longer running.
- Activity-aware. The agent knows, for example, that a Get Text target should not lean on the text it is meant to read, and that a Check target should not depend on whether the box is currently ticked. Each rewrite follows the rules that keep a selector stable for its specific use.
- Validated, not guessed. Every candidate selector is checked and scored before anything is written back. If the first attempt doesn't hold up, the agent refines it automatically and tries again, then keeps the strongest result.
- Explained. When a selector is recovered, you get a one-line root cause (why it broke) and the strategy used to make it durable — so you can trust the change at a glance.
Chromium Automation
We've added Chromium Automation, a new browser automation method that controls Chrome and Edge directly through the Chrome DevTools Protocol (CDP). UiPath launches and manages the browser session internally — no browser extension to install and no WebDriver binary to download or keep in sync with your browser version.
Two new options are available in the Browser automation mode dropdown on the Use Application/Browser activity, and in Project Settings > UI Automation Modern > Application/Browser:
- Chromium Automation - Best for development, debugging, unattended automation with a visible browser
- Chromium Automation Headless - Best for unattended automation on servers and CI/CD pipelines
Chromium Automation works with Chrome, Edge, and all Chromium-based browsers. Firefox and Safari are not supported by this method — for those, continue using Browser Extension or WebDriver.
Oracle Automation
UI Automation now supports automating Oracle Fusion Applications across both the Oracle ADF (Application Development Framework) and Oracle Redwood (Oracle JET) interfaces. A new identification approach surfaces stable selector attributes for each technology, so automations stay reliable despite the dynamic nature of these interfaces, and the full range of UI Automation activities can be used against Oracle Fusion screens.
For more information, refer to Oracle Fusion Automation.
macOS UI Automation
- Enhanced support for macOS menu-based controls in Select Item activities. You can now use Select Item with
AXPopUpButtonandAXMenuButtoncontrols, enabling reliable interaction with dropdown and menu-based UI elements that require opening a popup to access their items. - Added support for triple-click actions. This capability is available on:
- Windows and macOS using Hardware Events
- Windows and macOS using ChromiumAPI
- Added support for mouse drag actions on macOS.
UI Automation Modern
-
Enabled the Capture Elements feature in Object Repository in cross-platform projects.
-
The Set Runtime Browser activity can now resolve the default system browser at runtime in cross-platform projects. A new Use default system browser option allows selecting between the system default browser or a specific browser:
- Windows projects: Chrome, Edge, Firefox
- Cross-platform projects: Chrome, Edge, Safari
The activity’s display name now reflects the selected browser (literal value, Default system browser, or a truncated expression).
-
Added the Accessibility Check activity which enables users to perform Accessibility Testing in Test Manager instances within Test Cloud. This activity loads a web browser page, and performs accessibility checks as per the WCAG 2.1 (or higher) standard. Examples of accessibility checks include DOM and visual accessibility, contrast ratio, alt tags, tab order, improper ARIA roles.
Advanced Selector Editor
-
Added zoom controls to the Advanced Selector Editor and Selection Helper, allowing you to zoom in or out between 80% and 110% using +, -, and Reset options.
-
Added case-sensitive support for attributes in the Advanced Selector Editor.
-
Added an Edit target option to Use Application/Browser, exposing:
- Window selector
- Application properties (such as Browser URL, browser arguments, application path, application arguments, and working directory)
- The option to Update the window selector in child activities. This allows any changes made to the screen’s Window Selector to be automatically applied to all child elements.
-
The Advanced Selector Editor can now be used for editing Window selectors in addition to UI element Selectors.
-
In the UI Tree panel of UI Explorer, added a toggle to show or hide properties with empty values in the Selector compatible and Other(s) sections, helping display only relevant information.
-
Added a Filter properties option to the UI Tree panel, enabling filtering of properties for the selected node by name in both the Selector compatible and Other(s) sections.
Improvements
- Optimized Active Accessibility selectors to improve Extract Table Data performance for large tables.
Bug fixes
-
For link type nodes, the text was not retrieved and the tree path was generated using the internal SAP id. With this fix, the proper node text is used. This affected the SAP Expand Tree activity.
-
Fixed an issue where selectors could not be identified when interacting with HTML views in the CAMBIO Cosmic application (a Java-based healthcare application using JxBrowser).
-
Fixed an issue where the cross-platform Type Into activity became unresponsive on Automation Cloud Robot VMs.
-
Fixed native capture for glyphs and Japanese characters.
-
Fixed an issue where Java extensions continued to be reported as installed after uninstalling them.
Corrections, added August 11, 2026
The following ScreenPlay entries were omitted from the original v26.10.0 announcement and apply to the v26.10.0 release.
Expanded model selection for ScreenPlay
The Model drop-down on the ScreenPlay activity now offers a wider range of Screen Agent models, adding GPT-5.4, GPT-5.4 mini, GPT-5.5, Gemini 3.0 Flash Preview, and Path Mini. The added models are more capable and faster than the ones previously available, and are the largest contributor to the reduced execution latency in this release compared to the 25.10 GA release. Path Mini is a Qwen-based model fine-tuned by UiPath for screen understanding and UI interaction, hosted and served by UiPath on its own cloud infrastructure, and is the fastest option in the selection.
The full selection is:
Basic tier:
- Screen Agent (using Path Mini)
- Screen Agent (using Gemini 2.5 Flash)
- Screen Agent (using Gemini 3.0 Flash Preview)
- Screen Agent (using GPT-4.1 mini)
- Screen Agent (using GPT-5 mini)
- Screen Agent (using GPT-5.4 mini)
Standard tier:
- Screen Agent (using GPT-4.1)
- Screen Agent (using GPT-5)
- Screen Agent (using GPT-5.4)
- Screen Agent (using GPT-5.5)
- OpenAI - Operator
- Anthropic - Computer Use
For the characteristics of each model, refer to ScreenPlay.
New default model for ScreenPlay
The default model for the ScreenPlay activity changed from Screen Agent (using Gemini 2.5 Flash) to Screen Agent (using GPT-5.4). GPT-5.4 is a Standard tier model, whereas Gemini 2.5 Flash is a Basic tier one. Existing automations keep the model they were configured with; the new default applies to newly added ScreenPlay activities.
ScreenPlay execution beyond the indicated scope
ScreenPlay is no longer confined to the application or window passed as its scope. It can now work across multiple browser tabs and multiple applications within the same task.
ScreenPlay execution fixes
This release includes fixes to ScreenPlay execution reliability.
- v26.10.2
- UI Automation support for coding agents
- UI Automation Modern
- Selection Helper UI Tree
- Bug fixes
- v26.10.1
- Selection Helper
- UI Automation support for coding agents
- Improvements
- v26.10.0
- Agentic UIAutomation
- Healing Agent
- UI Automation support for coding agents
- Chromium Automation
- Oracle Automation
- macOS UI Automation
- UI Automation Modern
- Advanced Selector Editor
- Improvements
- Bug fixes
- Corrections, added August 11, 2026