🌉 UIBridge - AI-Powered Browser Automation for AI Agents

Primary Method: Use PowerShell helper functions for reliable, tested browser automation with built-in error handling and JSON validation.

🎯 What UIBridge Does

UIBridge provides reliable browser automation specifically designed for AI agents. It runs automation against a real browser with visual feedback and comprehensive error handling.

Perfect for: AI agents, automation testing, web scraping, form filling, UI testing.

---

🚀 Quick Start for AI Agents (Recommended Method)

Step 1: Install UIBridge

1npm install @sashbot/uibridge

Step 2: Start the Server

1# Start the UIBridge server (keep this running)
2node node_modules/@sashbot/uibridge/api-server.cjs

✅ Server Status: You should see UIBridge API Server running on http://localhost:3002

Step 3: Load PowerShell Helper Functions

1# Download the helper functions (includes JSON corruption fixes)
2curl -o uibridge-powershell-helpers.ps1 https://unpkg.com/@sashbot/uibridge@latest/uibridge-powershell-helpers.ps1
3
4# Load the functions (do this first in every session)
5. .\uibridge-powershell-helpers.ps1

Step 4: Verify Everything Works

1# Test server connection
2Test-UIBridgeServer
3
4# Start a complete automation session (opens browser and takes screenshot)
5Start-UIBridgeSession -Url "https://example.com"

🎉 That's it! You now have reliable browser automation with proper error handling.

---

🤖 Core AI Agent Commands (Primary Method)

🔗 Connection & Navigation

1# Always load helpers first in every session
2. .\uibridge-powershell-helpers.ps1
3
4# Test server connection
5Test-UIBridgeServer
6
7# Navigate to any URL
8Open-UIBridgePage -Url "https://example.com"
9Open-UIBridgePage -Url "http://localhost:3000"
10
11# Get current page information
12Get-UIBridgePageInfo

🖱️ Clicking Elements (Robust with Multiple Strategies)

1# Click by text content (RECOMMENDED - most reliable)
2Click-UIBridgeText -Text "Submit"
3Click-UIBridgeText -Text "Login"
4Click-UIBridgeText -Text "Save Changes"
5
6# Click by test ID (best for testing)
7Click-UIBridgeTestId -TestId "submit-btn"
8Click-UIBridgeTestId -TestId "login-button"
9
10# Click by CSS selector (when other methods don't work)
11Click-UIBridgeElement -Selector "#save-button"
12Click-UIBridgeElement -Selector ".submit-btn"
13Click-UIBridgeElement -Selector "button[type='submit']"

📸 Screenshots (Auto-Saved with Validation)

1# Basic screenshot (recommended)
2Take-UIBridgeScreenshot
3
4# Full page screenshot with options
5Take-UIBridgeScreenshot -FullPage $true -Format "png" -Quality 90
6
7# Screenshot with specific background color
8Take-UIBridgeScreenshot -BackgroundColor "white"
9Take-UIBridgeScreenshot -BackgroundColor "transparent"
10Take-UIBridgeScreenshot -BackgroundColor "auto"  # Auto-detect page background
11
12# List all saved screenshots
13Get-UIBridgeScreenshots

🔄 Complete Automation Workflows

1# Complete automation session with error handling
2Start-UIBridgeSession -Url "https://example.com"
3
4# Example: Form automation workflow
5. .\uibridge-powershell-helpers.ps1
6Test-UIBridgeServer
7Open-UIBridgePage -Url "https://example.com/form"
8Take-UIBridgeScreenshot  # Before
9Click-UIBridgeText -Text "Fill Form"
10Take-UIBridgeScreenshot  # After
11Click-UIBridgeTestId -TestId "submit-btn"
12Take-UIBridgeScreenshot  # Final result
13Get-UIBridgeScreenshots  # Review all screenshots

---

✅ Why PowerShell Helpers Are the Primary Method

🛡️ Built-in Reliability (v1.2.5+ includes JSON corruption fixes)

• JSON Validation: Every command validates JSON before sending to prevent corruption errors
• Consistent Formatting: All JSON uses proper depth and escaping
• Error Handling: Clear error messages with troubleshooting guidance
• Special Character Support: Handles quotes, apostrophes, unicode, emoji correctly

🎯 AI Agent Optimized

• Simple Function Calls: Click-UIBridgeText -Text "Submit" instead of complex JSON
• Multiple Click Strategies: Automatic fallback between text, testId, and CSS selectors
• Auto-Save Screenshots: Timestamps and organized file management
• Connection Testing: Built-in server health checks

🧪 Comprehensive Testing

• 17 Unit Tests: JSON generation, special characters, error handling
• Integration Tests: Real server and browser testing
• Regression Prevention: Prevents the return of JSON corruption bugs

1# Run tests to verify everything works
2pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1

---

🔧 When to Use Alternative Methods

The PowerShell helpers are the recommended primary method, but use alternatives for these specific scenarios:

🌐 Live Session Method (Visual Debugging)

Use when: You need to see automation happen in real-time in your own browser

1<!-- Add to your web app for live visual feedback -->
2<script src="http://localhost:3002/uibridge-client.js"></script>

1# Download live session helpers
2curl -o uibridge-live-session-helpers.ps1 https://unpkg.com/@sashbot/uibridge@latest/uibridge-live-session-helpers.ps1
3. .\uibridge-live-session-helpers.ps1
4
5# Control YOUR browser live
6Click-UIBridgeLiveText -Text "Submit"  # You see it happen in your browser!

🔧 Direct HTTP API

Use when:

• Working from non-PowerShell environments (Python, Node.js, etc.)
• Need custom JSON structures not supported by helpers
• Building your own wrapper functions

1# Direct HTTP for other languages
2curl -X POST http://localhost:3002/execute \
3  -H "Content-Type: application/json" \
4  -d '{"command":"click","selector":{"text":"Submit"}}'

📦 NPM Package Integration

Use when: Building JavaScript/TypeScript applications with UIBridge

1import { UIBridge } from '@sashbot/uibridge';
2const bridge = new UIBridge();
3await bridge.click({ text: 'Submit' });

---

🚨 Troubleshooting & Error Resolution

Problem: JSON corruption errors

✅ FIXED in v1.2.5+ The PowerShell helpers now generate valid JSON consistently:

1# Verify the fix works
2pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1
3# Should show: All tests passed! ✅

Problem: Server not accessible

1Test-UIBridgeServer
2# If this fails:
3# 1. Restart server: node node_modules/@sashbot/uibridge/api-server.cjs
4# 2. Check firewall/antivirus blocking port 3002
5# 3. Try different port: . .\uibridge-powershell-helpers.ps1 -ServerUrl "http://localhost:3003"

Problem: Elements not found

1# Try multiple strategies in order:
2Click-UIBridgeText -Text "Button Text"        # Try this first (most reliable)
3Click-UIBridgeTestId -TestId "button-id"      # If text doesn't work  
4Click-UIBridgeElement -Selector "#button-id"  # If test ID doesn't work
5
6# Take screenshot to see current page state
7Take-UIBridgeScreenshot

Problem: Special characters in text

1# These all work correctly with the fixed helpers:
2Click-UIBridgeText -Text "Text with ""quotes"" and 'apostrophes'"
3Click-UIBridgeText -Text "Button with emoji 🚀"
4Click-UIBridgeText -Text "Unicode: 测试 العربية русский"

---

🎯 AI Agent Best Practices

✅ DO (Primary Method):

1# 1. Always load helpers first
2. .\uibridge-powershell-helpers.ps1
3
4# 2. Test connection before automating
5Test-UIBridgeServer
6
7# 3. Use text-based clicking first (most reliable)
8Click-UIBridgeText -Text "Submit"
9
10# 4. Take screenshots to verify actions
11Take-UIBridgeScreenshot
12
13# 5. Use complete workflows
14Start-UIBridgeSession -Url "https://example.com"

⚠️ AVOID:

• Don't construct JSON manually - use the helper functions
• Don't skip Test-UIBridgeServer - always verify connection
• Don't use complex CSS selectors if text clicking works
• Don't forget to handle errors with try/catch blocks

---

📊 Helper Functions Reference

Core Functions

Function	Purpose	Example
Test-UIBridgeServer	Check server connection	Test-UIBridgeServer
Open-UIBridgePage	Navigate to URL	Open-UIBridgePage -Url "https://example.com"
Take-UIBridgeScreenshot	Capture page screenshot	Take-UIBridgeScreenshot -FullPage $true
Click-UIBridgeText	Click by text content	Click-UIBridgeText -Text "Submit"
Click-UIBridgeTestId	Click by test ID	Click-UIBridgeTestId -TestId "submit-btn"
Click-UIBridgeElement	Click by CSS selector	Click-UIBridgeElement -Selector "#btn"
Get-UIBridgePageInfo	Get page title/URL	Get-UIBridgePageInfo
Get-UIBridgeScreenshots	List saved screenshots	Get-UIBridgeScreenshots
Start-UIBridgeSession	Complete automation setup	Start-UIBridgeSession -Url "https://example.com"

Advanced Usage

1# Custom timeout for slow pages
2Click-UIBridgeText -Text "Submit" -Timeout 10000
3
4# High quality screenshot
5Take-UIBridgeScreenshot -Quality 95 -Format "png"
6
7# Multiple screenshots for comparison
8Take-UIBridgeScreenshot  # Before action
9Click-UIBridgeText -Text "Change Background"
10Take-UIBridgeScreenshot  # After action

---

🔬 Testing & Validation

Run Built-in Tests

1# Test JSON generation and helper functions
2pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1
3
4# Expected result:
5# 📊 Test Results Summary
6# Total Tests: 17
7# Passed: 17 ✅
8# Failed: 0 ✅

Integration with CI/CD

1# In your build scripts
2$testResult = & pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1
3if ($LASTEXITCODE -ne 0) {
4    throw "UIBridge PowerShell helper tests failed"
5}

---

🎉 Why UIBridge PowerShell Helpers Are Perfect for AI Agents

1. 🛡️ Reliability: Built-in JSON validation prevents corruption errors
2. 🎯 Simplicity: Click-UIBridgeText -Text "Submit" instead of complex APIs
3. 📸 Auto-Management: Screenshots auto-save with timestamps
4. 🔍 Error Handling: Clear error messages with troubleshooting guidance
5. 🤖 AI-Optimized: Functions designed specifically for AI agent workflows
6. ⚡ Fast Setup: Load helpers, test connection, start automating
7. 🧪 Tested: 17 unit tests ensure reliability
8. 🌐 Universal: Works with any web application

---

📚 Additional Resources

• 🔧 Advanced Configuration
• 🤖 AI Agent Guidelines
• 📦 NPM Package
• 🐛 GitHub Issues

---

🌉 Built for AI Agents: UIBridge provides the most reliable browser automation for AI agents with comprehensive error handling, JSON validation, and intuitive PowerShell functions.