Short: Object-level AmigaOS GUI automation Author: simond@irrelevant.org (Simon Dick) Uploader: simond irrelevant org (Simon Dick) Type: util/misc Version: 1.1 Architecture: m68k-amigaos Distribution: Aminet AmiPilot is an object-level GUI automation system for classic AmigaOS: find a window or gadget by ID, label, or role; act on it with genuinely synthesised input (real input.device events, not coordinate guessing); assert on the result. This archive has two pieces: - AmiInspect: a standalone Shell command that walks any window's gadget tree and prints its roles, labels, classes, IDs, positions, and states - the platform's first UIA-Inspect / browser element- picker equivalent. No host machine, server, or session required. - AmiPilotServer: a commodity that hosts the same walker plus a real action engine (click, type, drag, read state, launch a subject under test, move files, walk menus) behind a public ARexx port and, from 0.3, the same command set over a host-facing wire protocol - serial.device, or (from 0.4) TCP - so a script running on the SAME Amiga, or a Python client on a host machine (over the wire, or now a real serial port directly), can drive another program's GUI. See "Documentation" below for the full command/wire reference. Features -------- - Prints any window's gadget tree by role, label, class, ID, position, and state - target the active window, or any window by a substring of its title. - Plain GadTools gadgets: correctly distinguishes a checkbox from a button even though both produce the identical underlying gadget type, using GadTools' own documented attribute-probing contract rather than a guess. - BOOPSI/ReAction gadgets: reads the real, live class name (button.gadget, checkbox.gadget, layout.gadget, and more) via a documented NDK mechanism, and maps known classes to a role. - Genuine input.device event synthesis for clicks, typed text, and (from 0.4) drags (IECLASS_NEWPOINTERPOS/RAWKEY/RAWMOUSE), not coordinate-level or posted-message shortcuts - driven from an ARexx port (AmiPilotServer) or the same commands over serial.device or TCP from a host Python client. - From 0.4: locate a gadget by role and label text, or by position among several matches (ROLE=/LABEL=/INDEX=), not just by numeric ID or a manifest name; drag a gadget by a pixel offset (for a slider/scroller) or onto a second gadget (drag-and-drop/reorder); launch the test subject itself over the wire (LAUNCH); read and manage files inside an allowlisted directory (the file API); walk and select a window's live menu strip (MENU/MENUPICK). - Verified against real, unmodified stock AmigaOS software (including a real crash-causing bug found and fixed this way) and purpose-built conformance fixtures, not just software built for this project. - Honest, documented limits rather than silent gaps: a few specific, well-understood cases (PLACETEXT_IN and PLACETEXT_RIGHT button labels, a window.class window's layout.gadget-nested children) are genuinely unreadable at this tier, and AmiInspect says so rather than guessing - see the included guide's "Locator Tiers and Limits" page, including a table of which locator tier reaches which kind of UI and why. - From 0.5: wait for a real condition (a window appearing/closing, a gadget's text matching) instead of a guessed sleep, closing the classic click-then-check race (WAITFOR, CLICK's EXPECT=); community- authored "quirk profiles" for third-party applications you don't control, using the same manifest file format; a golden-tree mechanism for catching structural UI drift as a regression; and a bridge into MUI (Magic User Interface) applications through their own built-in ARexx port (MUIREXX). - From 1.0: push files from the host onto the Amiga, not just read them back (FSPUT); launch a program the way Workbench itself does - a genuine WBStartup message with tooltype overrides and project arguments (WBLAUNCH); capture a screen or window as a screenshot, on classic planar screens or genuine Picasso96/RTG boards, decoded host-side to PNG and IFF ILBM (SCREENSHOT); move and resize whole windows via the same synthesized drags gadgets already get (WINDOWMOVE/WINDOWSIZE); and wait for an Intuition Requester to appear (WAITFOR REQUESTER, detection only). - From 1.1: point at a gadget on the real screen and get back its exact locator interactively, no batch dump needed (PICK), reachable standing at the machine (AmiInspect PICK), from an ARexx script, or over the wire; CLICK now genuinely dismisses Intuition Requesters too, both window-owned and system-wide (e.g. a real disk-swap prompt); menu items with no keyboard shortcut are now picked via a genuine synthesized pointer sequence instead of being rejected outright; a cooperative escape hatch (WHERE/WHEREGADGET) reaches gadgets nested inside a window.class window's layout.gadget - permanently invisible to plain structural walking - via a small, optional ARexx port the target application itself exposes; INTEGER_KIND GadTools gadgets are now told apart from STRING_KIND; and twelve more WB3.2-era BOOPSI/ReAction gadget classes (clicktab, colorwheel, datebrowser, fuelgauge, four get*.gadget choosers, gradientslider, palette, sketchboard, speedbar, texteditor) get a real role instead of reporting as custom. Requirements ------------ - AmigaOS 2.04 (V37) or later. Plain 68000, no FPU - nothing here does float work or needs anything newer than a stock CPU. - gadtools.library (any version) is optional but recommended - without it, gadget classification degrades gracefully rather than failing. - rexxsyslib.library is required to run AmiPilotServer's ARexx port (AmiInspect needs neither this nor gadtools.library to run at all). Installation ------------ Copy "AmiInspect" and/or "AmiPilotServer" to C: (or anywhere on your command Path). No reboot, no configuration. Quick start ----------- AmiInspect inspect the active window AmiInspect WINDOW=Prefs inspect the first window whose title contains "Prefs" Run AmiPilotServer start the commodity; opens an ARexx port named AMIPILOT.1 (or the next free slot) Run AmiPilotServer SERIAL also carry the same commands over serial.device, for a host Python client Run AmiPilotServer TCP also carry the same commands over TCP TCPPORT=6800 (bsdsocket.library), for real hardware or an emulator with no serial bridge AI disclosure ------------- Be aware: AmiPilot was written largely by an AI coding agent (Anthropic's Claude, via Claude Code), working under human direction, review, and on-target testing. Every feature in this release was verified on-target under the Copperline emulator against real AmigaOS 3.2.3 before being considered done. The entire source is BSD-licensed and open for review. Documentation ------------- The full command reference (Shell, ARexx, and wire protocol), locator-tier/limits documentation, and troubleshooting notes are online at https://sidick.github.io/amipilot/ and included as amipilot.guide in this archive (AmigaGuide/MultiView, OS 2.x+) for offline/on-Amiga reading - same source, both forms. See the Changelog there for what's new in each release. Source and licence ------------------- Free and open source, BSD 2-Clause. Full source, documentation and issue tracker: https://github.com/sidick/amipilot