@=j( UdZddlmZddlmZmZdddiddgd d d d dgd dd dddddddddddddddddddidddd d!dgd"d#d d$ddgd%d&d'd(d)dd*dd+dd,dd-dddiddd.d d/dddiddd0d d1dgd2d3d d4dd5dd6dd7dd8dd9dd:dd;ddd?dd>d@ddAdgdBdCZdDedE<dIdGZdHS)JuNSchema for the generic `computer_use` tool. Model-agnostic. Any tool-calling model can drive this. Vision-capable models should prefer `capture(mode='som')` then `click(element=N)` — much more reliable than pixel coordinates. Pixel coordinates remain supported for models that were trained on them (e.g. Claude's computer-use RL). ) annotations)AnyDict computer_useuDrive the desktop in the background via cua-driver — screenshots, mouse, keyboard, scroll, drag — without stealing the user's cursor or keyboard focus. Supported on macOS, Windows, and Linux. Preferred workflow: call with action='capture' (mode='som' gives numbered element overlays), then click by `element` index for reliability. Pixel coordinates are supported for models trained on them. Works on any window — hidden, minimized, or behind another app. Requires cua-driver to be installed.objectactionstring) captureclick double_click right_click middle_clickdragscrolltypekey set_valuewait list_apps focus_appu Which action to perform. `capture` is free (no side effects). All other actions require approval unless auto-approved. Use `set_value` for select/popup elements and sliders — it selects the matching option directly without opening the native menu (no focus steal).)renum descriptionmode)somvisionaxuCapture mode. `som` (default) is a screenshot with numbered overlays on every interactable element plus the AX tree — best for vision models, lets you click by element index. `vision` is a plain screenshot. `ax` is the accessibility tree only (no image; useful for text-only models).appuOptional. Limit capture/action to a specific app (by name, e.g. 'Safari', or bundle ID, 'com.apple.Safari'). If omitted, operates on the frontmost app's window. Pass app='screen' (or 'desktop') to capture the OS desktop/shell surface — e.g. to see the wallpaper or click the taskbar. Note: capture is per-window; a single image cannot span multiple monitors, so on a multi-screen setup capture one window or display at a time.)rr max_elementsintegeruOptional cap on the AX `elements` array returned by `action='capture'`. Default 100, hard maximum 1000. Dense UIs (Electron apps such as Obsidian or VS Code, JetBrains IDEs) can publish 500+ AX nodes — capping prevents a single capture from blowing session context. When the cap trims the response, `total_elements` and `truncated_elements` are surfaced in the result so you can re-call with `app=` to narrow scope or raise `max_elements` when the full tree is required. Has no effect on `mode='som'` / `mode='vision'` when a screenshot is included in the response; only the rare image-missing fallback returns an `elements` array and is subject to the cap.di)rrdefaultminimummaximumelementzoThe 1-based SOM index returned by the last `capture(mode='som')` call. Strongly preferred over raw coordinates. coordinatearrayrzPixel coordinates [x, y] in logical screen space (as returned by capture width/height). Only use this if no element index is available.)ritemsminItemsmaxItemsrbutton)leftrightmiddlezMouse button. Defaults to left. modifiers) cmdshiftoptionaltctrlfnwinwindowssupermeta)rrz%Modifier keys held during the action.)rr)r from_elementzSource element index (drag). to_elementzTarget element index (drag).from_coordinatez3Source [x,y] (drag; use when no element available). to_coordinatez3Target [x,y] (drag; use when no element available). direction)updownr-r.zScroll direction.amountzScroll wheel ticks. Default 3.valuezFor action='set_value': the value to set on the element. For AXPopUpButton / select dropdowns, pass the option's display label (e.g. 'Blue'). For sliders and other AXValue-settable elements, pass the numeric or string value.textz+Text to type (respects the current layout).keyszUKey combo, e.g. 'cmd+s', 'ctrl+alt+t', 'return', 'escape', 'tab'. Use '+' to combine.numberzSeconds to wait. Max 30.booleanuOnly for action='focus_app'. If true, brings the window to front (DISRUPTS the user). Default false — input is routed to the app without raising, matching the background co-work model.zIf true, take a follow-up capture after the action and include it in the response. Saves a round-trip when you need to verify an action's effect.)seconds raise_window capture_after)r propertiesrequired)namer parametersDict[str, Any]COMPUTER_USE_SCHEMAreturnctS)z2Return the generic OpenAI function-calling schema.)rP?/home/rurouni/.hermes/hermes-agent/tools/computer_use/schema.pyget_computer_use_schemarVs rTN)rQrO) __doc__ __future__rtypingrrrP__annotations__rVrSrTrUr[sv#"""""  w   H%w 6  ///-  7w N  7  Ow j ! *)kw X !'Yw h  ),5  iw ~  333@w H $ G  Iw ` Y-KMMaw d 9+IKKew h  ),1T   iw t  ),1T uw B  7772Cw L !?Mw V  SWw j  Lkw r  ;sw B!9 "="B_w w w pJu{{I'I'IIIIXrT