Skip to content

mouse

This directory provides a comprehensive, voice-controlled mouse management system designed for Talon. It coordinates actions for clicking, dragging, standard scrolling, continuous scrolling, eye-tracker-based "gaze scrolling," cursor visibility control, and sound-based navigation (using pops or hisses).

Clicking, Dragging, and Eye Tracking

Core mouse manipulation and click behaviors are defined in mouse.py and mapped to voice commands in mouse.talon.

  • Clicking & Dragging: Talon commands facilitate left, right, and middle clicks (including modifier keys like control or shift), as well as double/triple clicking. Drag operations can be initiated ("drag start") and automatically finished ("drag end") across commands.
  • Pop Noise Integration: The pop sound trigger is highly configurable. If configured, a "pop" sound can be used to execute a standard left click, terminate an ongoing mouse drag, or stop a continuous scroll event.
  • State Management (Wake/Sleep): The "control off" command calls mouse_sleep(), which gracefully disables eye/zoom tracking, saves current tracking preferences so they can be restored when the system is awakened ("control mouse"), stops any continuous scroll jobs, and terminates active mouse drags.

Continuous, Gaze, and Hiss Scrolling

The scrolling engine manages diverse and highly ergonomic forms of scrolling, implemented across mouse_scroll.py, continuous_scrolling.talon, and continuous_scrolling_direction.talon-list.

  • Direction Mapping: The list defined in continuous_scrolling_direction.talon-list maps natural language directions like "upper", "downer", "lefter", and "righter" to their respective scrolling cardinal directions.
  • Continuous Scrolling: Saying "wheel upper" or "wheel downer" launches a scrolling loop managed by the Scroller and ScrollingState classes in mouse_scroll.py. This loop uses a background interval (cron.interval) to scroll smoothly. It supports configurable linear acceleration and custom speed multipliers.
  • On-the-Fly Speed Adjustment: When continuous scrolling is active, Talon registers the user.continuous_scrolling tag. This activates commands in continuous_scrolling.talon, allowing the user to say a small number (e.g., "12") to dynamically adjust the scrolling speed or say "stop" to cancel the scroll.
  • Gaze Scrolling: Utilizing eye tracker coordinates, gaze scrolling adjusts scroll speed and direction based on how far the user's gaze is from the vertical center of the active window. This produces an intuitive "scroll where I look" behavior.
  • Hiss-activated Scrolling: By leveraging the continuous "hiss" noise, users can scroll dynamically. Setting user.mouse_enable_hiss_scroll to true allows a prolonged "hiss" sound to continuously scroll in a pre-configured direction until the user stops hissing.
  • Visual Overlay: When continuous or gaze scrolling is active, a subtle IMGUI-based status window can display the current scroll speed/mode and provide a clickable "Stop" button.

Cursor Visibility Control

To prevent eye-tracker distraction, users can toggle the visibility of the operating system's mouse cursor. This behavior is managed by mouse_cursor.py and exposed to voice commands in mouse_cursor.talon.

  • Cross-Platform Cursor Management: On macOS and Linux, the module calls Talon’s built-in ctrl.cursor_visible() API. On Windows, where standard API support can vary, it interfaces directly with the Windows Registry (Control Panel\Cursors) and invokes SystemParametersInfoA via ctypes to dynamically exchange the system cursors with a transparent, hidden .cur resource.

For a detailed overview of command structures, voice phrasing, and math variables used in calculating continuous scroll speeds, see README.md.