Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,27 @@ python -m pytest test/integrated_test/
python -m build
```

## Feature Delivery Rules

### Every feature must ship both a headless API and a GUI surface

No feature is complete unless it can be driven entirely without the GUI **and** has a corresponding GUI affordance. Concretely:

- **Headless core in `utils/` or `wrapper/`**: all business logic lives in a module with zero `PySide6` imports. Users must be able to `import je_auto_control` and call the feature without ever instantiating a Qt class.
- **Re-export from the package facade**: add the public functions / classes to `je_auto_control/__init__.py` and its `__all__` so `import je_auto_control as ac; ac.<feature>(...)` works out of the box.
- **Executor command coverage**: wire an `AC_*` command into `utils/executor/action_executor.py` so the feature is usable from JSON action files, the socket server, the scheduler, and the visual script builder — all without Python glue.
- **GUI tab or control is a thin wrapper**: the Qt widget must only translate user input into calls on the headless core. It must not contain business logic that would be unreachable headlessly.
- **The top-level package stays Qt-free**: `import je_auto_control` MUST NOT import `PySide6`. The GUI entry point is loaded lazily inside `start_autocontrol_gui()`. Verify with:

```python
import sys, je_auto_control # noqa
assert not any("PySide6" in m for m in sys.modules)
```

- **Tests cover the headless path**: at least one unit test in `test/unit_test/` must exercise the feature through its non-GUI API with no Qt imports.

Features that are inherently interactive (e.g. region picking with the mouse, template cropping) still count as GUI-only — but they must accept programmatic equivalents (e.g. `screenshot(screen_region=[...])` with explicit coordinates) so scripts can replay the same effect headlessly.

## Coding Standards

### Security First
Expand Down
75 changes: 59 additions & 16 deletions docs/source/Eng/doc/cli/cli_doc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,47 +2,90 @@
Command-Line Interface
=======================

AutoControl can be used directly from the command line to execute automation scripts.
Two CLI entry points are provided:

Execute a Single Action File
=============================
- ``python -m je_auto_control`` — legacy flag-style runner for one-off
execute / create-project operations. Also launches the GUI when called
with no arguments.
- ``python -m je_auto_control.cli`` — subcommand-based runner for running
scripts, listing scheduler jobs, and starting the socket / REST servers.

Subcommand CLI (``python -m je_auto_control.cli``)
==================================================

Run a script
------------

.. code-block:: bash

python -m je_auto_control --execute_file "path/to/actions.json"
python -m je_auto_control.cli run script.json
python -m je_auto_control.cli run script.json --var count=10 --var name=alice
python -m je_auto_control.cli run script.json --dry-run

``--var name=value`` is parsed as JSON when the value parses, otherwise
it is treated as a plain string. ``--dry-run`` records every action
through the executor without invoking any side effects.

List scheduler jobs
-------------------

.. code-block:: bash

python -m je_auto_control.cli list-jobs

Start the TCP socket server
---------------------------

.. code-block:: bash

python -m je_auto_control.cli start-server --host 127.0.0.1 --port 9938

# Short form
Start the REST API server
-------------------------

.. code-block:: bash

python -m je_auto_control.cli start-rest --host 127.0.0.1 --port 9939

Endpoints: ``GET /health``, ``GET /jobs``, ``POST /execute`` with
``{"actions": [...]}``.

Legacy flag-style CLI (``python -m je_auto_control``)
=====================================================

Execute a single action file
----------------------------

.. code-block:: bash

python -m je_auto_control --execute_file "path/to/actions.json"
python -m je_auto_control -e "path/to/actions.json"

Execute All Files in a Directory
================================
Execute all files in a directory
--------------------------------

.. code-block:: bash

python -m je_auto_control --execute_dir "path/to/action_files/"

# Short form
python -m je_auto_control -d "path/to/action_files/"

Execute a JSON String Directly
==============================
Execute a JSON string directly
------------------------------

.. code-block:: bash

python -m je_auto_control --execute_str '[["AC_screenshot", {"file_path": "test.png"}]]'

Create a Project Template
=========================
Create a project template
-------------------------

.. code-block:: bash

python -m je_auto_control --create_project "path/to/my_project"

# Short form
python -m je_auto_control -c "path/to/my_project"

Launch the GUI
==============
--------------

.. code-block:: bash

Expand Down
211 changes: 211 additions & 0 deletions docs/source/Eng/doc/new_features/new_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
=====================
New Features (2026-04)
=====================

This page documents the April 2026 additions to AutoControl. Every new
feature ships with a headless Python API **and** a GUI affordance, and is
wired into the executor so it works from JSON scripts, the socket server,
the REST API, and the CLI without any Python glue.

.. contents::
:local:
:depth: 2


Clipboard
=========

Headless::

import je_auto_control as ac
ac.set_clipboard("hello")
text = ac.get_clipboard()

Action-JSON commands::

[["AC_clipboard_set", {"text": "hello"}]]
[["AC_clipboard_get", {}]]

Backends: Windows (Win32 via ``ctypes``), macOS (``pbcopy``/``pbpaste``),
Linux (``xclip`` or ``xsel``). A ``RuntimeError`` is raised if no backend
is available.


Dry-run / step-debug executor
=============================

Run an action list through the executor without invoking any side effects
— useful for validating JSON scripts::

from je_auto_control.utils.executor.action_executor import executor
record = executor.execute_action(actions, dry_run=True)

``step_callback`` lets you observe each action before it runs::

executor.execute_action(actions, step_callback=lambda a: print(a))

From the CLI::

python -m je_auto_control.cli run script.json --dry-run


Global hotkey daemon (Windows)
==============================

Bind OS-level hotkeys to action-JSON scripts::

from je_auto_control import default_hotkey_daemon
default_hotkey_daemon.bind("ctrl+alt+1", "scripts/greet.json")
default_hotkey_daemon.start()

Supported modifiers: ``ctrl``, ``alt``, ``shift``, ``win`` / ``super`` /
``meta``. Keys: letters, digits, ``f1`` … ``f12``, arrows, ``space``,
``enter``, ``tab``, ``escape``, ``home``, ``end``, ``insert``, ``delete``,
``pageup``, ``pagedown``.

macOS and Linux currently raise ``NotImplementedError`` on
``start()`` — the Strategy-pattern interface is in place so backends can
be added later.

GUI: **Hotkeys** tab (bind/unbind, start/stop daemon, live fired count).


Event triggers
==============

Poll-based triggers fire an action script when a screen/state change is
detected::

from je_auto_control import default_trigger_engine, ImageAppearsTrigger
default_trigger_engine.add(ImageAppearsTrigger(
trigger_id="", script_path="scripts/click_ok.json",
image_path="templates/ok_button.png", threshold=0.85,
repeat=True,
))
default_trigger_engine.start()

Available trigger types:

- ``ImageAppearsTrigger`` — template match on the current screen
- ``WindowAppearsTrigger`` — title substring match
- ``PixelColorTrigger`` — pixel color within tolerance
- ``FilePathTrigger`` — mtime change on a path

GUI: **Triggers** tab (add/remove/start/stop, live fired count).


Cron scheduling
===============

Five-field cron (``minute hour day-of-month month day-of-week``) with
``*``, comma-lists, ``*/step``, and ``start-stop`` ranges::

from je_auto_control import default_scheduler
job = default_scheduler.add_cron_job(
script_path="scripts/daily.json",
cron_expression="0 9 * * 1-5", # 09:00 on weekdays
)
default_scheduler.start()

Interval and cron jobs coexist in the same scheduler; ``job.is_cron``
tells them apart. GUI: **Scheduler** tab has cron/interval radio.


Plugin loader
=============

A plugin file is any ``.py`` defining top-level callables whose names
start with ``AC_``. Each one becomes a new executor command::

# my_plugins/greeting.py
def AC_greet(args=None):
return f"hello, {args['name']}"

::

from je_auto_control import (
load_plugin_directory, register_plugin_commands,
)
commands = load_plugin_directory("my_plugins/")
register_plugin_commands(commands)

# Now usable from JSON:
# [["AC_greet", {"name": "world"}]]

GUI: **Plugins** tab (browse directory, one-click register).

.. warning::
Plugin files execute arbitrary Python. Only load from directories
under your own control.


REST API server
===============

A stdlib-only HTTP server that exposes the executor and scheduler::

from je_auto_control import start_rest_api_server
server = start_rest_api_server(host="127.0.0.1", port=9939)

Endpoints:

- ``GET /health`` — liveness probe
- ``GET /jobs`` — scheduler job list
- ``POST /execute`` with body ``{"actions": [...]}`` — run actions

GUI: **Socket Server** tab now has a separate REST section with its own
host/port and a ``0.0.0.0`` opt-in.

.. note::
Defaults to ``127.0.0.1`` per CLAUDE.md policy. Bind to ``0.0.0.0``
only when you have authenticated the network boundary.


CLI runner
==========

A thin subcommand-based CLI over the headless APIs::

python -m je_auto_control.cli run script.json
python -m je_auto_control.cli run script.json --var name=alice --dry-run
python -m je_auto_control.cli list-jobs
python -m je_auto_control.cli start-server --port 9938
python -m je_auto_control.cli start-rest --port 9939

``--var name=value`` is parsed as JSON when possible (so ``count=10``
becomes an int), otherwise treated as a string.


Multi-language GUI (i18n)
=========================

Live language switching via the **Language** menu. Built-in packs:

- English
- Traditional Chinese (繁體中文)
- Simplified Chinese (简体中文)
- Japanese (日本語)

Register additional languages at runtime::

from je_auto_control.gui.language_wrapper.multi_language_wrapper import (
language_wrapper,
)
language_wrapper.register_language("French", {"menu_file": "Fichier", ...})

Missing keys fall through to the English default, so a feature ships
with usable labels even before its translations land.


Closable tabs + menu bar
========================

The main window is now a ``QMainWindow`` with:

- **File** → Open Script, Exit
- **View → Tabs** → checkable entries for every tab (restore closed tabs)
- **Tools** → Start hotkey daemon / scheduler / trigger engine
- **Language** → select a registered language pack
- **Help** → About

Close any tab with its ✕ button; re-open it via *View → Tabs*.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ Comprehensive guides for all AutoControl features.
doc/critical_exit/critical_exit_doc
doc/cli/cli_doc
doc/create_project/create_project_doc
doc/new_features/new_features_doc
Loading
Loading