From 7facf5558e14a3e68ab10749e6929403e8cdfcd8 Mon Sep 17 00:00:00 2001 From: jamesoncollins <35897639+jamesoncollins@users.noreply.github.com> Date: Sun, 31 May 2026 22:05:46 -0400 Subject: [PATCH] Document README preservation and sanitize templates --- .gitignore | 1 + README.md | 102 ++++++++++++----------------- docs/ARCHITECTURE.md | 90 ++++++++++++++++++++++++++ docs/CONFIGURATION.md | 70 ++++++++++++++++++++ docs/GPT_TOOLS.md | 71 ++++++++++++++++++++ docs/HANDLERS.md | 135 +++++++++++++++++++++++++++++++++++++++ docs/OPERATIONS.md | 92 ++++++++++++++++++++++++++ docs/README_MIGRATION.md | 25 ++++++++ docs/TESTING.md | 74 +++++++++++++++++++++ secret.example.txt | 15 +++++ 10 files changed, 614 insertions(+), 61 deletions(-) create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/CONFIGURATION.md create mode 100644 docs/GPT_TOOLS.md create mode 100644 docs/HANDLERS.md create mode 100644 docs/OPERATIONS.md create mode 100644 docs/README_MIGRATION.md create mode 100644 docs/TESTING.md create mode 100644 secret.example.txt diff --git a/.gitignore b/.gitignore index 312f3eb..1c8279f 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ __pycache__ *.webm *.txt *secret* +!secret.example.txt *.json* *.jpg /downloaded_video.mp4.in diff --git a/README.md b/README.md index bb54844..5002210 100644 --- a/README.md +++ b/README.md @@ -1,83 +1,63 @@ # turbo-bot -## How this works +TurboBot is a Python Signal bot that runs against the signal-cli REST API. It routes messages through dynamically discovered handlers in `handlers/`, supports hashtag commands such as `#gpt`, and can return text plus base64-encoded attachments. -A docker compose file is used to run a signalbot, and the signal rest api. +## Documentation -The signalbot docker will automatically get this repo and execute run.py. +- [Architecture](docs/ARCHITECTURE.md): runtime flow, dispatch, handler discovery, and important paths. +- [Configuration](docs/CONFIGURATION.md): required environment variables and `secret.txt` setup. +- [Operations](docs/OPERATIONS.md): Docker Compose services, bootstrap flow, auto-updates, and Signal linking. +- [Writing handlers](docs/HANDLERS.md): how to add new bot features. +- [GPT function tools](docs/GPT_TOOLS.md): how to add tools callable by the `#gpt` handler. +- [Testing](docs/TESTING.md): local test commands, mocks, and integration-test caveats. +- [README migration checklist](docs/README_MIGRATION.md): where the original README content moved and why no setup guidance was dropped. -A second signalbot docker is made that checks out the devel branch instead of main. +## Quick start -Both containers monitor the github repo and will automatic download updated code. +1. Start the Docker Compose stack. +2. Link `signal-cli` to a Signal account/device if this is the first run. +3. Copy the sample secret file and edit it with real values: + ```bash + cp secret.example.txt secret.txt + ``` -## Running +4. Configure at least: -Execute the docker compose file. In the signalbot docker make sure you make a secrets.txt file that has these vairables: - -``` -export SIGNAL_API_URL=signal-cli:8181 # URL for the signal-cli API -export BOT_NUMBER="+1555555555" # The registered Signal number for your bot -export CONTACT_NUMERS="+1555555555" # true/false, a single contact, a ; seperated list of contacts -export GROUP_NAMES="MYGROUP" # true/false, a single group, a ; seperated list of groups -export IGNORE_GROUPS="TurboBot Devel" #optional -export INSTA_USERNAME="myuser" -export INSTA_PASSWORD="mypassword" -export OPENAI_API_KEY="keygoeshere" -``` - -Update the docker-compose file to point the signal-cli bot(s) to your repo, -or use this one. Default file makes one for main branch and one for devel -branch. The run.sh script will just fail back to bash if you dont supply -a secrets.sh file in the same folder as the repo. - -When you first run this you need to boot signal-cli in normal mode in order -to link it to your account. You do that by having it generate a qr code -that you scan with your phone. See the signalbot documentation. tldr: -http://localhost:8181/v1/qrcodelink?device_name=local + ```bash + export SIGNAL_API_URL="signal-cli:8181" + export BOT_NUMBER="+15555555555" + export CONTACT_NUMBERS="+15555555555" + export GROUP_NAMES="My Signal Group" + ``` +See [Configuration](docs/CONFIGURATION.md) for all supported variables. -Manually running signal-cli from command line. Be sure to stop the instance -first: +## Signal linking -``` -docker run --env "MODE=json-rpc" --env "PORT=8181" --env "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" --env "GIN_MODE=release" --env "BUILD_VERSION=0.90" --env "SIGNAL_CLI_CONFIG_DIR=/home/.local/share/signal-cli" --env "SIGNAL_CLI_UID=1000" --env "SIGNAL_CLI_GID=1000" --entrypoint "/entrypoint.sh" --volume "/share/CACHEDEV1_DATA/Container/container-station-data/lib/docker/volumes/app-1_signal-cli-data/_data:/home/.local/share/signal-cli" bbernhard/signal-cli-rest-api:latest -``` +When you first run this, start signal-cli and link it to your account by scanning a QR code with your phone. With the REST API exposed locally, the QR-code endpoint is typically: -This was generated with: - -``` -container_id="signal-cli" -docker inspect $container_id | jq -r ' - .[] | - "docker run " + - (if .Config.Env then (.Config.Env | map("--env \"" + . + "\"") | join(" ")) else "" end) + " " + - (if .Config.Entrypoint then "--entrypoint \"" + (.Config.Entrypoint | join(" ")) + "\" " else "" end) + - (if .Mounts then (.Mounts | map("--volume \"" + .Source + ":" + .Destination + "\"") | join(" ")) else "" end) + " " + - (if .Config.Cmd then (.Config.Cmd | join(" ")) else "" end) + - " " + .Config.Image -' +```text +http://localhost:8181/v1/qrcodelink?device_name=local ``` +See [Operations](docs/OPERATIONS.md) for more deployment details. -# Development - -You can develop in windows, linux, wsl, and mac all relatively easily. - -For Windows I'd suggest using miniconda. - -Running the tests on your own windows or linux box is pretty easy. +## Development -If on windows just install miniconda, make a new environment, `conda install python==3` and then run `pip3 install -r requirements.txt`. +Create a Python environment, install dependencies, and run tests: -TODO: we should really match the version of python on the real system... which i just realized we dont control - -Then you can run all the tests with `python -m unittest discover -s tests -p "test_*.py"` - -You can also do it in WSL 1 or 2. +```bash +pip install -r requirements.txt +python -m unittest discover -s tests -p "test_*.py" +``` -Some handlers might also require apt-get packages. i.e. ffmpeg. You can get -ffmpeg in miniconda (i.e. conda install ffmpeg) or wsl (via apt-get or whatever). +Some handlers require system packages, especially `ffmpeg`. Docker deployments install packages from `pkglist`. +## Manual signal-cli reference +If you need to reproduce a running `signal-cli` container manually, stop the existing instance first and adapt this example to your local volume paths: +```bash +docker run --env "MODE=json-rpc" --env "PORT=8181" --env "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" --env "GIN_MODE=release" --env "BUILD_VERSION=0.90" --env "SIGNAL_CLI_CONFIG_DIR=/home/.local/share/signal-cli" --env "SIGNAL_CLI_UID=1000" --env "SIGNAL_CLI_GID=1000" --entrypoint "/entrypoint.sh" --volume "/share/CACHEDEV1_DATA/Container/container-station-data/lib/docker/volumes/app-1_signal-cli-data/_data:/home/.local/share/signal-cli" bbernhard/signal-cli-rest-api:latest +``` diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..376bacc --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,90 @@ +# Architecture + +TurboBot is a Python Signal bot that routes incoming messages through a small handler framework. The runtime entrypoint is `run.py`, and feature code lives mostly in `handlers/`, `utils/`, and `tool_functions/`. + +## Runtime overview + +`run.py` configures and starts a `SignalBot` instance using environment variables such as `SIGNAL_API_URL` and `BOT_NUMBER`. The bot registers `TurboBotCommand`, whose `handle` method is the main message-processing path. + +Every normal bot reply is prefixed with `LOGMSG`, currently: + +```text +----TURBOBOT---- +``` + +`TurboBotCommand.handle` reads the raw Signal message, extracts useful metadata, determines whether the message is private or group-based, applies allow/ignore configuration, and then dispatches matching messages to handlers. + +## Authorization and routing + +Message routing is controlled by environment variables parsed through `utils.misc_utils.parse_env_var`: + +- `CONTACT_NUMBERS`: allowed private contacts. Use `true` to allow all, `false`/unset to allow none depending on call site, a single value, or a semicolon-separated list. +- `GROUP_NAMES`: allowed group names. Use `true` to allow all, or a semicolon-separated list of group names. +- `IGNORE_GROUPS`: optional group names that should be ignored even if otherwise allowed. + +Group messages are resolved from Signal internal group IDs to group metadata using `find_group_by_internal_id` in `run.py`. + +## Built-in command handling + +Before dynamic handlers are consulted, `TurboBotCommand.handle` checks a few direct commands and special cases, including: + +- `#ping`: health check response. +- Reddit URLs: direct video download/reply path. +- `#status`: machine and git status. +- `#reboot`: exits the bot process so the surrounding launcher can restart it. +- `#help`: lists help text from all dynamically discovered handlers. + +## Handler discovery + +Handlers are discovered by `BaseHandler.get_all_handlers()` in `handlers/base_handler.py`. + +Discovery rules: + +1. Iterate over Python modules in the `handlers/` directory. +2. Import each module as `handlers.`. +3. Inspect classes in the module. +4. Include classes that subclass `BaseHandler`. +5. Exclude `BaseHandler` itself. +6. Exclude classes with `is_intermediate = True`. + +Because discovery is dynamic, most new handlers do not need to be registered manually. Put the handler class in `handlers/`, subclass `BaseHandler` or a subclass such as `HashtagHandler`, and make sure it is not marked intermediate. + +## Handler execution contract + +The base contract is defined by `handlers/base_handler.py`: + +- `can_handle(self) -> bool`: return true if the handler should process the current input string. +- `process_message(self, msg, attachments) -> dict`: return a dictionary with: + - `message`: text to send back to Signal. + - `attachments`: a list of base64-encoded attachments. +- `get_name() -> str`: a human-readable handler name used by help output. +- `get_help_text() -> str`: help text used by `#help`. + +`BaseHandler.process_message` assumes subclass implementations provide `get_message()` and `get_attachments()`. Handlers can override `process_message` when they need custom behavior. + +## Hashtag command abstraction + +`handlers/hashtag_handler.py` provides `HashtagHandler`, a convenience subclass for commands such as `#gpt`, `#mmw`, `#golf`, and `#asteroid`. + +A hashtag handler supplies: + +- `get_hashtag()`: the command pattern, such as `r"#gpt"`. +- `get_substring_mapping()`: positional dot-argument names and defaults. + +For example, a message like `#gpt.gpt-4.1 explain this` can be split into the command, model substring, and cleaned prompt text. + +## GPT tool subsystem + +`handlers/gpt_handler.py` implements the `#gpt` command and dynamically loads optional function tools from `tool_functions/`. + +Tool modules are ordinary Python files that expose `TOOL_SPEC` and `TOOL_FN`. See `docs/GPT_TOOLS.md` for the tool-authoring contract. + +## Important paths + +- `run.py`: application entrypoint and message dispatch. +- `handlers/base_handler.py`: common handler base class and dynamic discovery. +- `handlers/hashtag_handler.py`: helper for hashtag/dot-argument command handlers. +- `handlers/gpt_handler.py`: OpenAI-backed GPT and image-generation handler. +- `tool_functions/`: dynamically loaded GPT function tools. +- `utils/`: shared helpers for env parsing, media conversion, Reddit, video scraping, machine info, and git info. +- `tests/`: unittest-based test suite with Signal API mocks. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md new file mode 100644 index 0000000..59eb03a --- /dev/null +++ b/docs/CONFIGURATION.md @@ -0,0 +1,70 @@ +# Configuration + +TurboBot is configured mostly through environment variables loaded by `run.sh` before `run.py` starts. + +## Secret file + +`run.sh` currently looks for a file named `secret.txt` in the repository root and sources it as shell code. + +Create a local `secret.txt` from the checked-in template: + +```bash +cp secret.example.txt secret.txt +``` + +Then edit `secret.txt` with real values. Do not commit real secrets. + +Older project notes referred to `secrets.txt` or `secrets.sh`; the current launcher reads `secret.txt`. + +## Environment variables + +### Required for normal operation + +```bash +export SIGNAL_API_URL="signal-cli:8181" +export BOT_NUMBER="+15555555555" +``` + +- `SIGNAL_API_URL`: URL or host:port for the signal-cli REST API. +- `BOT_NUMBER`: registered Signal phone number for the bot. + +### Message allow/ignore controls + +```bash +export CONTACT_NUMBERS="+15555555555;+15555555556" +export GROUP_NAMES="My Signal Group;Another Signal Group" +export IGNORE_GROUPS="TurboBot Devel" +``` + +- `CONTACT_NUMBERS`: private contacts allowed to use the bot. +- `GROUP_NAMES`: Signal groups allowed to use the bot. +- `IGNORE_GROUPS`: optional group names to ignore even if otherwise allowed. + +`CONTACT_NUMBERS`, `GROUP_NAMES`, and `IGNORE_GROUPS` are parsed by `utils.misc_utils.parse_env_var`: + +- unset or empty values become `None`. +- exact lowercase `true` and `false` become booleans. +- semicolon-separated values become lists. +- a single non-empty value becomes a one-item list. + +### Optional feature credentials + +```bash +export OPENAI_API_KEY="replace-with-your-openai-api-key" +export INSTA_USERNAME="myuser" +export INSTA_PASSWORD="replace-with-your-instagram-password" +``` + +- `OPENAI_API_KEY`: enables the `#gpt` handler and OpenAI model/tool calls. +- `INSTA_USERNAME` and `INSTA_PASSWORD`: used by Instagram-related functionality when enabled. + +## Docker Compose configuration + +`docker-compose.yml` also sets deployment variables for the bot containers: + +- `GIT_REPO_URL`: repository to clone/fetch. +- `GIT_REPO_PATH`: path inside the container where the repository is stored. +- `GIT_REPO_BRANCH`: branch to reset to and run. +- `SETUP_SCRIPT_NAME`: bootstrap script name, normally `setup.sh`. + +See `docs/OPERATIONS.md` for the full container startup flow. diff --git a/docs/GPT_TOOLS.md b/docs/GPT_TOOLS.md new file mode 100644 index 0000000..21fbad1 --- /dev/null +++ b/docs/GPT_TOOLS.md @@ -0,0 +1,71 @@ +# GPT function tools + +The `#gpt` handler can expose local Python functions as model-callable tools. Tool modules live in `tool_functions/` and are loaded dynamically by `handlers/gpt_handler.py`. + +## Loader rules + +`load_function_tools()` applies these rules: + +1. Look in `tool_functions/`. +2. Load files ending in `.py`. +3. Skip files whose names start with `_`. +4. Import each module. +5. Read `TOOL_SPEC` and `TOOL_FN` from the module. +6. Use `TOOL_SPEC["name"]` as the tool name. +7. Store `TOOL_FN` as the function to execute when the model calls that tool. + +Modules missing `TOOL_SPEC`, `TOOL_FN`, or a tool name are skipped with a warning. + +## Tool function contract + +A tool function should: + +- Accept keyword arguments described by `TOOL_SPEC["parameters"]`. +- Validate inputs and raise clear exceptions for invalid data. +- Return a dictionary when possible. +- Include text output under a key such as `text`. +- Include base64 attachments under `attachments` when generating files or images. + +Existing examples: + +- `tool_functions/coin_flip.py`: simple text result. +- `tool_functions/plot_from_data.py`: generates a plot image attachment. + +## Minimal example + +```python +from typing import Any, Dict + + +def add_numbers(a: float, b: float) -> Dict[str, Any]: + return {"text": f"{a} + {b} = {a + b}", "attachments": []} + + +TOOL_SPEC: Dict[str, Any] = { + "type": "function", + "name": "add_numbers", + "description": "Add two numbers and return the sum.", + "parameters": { + "type": "object", + "additionalProperties": False, + "properties": { + "a": {"type": "number"}, + "b": {"type": "number"}, + }, + "required": ["a", "b"], + }, +} + +TOOL_FN = add_numbers +``` + +## Attachment behavior + +When a tool returns base64-encoded attachments, `handlers/gpt_handler.py` collects them and includes them in the Signal response. Keep attachments small enough for Signal to send reliably. + +## Safety notes + +- Do not expose tools that can run arbitrary shell commands from model-provided arguments. +- Validate URLs, filenames, and numeric ranges. +- Avoid writing secrets or sensitive local files into tool responses. +- Prefer deterministic tools with narrow parameter schemas. diff --git a/docs/HANDLERS.md b/docs/HANDLERS.md new file mode 100644 index 0000000..69ff30e --- /dev/null +++ b/docs/HANDLERS.md @@ -0,0 +1,135 @@ +# Writing handlers + +Handlers are the main extension point for TurboBot. A handler decides whether it can process an incoming Signal message and, if so, returns text and optional base64 attachments for the bot to send. + +## Choosing a base class + +Use `BaseHandler` directly when matching arbitrary message text, URLs, or attachments. + +Use `HashtagHandler` when implementing command-style features such as: + +- `#gpt` +- `#mmw` +- `#golf` +- `#asteroid` +- `#numberwang` + +`HashtagHandler` handles command detection, dot-separated substring parsing, and cleaned input extraction. + +## BaseHandler contract + +A direct `BaseHandler` subclass should usually implement: + +```python +from handlers.base_handler import BaseHandler + + +class MyHandler(BaseHandler): + def can_handle(self) -> bool: + return "hello bot" in self.input_str.lower() + + def process_message(self, msg, attachments): + return { + "message": "Hello from TurboBot!", + "attachments": [], + } + + @staticmethod + def get_name() -> str: + return "My Handler" + + @staticmethod + def get_help_text() -> str: + return "Responds when a message contains 'hello bot'." +``` + +The returned dictionary should contain: + +- `message`: response text. +- `attachments`: a list of base64-encoded files. Use an empty list when there are no attachments. + +## HashtagHandler contract + +A `HashtagHandler` subclass typically implements: + +```python +from handlers.hashtag_handler import HashtagHandler + + +class EchoHandler(HashtagHandler): + is_intermediate = False + + def get_hashtag(self) -> str: + return r"#echo" + + def get_substring_mapping(self) -> dict: + return {0: ("mode", "normal")} + + def get_message(self) -> str: + if self.hashtag_data.get("mode") == "help": + return self.get_help_text() + return self.cleaned_input + + def get_attachments(self) -> list: + return [] + + @staticmethod + def get_name() -> str: + return "Echo Handler" + + @staticmethod + def get_help_text() -> str: + return "Usage: #echo[.mode] text to echo back." +``` + +### Dot substrings + +`get_substring_mapping()` maps dot-argument positions to a data key and default. For example: + +```python +return {0: ("model", "gpt-4.1")} +``` + +For a message like: + +```text +#gpt.gpt-4.1 summarize this +``` + +The handler can read: + +- `self.hashtag_data["model"] == "gpt-4.1"` +- `self.cleaned_input == "summarize this"` + +## Help output + +When a user sends `#help`, `run.py` loads all handlers with `BaseHandler.get_all_handlers()` and calls each handler's: + +- `get_name()` +- `get_help_text()` + +Keep help text short enough to fit comfortably in one Signal reply. + +## Registration + +Most handlers do not need a registry entry. To make a handler discoverable: + +1. Add a `.py` file under `handlers/`. +2. Define a class that subclasses `BaseHandler` or `HashtagHandler`. +3. Set `is_intermediate = False`, or omit the attribute. +4. Implement `can_handle()` or `get_hashtag()` as appropriate. +5. Implement help text. + +Classes with `is_intermediate = True` are skipped by dynamic discovery. This is useful for abstract helper classes. + +## Testing a handler + +Use the existing unittest structure: + +- `tests/TurboTestCase.py`: shared bot test harness. +- `tests/test_run.py`: examples of top-level command and handler tests. +- `tests/test_mmw.py`: hashtag command test examples. +- `tests/test_asteroid.py`: external API handler example. +- `tests/test_ytdlp.py`: media-download handler examples. + +Prefer mocking network calls in new tests unless the test is intentionally integration-style. diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md new file mode 100644 index 0000000..c41c102 --- /dev/null +++ b/docs/OPERATIONS.md @@ -0,0 +1,92 @@ +# Operations + +This guide describes the Docker-based deployment flow, Signal linking, and update behavior. + +## Services + +`docker-compose.yml` defines these services: + +- `signal-cli`: runs `bbernhard/signal-cli-rest-api` in JSON-RPC mode. +- `signal-bot`: runs TurboBot from the configured main branch. +- `signal-bot-devel`: runs TurboBot from the configured development branch. +- `signal-bot-fs-editor`: optional filesystem editor/debug container that mounts bot volumes. + +## Volumes + +The compose file defines persistent volumes: + +- `signal-cli-data`: Signal account/device state. +- `signal-bot-data`: repository data for the main bot container. +- `signal-bot-devel-data`: repository data for the development bot container. + +## Bootstrap flow + +The bot containers use a shell command in `docker-compose.yml` to: + +1. Build `SETUP_SCRIPT_URL` from `GIT_REPO_URL`, `GIT_REPO_BRANCH`, and `SETUP_SCRIPT_NAME`. +2. Download `setup.sh` from that branch. +3. Mark it executable. +4. Run it. + +`setup.sh` then: + +1. Creates `GIT_REPO_PATH`. +2. Initializes the git repository if needed. +3. Configures the configured directory as a safe git directory. +4. Fetches from origin. +5. Resets hard to `origin/${GIT_REPO_BRANCH}`. +6. Initializes submodules. +7. Installs Python requirements. +8. Installs apt packages from `pkglist`. +9. Runs `run.sh`. + +## Auto-update flow + +`run.sh` launches `run.py` and monitors git for upstream changes. + +The default loop: + +1. Sleep for `CHECK_INTERVAL` seconds. +2. Run `git remote update`. +3. Compare local `@` with upstream `@{u}`. +4. If different, run `git pull`. +5. Kill the current Python process. +6. Relaunch `run.py`. + +If `run.py` exits with a code other than `143`, `run.sh` sets an exit flag and exits. + +## Operational warnings + +- `setup.sh` runs `git reset --hard origin/${GIT_REPO_BRANCH}`. Local uncommitted edits inside the deployed repository can be discarded. +- `run.sh` sources `secret.txt` as shell code. Keep permissions tight and never commit real secrets. +- Media handlers may require `ffmpeg` and working network access. +- Some handlers depend on third-party services that can rate-limit, change HTML/API behavior, or require credentials. + +## Linking Signal + +On first setup, run `signal-cli` in normal mode and link it to a Signal account/device. With the REST API running locally, the QR-code link endpoint is typically: + +```text +http://localhost:8181/v1/qrcodelink?device_name=local +``` + +Open that URL and scan the QR code with Signal on your phone. Keep the `signal-cli-data` volume so the linked device state persists. + +## Manual signal-cli reference + +The README contains an example `docker run` command generated from an existing `signal-cli` container. Use that only as a troubleshooting reference; the compose setup is the normal path. + +If you need to regenerate that style of command from a running container, the original project notes used `docker inspect` and `jq` like this: + +```bash +container_id="signal-cli" +docker inspect $container_id | jq -r ' + .[] | + "docker run " + + (if .Config.Env then (.Config.Env | map("--env \"" + . + "\"") | join(" ")) else "" end) + " " + + (if .Config.Entrypoint then "--entrypoint \"" + (.Config.Entrypoint | join(" ")) + "\" " else "" end) + + (if .Mounts then (.Mounts | map("--volume \"" + .Source + ":" + .Destination + "\"") | join(" ")) else "" end) + " " + + (if .Config.Cmd then (.Config.Cmd | join(" ")) else "" end) + + " " + .Config.Image +' +``` diff --git a/docs/README_MIGRATION.md b/docs/README_MIGRATION.md new file mode 100644 index 0000000..f395e03 --- /dev/null +++ b/docs/README_MIGRATION.md @@ -0,0 +1,25 @@ +# README migration checklist + +This checklist maps the original README content to the current documentation set so future reviewers can confirm that information was reorganized rather than dropped. + +| Original README topic | Current location | +| --- | --- | +| Docker Compose runs the Signal bot and Signal REST API | `README.md` quick start; `docs/OPERATIONS.md` services section | +| Main and devel bot containers use different branches | `docs/OPERATIONS.md` services section | +| Bot containers monitor GitHub and download updates | `docs/OPERATIONS.md` auto-update flow | +| Required Signal API and bot number environment variables | `README.md` quick start; `docs/CONFIGURATION.md` required variables | +| Contact/group allow lists and ignored groups | `README.md` quick start; `docs/CONFIGURATION.md` message allow/ignore controls | +| Instagram and OpenAI environment variables | `docs/CONFIGURATION.md` optional feature credentials; `secret.example.txt` | +| Historical `secrets.txt`/`secrets.sh` wording | `docs/CONFIGURATION.md` notes the current launcher reads `secret.txt` | +| Updating `docker-compose.yml` to point at the desired repo/branches | `docs/CONFIGURATION.md` Docker Compose configuration; `docs/OPERATIONS.md` bootstrap flow | +| Linking Signal with the QR-code endpoint | `README.md` Signal linking; `docs/OPERATIONS.md` linking Signal | +| Manual `signal-cli` `docker run` example | `README.md` manual signal-cli reference | +| `docker inspect`/`jq` command used to generate the manual `docker run` command | `docs/OPERATIONS.md` manual signal-cli reference | +| Windows/Linux/WSL/macOS development note | `docs/TESTING.md` required dependencies | +| Miniconda recommendation for Windows | `docs/TESTING.md` required dependencies | +| Installing Python dependencies and running unittest discovery | `README.md` development; `docs/TESTING.md` basic command and dependencies | +| `ffmpeg` and other system package caveats | `README.md` development; `docs/TESTING.md` dependencies and troubleshooting | + +## Secret-data review + +The checked-in secret template intentionally contains only placeholders. Real local credentials should live in `secret.txt`, which remains ignored by `.gitignore` through the existing `*.txt` and `*secret*` patterns. The repository explicitly un-ignores only `secret.example.txt` so users have a safe template to copy. diff --git a/docs/TESTING.md b/docs/TESTING.md new file mode 100644 index 0000000..45328dc --- /dev/null +++ b/docs/TESTING.md @@ -0,0 +1,74 @@ +# Testing + +TurboBot uses Python `unittest` tests under `tests/`. + +## Basic command + +Run all tests with: + +```bash +python -m unittest discover -s tests -p "test_*.py" +``` + +## Test infrastructure + +`tests/TurboTestCase.py` provides the shared bot test harness. Individual tests patch Signal API calls with mocks from `signalbot.utils`, especially: + +- `ReceiveMessagesMock` +- `SendMessagesMock` + +This allows tests to feed synthetic Signal messages into the bot and inspect outgoing replies without talking to a real Signal server. + +## Local deterministic tests + +Prefer adding tests that: + +- Avoid real network access. +- Mock external APIs. +- Mock file downloads and media conversion. +- Assert on send count, message text, and attachment count. + +`tests/test_run.py` and `tests/test_mmw.py` contain examples of command-level assertions. + +## Network or integration-sensitive tests + +Some existing tests and handlers may involve live external services or external binaries, including: + +- Reddit downloads. +- YouTube, TikTok, X/Twitter, and Bluesky media downloads. +- NASA/JPL asteroid API requests. +- OpenAI model calls. +- Instagram login/download behavior. +- Finance/ticker data through `yfinance`. + +These can fail because of network outages, API changes, rate limits, missing credentials, or missing system dependencies. When adding new tests for these areas, prefer mocking the service boundary unless the test is explicitly intended as an integration test. + +## Required dependencies + +Install Python dependencies: + +```bash +pip install -r requirements.txt +``` + +Development should work on Windows, Linux, WSL, and macOS. On Windows, the original project notes recommend Miniconda: create a new environment, install Python 3, and then run `pip3 install -r requirements.txt`. + +Some handlers also require apt/system packages from `pkglist`, especially media tooling such as `ffmpeg`. If you are developing outside Docker, install `ffmpeg` through your platform package manager, Conda, WSL, or similar. + +## Troubleshooting + +### Missing `signalbot_local/` + +`run.py` and tests try to add `signalbot_local/` to `sys.path` if present. This is optional in many environments, but useful when testing a local copy of the `signalbot` package. + +### Missing environment variables + +Some paths expect variables such as `SIGNAL_API_URL`, `BOT_NUMBER`, `CONTACT_NUMBERS`, `GROUP_NAMES`, or `OPENAI_API_KEY`. Tests often mock around these, but local manual runs need a configured `secret.txt`. + +### Missing external binaries + +Video download or conversion paths may require `ffmpeg`. Install packages from `pkglist` or use the Docker environment. + +### Network failures and rate limits + +Handlers that scrape or call third-party services may fail even if the code is correct. Re-run after confirming network access, credentials, and service availability. diff --git a/secret.example.txt b/secret.example.txt new file mode 100644 index 0000000..ce039db --- /dev/null +++ b/secret.example.txt @@ -0,0 +1,15 @@ +# Copy this file to secret.txt and fill in real values. +# Do not commit real secrets. + +export SIGNAL_API_URL="signal-cli:8181" +export BOT_NUMBER="+15555555555" + +# Allow private contacts and groups. Use semicolons for multiple values. +export CONTACT_NUMBERS="+15555555555" +export GROUP_NAMES="My Signal Group" +export IGNORE_GROUPS="TurboBot Devel" + +# Optional feature credentials. +export INSTA_USERNAME="myuser" +export INSTA_PASSWORD="replace-with-your-instagram-password" +export OPENAI_API_KEY="replace-with-your-openai-api-key"