19 sim shim should use generic dispatch like the real robot shim

.gitignore

@@ -6,4 +6,8 @@ __pycache__/
 .qibullet/
 *.run
 *.tar.gz
+
+# internal docs
+docs/
 roadmap/
+

README.md

@@ -9,7 +9,7 @@ Switch between modes by setting `NAOQI_IP` and `NAOQI_PORT` in your environment
 - **Robot** `NAOQI_IP=<robot.ip>` and `NAOQI_PORT=9559` runs the Python 2 `pynaoqi` bridge.
 - **Simulation** `NAOQI_IP=127.0.0.1` (or unset) runs the Python 3 `qiBullet` simulator.
 
-The client is agnostic to whether you connect to a physical or simulated robot. In both cases a Flask **shim server** on port 5000 exposes a unified HTTP API to your Python 3 client code. 
+The client is agnostic to whether you connect to a physical or simulated robot. In both cases a Flask **shim server** on port 5000 exposes a unified HTTP API to your Python 3 client code (see [`py3-naoqi-bridge/README.md`](py3-naoqi-bridge/README.md) for the wire contract and `NaoqiClient` reference).
 
 ## What's in the image
 
@@ -107,13 +107,18 @@ PepperBox/
 ├── entrypoint.sh           # mode dispatch + pre-flight checks
 ├── run.sh                  # standalone host launcher
 ├── setup.sh                # one-shot pynaoqi installer
-├── src/
-│   ├── shim_server.py      # qiBullet (sim) shim
+├── src/                    # qiBullet (sim) backend
+│   ├── shim_server.py      # Flask route + create_app factory
+│   ├── dispatcher.py       # per-module dispatch returning (result, is_stub)
+│   ├── driver.py           # QiBulletDriver sim lifecycle
+│   ├── post_tasks.py       # TaskRegistry for NAOqi post() async dispatch
+│   ├── adapters/           # per-module NAOqi adapters (motion, memory, ...)
 │   └── setup_wizard.py     # qiBullet asset installer
-└── py3-naoqi-bridge/
-    ├── shim_server.py      # pynaoqi (physical) shim
+└── py3-naoqi-bridge/        # pynaoqi (physical) backend + shared Python 3 client
+    ├── shim_server.py       # pynaoqi (physical) shim
+    ├── naoqi_proxy.py       # NaoqiClient — used against both shims
     ├── video_streamer.py
-    └── ...                 # NAOqi-side services
+    └── ...                  # NAOqi-side services and clients
 ```
 
 ## Develop

py3-naoqi-bridge/README.md

@@ -1,6 +1,6 @@
 # Python 3 to NAOqi Bridge
 
-This directory contains the Python 2.7 "shim" server and the Python 3 "proxy" client, which together form a bridge to allow Python 3 applications to communicate with the `naoqi` robot SDK.
+This directory contains the Python 2.7 "shim" server and the Python 3 "proxy" client. When put together they form a bridge to allow Python 3 applications to communicate with the `naoqi` robot SDK.
 
 ## Purpose
 
@@ -9,96 +9,56 @@ The `pynaoqi` SDK is only compatible with Python 2.7. The Python 2.7 shim server
 ## Components
 
 *   **`shim_server.py`**: The Python 2.7 Flask server that exposes the NAOqi API via HTTP.
-*   **`naoqi_proxy.py`**: The Python 3 client library that provides a convenient interface to interact with the shim server.
+*   **`naoqi_proxy.py`**: The Python 3 client library that provides an interface to interact with the shim server. Exposes the optional `warn_on_stubs` flag and `SimStubWarning` class for sim-mode introspection.
 *   **`examples/`**: Directory containing example scripts demonstrating the usage of the `NaoqiClient` (e.g., `basic_usage_example.py`, `helloworld_in_python3.py`).
-*   **`test_naoqi_proxy.py`**: A comprehensive test suite for the `naoqi_proxy` client.
+*   **`tests/`**: Unit and integration tests. `test_audio_publisher.py` covers the ZMQ audio publisher; `test_motion.py` is a scripted integration test against a running shim; `test_sim_stub_warning.py` covers the client-side opt-in warning behavior with mocked HTTP responses.
 
-## How to Run the Bridge
+## Using the NaoqiClient
 
-To use the bridge, both the Python 2.7 shim server and the NAOqi instance (simulated or physical robot) must be running and accessible.
+The shim server is launched by the project-level `./run.sh` — see the top-level README for setup and configuration (`NAOQI_IP`, `NAOQI_PORT`, `robot.env`, etc.). Once the shim is running on port 5000, Python 3 code uses the `NaoqiClient` to interface.
 
-### 1. Configure the NAOqi Connection
+### Initialisation
 
-The shim server connects to the NAOqi instance using the IP address and port specified in the `robot.env` file. Create this file in the `py3-naoqi-bridge` directory if it doesn't exist, and add the `NAOQI_IP` and `NAOQI_PORT`:
-
-```
-NAOQI_IP=127.0.0.1
-NAOQI_PORT=37647
-```
-
-Replace `127.0.0.1` and `41703` with the actual IP address and port of your NAOqi instance.
-
-### 2. Run the Python 2.7 Shim Server
-
-The server must be run from within the PepperBox Docker container. Launch it from the terminal using Python 2.7:
+```python
+from naoqi_proxy import NaoqiClient, NaoqiProxyError
 
-```bash
-python /home/pepperdev/apps/py3-naoqi-bridge/shim_server.py
+client = NaoqiClient(host="localhost", port=5000)
 ```
 
-This server will listen for incoming requests on `http://0.0.0.0:5000`.
-
-### 3. Use the Python 3 Proxy Client
-
-Once the shim server is running, you can use the `NaoqiClient` in your Python 3 applications. The client defaults to connecting to `http://localhost:5000`.
-
-#### Initialiation
+Optional sim-mode introspection:
 
 ```python
-from naoqi_proxy import NaoqiClient, NaoqiProxyError
-
-client = NaoqiClient(host="localhost", port=5000) # Host and port of the shim server
+client = NaoqiClient(warn_on_stubs=True)   # produce SimStubWarning on stub responses
+# or set NAOQI_SIM_WARN_STUBS=1 in the environment
 ```
 
-#### Calling NAOqi Methods
+### Calling NAOqi Methods
 
-Access NAOqi modules as attributes of the `client` object, and then call their methods. Arguments are passed directly.
+NAOqi modules are accessed as attributes of the client; methods are called directly with positional and keyword arguments.
 
 ```python
 try:
-    # Example: Make the robot say something
     client.ALTextToSpeech.say("Hello from Python 3 proxy")
-
-    # Example: Get robot configuration
     robot_config = client.ALMotion.getRobotConfig()
-    print(f"Robot configuration: {robot_config}")
-
-    # Example: Insert data into ALMemory
     client.ALMemory.insertData("myKey", "myValue")
-
 except NaoqiProxyError as e:
     print(f"NAOqi Proxy Error: {e}")
-except Exception as e:
-    print(f"An unexpected error occurred: {e}")
 ```
 
-#### Error Handling
-
-Errors originating from the NAOqi instance or communication issues will be raised as `NaoqiProxyError` exceptions on the Python 3 client side. It is recommended to wrap your NAOqi calls in `try...except NaoqiProxyError` blocks.
+Errors from the shim or the underlying NAOqi instance surface as `NaoqiProxyError`. Wrap calls in `try...except NaoqiProxyError` blocks.
 
 ## Testing the Bridge
 
-To verify the functionality of the bridge, you can run the provided test suite:
-
-1.  Ensure the Docker container is running.
-2.  Ensure the Python 2.7 shim server (`shim_server.py`) is running inside the container.
-3.  Execute the tests using Python 3:
+Unit tests for the client library and the audio publisher run inside the pepper-box container via the project-wide wrapper:
 
-    ```bash
-    python3 /home/pepperdev/apps/py3-naoqi-bridge/test_naoqi_proxy.py
-    ```
-
-### Tested Functionality
-
-The `test_naoqi_proxy.py` suite validates the following aspects of the bridge:
-
-*   **Basic Communication**: Successful invocation of `ALTextToSpeech.say` and `ALMotion.getRobotConfig`.
-*   **Error Propagation**: Correct handling and raising of `NaoqiProxyError` for non-existent NAOqi modules or methods.
-*   **Argument Handling**: Successful passing of various data types (strings, numbers, booleans, lists, dictionaries) to `ALMemory.insertData`.
+```bash
+./tests/run-in-docker.sh py3-naoqi-bridge/tests/test_sim_stub_warning.py -v
+./tests/run-in-docker.sh py3-naoqi-bridge/tests/test_audio_publisher.py -v
+```
 
-### Known Limitations
+The scripted integration test `tests/test_motion.py` requires a running shim and a real or simulated robot.
 
-*   **ALMemory String Retrieval in Simulator**: When using a simulated NAOqi instance, retrieving string values that were part of a list inserted into `ALMemory` (e.g., `client.ALMemory.insertData("myList", [1, "two", False])` and then `client.ALMemory.getData("myList")`) may result in the string values being returned as `None`. This appears to be a limitation of the simulated NAOqi environment rather than a bug in the bridge implementation. The `test_06_get_data_from_memory` test case in `test_naoqi_proxy.py` is currently commented out to reflect this.
+For the wider sim-side test suite (adapters, dispatcher, Flask routes) see the top-level `tests/unit/` directory and the project root README.
 
 ## API Reference (Shim Server)
 
@@ -116,9 +76,9 @@ The request body must be a JSON object with the following fields:
 *   `args` (array, optional): A list of positional arguments to pass to the method. Defaults to `[]`.
 *   `kwargs` (object, optional): A dictionary of keyword arguments to pass to the method. Defaults to `{}`.
 
-### Example Usage (cURL)
+### Example Usage (curl)
 
-Here is an example of using `curl` to make the robot say "Hello" via the shim server directly.
+Here is an example of using `curl` to make the robot say "Hello, world!" via the shim server directly.
 
 ```bash
 curl -X POST \

py3-naoqi-bridge/naoqi_proxy.py

@@ -1,9 +1,19 @@
 import json
+import os
 import urllib.request
+import urllib.error
+import warnings
+
 
 class NaoqiProxyError(Exception):
-    """Custom exception for errors from the NAOqi shim server."""
-    pass
+    """Raised when the shim returns an error response."""
+
+
+class SimStubWarning(UserWarning):
+    """Triggered when the sim shim returns a stub (no-op) response and the
+    client has opted into stub-aware warnings via `warn_on_stubs=True` or
+    the `NAOQI_SIM_WARN_STUBS=1` env var."""
+
 
 class NaoqiModule:
     def __init__(self, client, module_name):
@@ -21,8 +31,13 @@ def method_caller(*args, **kwargs):
         return method_caller
 
 class NaoqiClient:
-    def __init__(self, host="localhost", port=5000):
+    def __init__(self, host="localhost", port=5000, warn_on_stubs=None):
         self._shim_url = f"http://{host}:{port}/api/call"
+        if warn_on_stubs is None:
+            warn_on_stubs = os.environ.get("NAOQI_SIM_WARN_STUBS", "").lower() in (
+                "1", "true", "yes"
+            )
+        self._warn_on_stubs = warn_on_stubs
 
     def __getattr__(self, module_name):
         return NaoqiModule(self, module_name)
@@ -34,32 +49,46 @@ def _call_shim(self, module, method, args, kwargs):
             "args": list(args),
             "kwargs": kwargs,
         }
-
         req = urllib.request.Request(
             self._shim_url,
-            data=json.dumps(payload).encode('utf-8'),
-            headers={'Content-Type': 'application/json'}
+            data=json.dumps(payload).encode("utf-8"),
+            headers={"Content-Type": "application/json"},
         )
 
         try:
             with urllib.request.urlopen(req) as response:
-                response_data = json.loads(response.read().decode('utf-8'))
+                response_data = json.loads(response.read().decode("utf-8"))
+                if self._warn_on_stubs and self._is_sim_stub(response):
+                    warnings.warn(
+                        "{}.{} is a sim stub (no qibullet equivalent); "
+                        "returning {!r}".format(module, method, response_data.get("result")),
+                        SimStubWarning,
+                        stacklevel=3,
+                    )
                 if response.status == 200:
                     return True, response_data.get("result"), None
-                else:
-                    error_message = response_data.get("error", "Unknown error from shim server")
-                    return False, None, error_message
+                return False, None, response_data.get("error", "Unknown error")
         except urllib.error.HTTPError as e:
             error_message = f"HTTP Error {e.code}: {e.reason}"
             try:
-                error_data = json.loads(e.read().decode('utf-8'))
+                error_data = json.loads(e.read().decode("utf-8"))
                 error_message += f" - {error_data.get('error', 'No error details')}"
             except json.JSONDecodeError:
-                pass # Not a JSON error response
+                pass
             return False, None, error_message
         except urllib.error.URLError as e:
             return False, None, f"URL Error: {e.reason}"
         except json.JSONDecodeError:
             return False, None, "Failed to decode JSON response from shim server."
+        except SimStubWarning:
+            # Let SimStubWarning propagate even if it's been turned into an errorbby the warnings filter.
+            raise
         except Exception as e:
-            return False, None, f"An unexpected error occurred: {e}"
+            return False, None, f"An unexpected error occurred: {e}"
+
+    @staticmethod
+    def _is_sim_stub(response):
+        try:
+            return response.headers.get("X-Sim-Stub") == "1"
+        except Exception:
+            return False