diff --git a/.agents/skills/local-dev/SKILL.md b/.agents/skills/local-dev/SKILL.md new file mode 100644 index 00000000..aa044f7a --- /dev/null +++ b/.agents/skills/local-dev/SKILL.md @@ -0,0 +1,313 @@ +--- +name: local-dev +description: >- + Run, test, and debug the photofield server locally. Use when building, + running, or testing the server, making API calls, calling MCP tools, or + inspecting runtime state. Covers server lifecycle via `./tools/agent.sh server`, + generic HTTP calls via `./tools/agent.sh api`, MCP tool calls, database inspection, + error debugging, and common fixes. +--- + +# Local Development — Running and Testing the Server + +This skill covers building, running, testing, and debugging the photofield +server on your local machine. + +All tool invocation, server management, and API calls go through +`./tools/agent.sh` — a unified harness that handles the server lifecycle, +generic HTTP calls, and MCP tool invocation with session management, SSE +parsing, and named-arg parsing. + +## Verbose Flag + +The harness accepts `--verbose`, `-v`, and `-V`. Any of these can be placed +directly after `./tools/agent.sh`: + +```bash +./tools/agent.sh --verbose mcp call get_photo --file_id 1 +./tools/agent.sh -v mcp call get_photo --file_id 1 +./tools/agent.sh -V mcp call get_photo --file_id 1 +``` + +Or avoid the flag entirely via environment variable: +```bash +AGT_VERBOSE=1 ./tools/agent.sh mcp call get_photo --file_id 1 +``` + +## 1. Build + +```bash +go build -o photofield . +``` + +Kill any old instance before rebuilding: + +```bash +./tools/agent.sh server kill +``` + +## 2. Configuration + +The server reads `data/configuration.yaml`. Without it, collections point to +empty directories. + +**Minimal setup:** + +```bash +mkdir -p data +cat > data/configuration.yaml <` to manage the server process: + +```bash +# Start (auto-detects if already running) +./tools/agent.sh server start + +# Stop gracefully (uses PID file) +./tools/agent.sh server stop + +# Restart +./tools/agent.sh server restart + +# Check status (shows PID and port listeners) +./tools/agent.sh server status + +# Aggressive kill (PID file + all port listeners including exiftool) +./tools/agent.sh server kill +``` + +**How it works:** `server start` launches the binary with `nohup` and writes a +PID file to `data/agent.pid`. It then polls the server endpoint until +ready (up to 30s). `server stop` reads the PID file and sends SIGTERM. +`server kill` sends SIGKILL to the PID and anything else listening on the port. + +**Important:** The server does **not** auto-scan. After starting, run a scan: + +```bash +./photofield -scan test +# or from another directory: +AGT_BIN=/path/to/photofield ./tools/agent.sh server start && ./photofield -scan test +``` + +The server listens on port `8080` by default (override with `AGT_PORT`). + +**Note:** With `AGT_START=true` (the default), `mcp call`, `mcp quick`, and +`mcp shell` will auto-start the server if it is not already running. The +`server start`, `server stop`, `server restart`, `server status`, and +`server kill` commands manage the process regardless of `AGT_START`. + +### Stale PID cleanup + +If the server crashes without a clean shutdown, the PID file may become stale. +The harness auto-detects this: `server start` will clean up a dead PID file +and launch a new instance. `server status` also removes stale entries. + +## 4. MCP Tool Calls + +Use `./tools/agent.sh mcp` to call MCP tools. The harness handles the session +handshake (initialize + initialized notification), session ID extraction from +response headers, SSE response parsing, and named-arg to JSON conversion. + +### Tool commands + +```bash +# Call a tool with JSON args +./tools/agent.sh mcp call list_collections '{}' + +# Call with named args (auto-detects --key val pairs) +./tools/agent.sh mcp call search_photos --query 'beach' --collection_id 'test' --limit 3 + +# Verbose mode — shows full raw JSON response +./tools/agent.sh --verbose mcp call get_photo --file_id 1 --w 200 + +# Smoke test (calls list_collections by default) +./tools/agent.sh mcp quick + +# Smoke test with a specific tool +./tools/agent.sh mcp quick get_photo --file_id 1 + +# Interactive REPL +./tools/agent.sh mcp shell +``` + +### Argument modes + +| Mode | Syntax | +|------|--------| +| JSON | `./tools/agent.sh mcp call ''` | +| Named | `./tools/agent.sh mcp call --key val` | + +Named args are auto-converted to JSON: numbers stay numeric, `true`/`false` +become booleans, `null` stays null, everything else is quoted as strings. +`agent.sh` auto-detects the named-arg mode when the first token starts with +`--`, so no explicit `-- --` boundary is ever needed. + +### Output + +Non-verbose mode shows a clean summary: +``` +✓ list_collections — 22 items +✓ events — 19 events +✓ get_photo +``` + +Errors show with a red ✗ and the error message. Set `--verbose` (see Verbose Flag above) for full raw JSON on every call. + +**Output streams:** The `log_*` helpers (`ℹ`, `▶`) go to stderr. Tool result +summaries (`✓`, `✗`) and raw JSON output go to stdout. This lets you pipe tool +results: `./tools/agent.sh mcp call list_collections '{}' | jq '.collections'`. + +### From another directory + +```bash +AGT_BIN=/path/to/photofield ./tools/agent.sh mcp call list_collections '{}' +AGT_URL=http://remote-host:9000/mcp ./tools/agent.sh mcp call list_collections '{}' +``` + +## 5. Generic API Calls + +Use `./tools/agent.sh api` for arbitrary HTTP calls to any server endpoint. This is +useful for testing non-MCP routes, debugging, or calling endpoints that don't +have a dedicated tool. + +```bash +# GET request (health check) +./tools/agent.sh api GET http://localhost:8080/health + +# POST with JSON body +./tools/agent.sh api POST http://localhost:8080/api/collections \ + '{"name":"my-collection","dirs":["/path/to/photos"]}' + +# POST with named args (auto-constructs JSON body) +./tools/agent.sh api POST http://localhost:8080/api/collections \ + --name my-collection --dirs /path/to/photos + +# PUT / DELETE +./tools/agent.sh api DELETE http://localhost:8080/api/collections/test +``` + +The output shows the HTTP status code, pretty-printed JSON when possible, and +the response body (truncated if over 500 chars). Note: `AGT_VERBOSE=1` does +not currently change the truncation behavior for API calls. + +### Health Check + +The server exposes a health check endpoint at `/health`: + +```bash +./tools/agent.sh api GET http://localhost:8080/health +``` + +Returns `{"status": "ok"}` when healthy. + +## 6. Test the Server + +### Tool tests + +```bash +# Basic call +./tools/agent.sh mcp call get_photo --file_id 1 + +# Metadata-only call +./tools/agent.sh mcp call get_photo_metadata --file_id 1 + +# Error handling +./tools/agent.sh mcp call get_photo --file_id 999999 + +# Verbose debugging +./tools/agent.sh --verbose mcp call search_photos --query 'test' --collection_id 'test' +``` + +### API tests + +```bash +# Check health +./tools/agent.sh api GET http://localhost:8080/health + +# List collections via API (alternative to mcp call) +./tools/agent.sh api GET http://localhost:8080/api/collections +``` + +## 7. Inspect Errors and Crashes + +The harness captures the server's **entire stdout and stderr** to +`data/agent.log` via `nohup`. Panics and errors appear in this log: + +```bash +tail -100 data/agent.log +``` + +### Session warnings + +If the server does not return a `Mcp-Session-Id` header, the harness continues +without one (some MCP servers don't require it). You will see a benign info +message: `No session ID (server may not require one)`. + +| Symptom | Cause | Fix | +|---------|-------|-----| +| `cannot create context from nil parent` | Nil context passed to `WithTimeout` | Add `if ctx == nil { ctx = context.Background() }` in handler | +| `file not found: N` | Photo ID doesn't exist | Scan collection or check DB | +| Empty response data | Rendering panic | Check server log | +| Schema says all fields required | SDK infers from Go struct pointers | Use explicit `InputSchema` in tool registration | +| Server not responding | Old binary running | `./tools/agent.sh server kill` then rebuild | +| No photos found | Config points to empty dirs | Create `data/configuration.yaml` | + +## 8. Inspect Runtime State + +### Database + +```bash +sqlite3 data/photofield.cache.db "SELECT id, width, height FROM infos ORDER BY id;" +sqlite3 data/photofield.cache.db ".tables" +``` + +### Via the harness + +```bash +# List collections +./tools/agent.sh mcp call list_collections '{}' + +# Check events for a collection +./tools/agent.sh mcp call events --collection_id 'test' + +# Search photos +./tools/agent.sh mcp call search_photos --query 'faces' --collection_id 'test' --limit 5 +``` + +## Environment Variables + +| Variable | Default | Purpose | +|----------|---------|---------| +| `AGT_PORT` | `8080` | Server port | +| `AGT_BIN` | `./photofield` | Path to binary | +| `AGT_DATA_DIR` | `./data` | Data directory | +| `AGT_START` | `true` | Auto-start server if not running | +| `AGT_URL` | (derived) | Full MCP endpoint URL | +| `AGT_API_BASE` | `http://localhost:$PORT` | Base URL for API calls | +| `AGT_VERBOSE` | `0` | Verbose output | + +## Quick Reference + +| Command | Purpose | +|---------|---------| +| `go build -o photofield .` | Build the server | +| `./photofield -scan ` | Scan a collection | +| `./tools/agent.sh server start` | Start the server | +| `./tools/agent.sh server stop` | Stop the server | +| `./tools/agent.sh server restart` | Restart the server | +| `./tools/agent.sh server status` | Show PID/port status | +| `./tools/agent.sh server kill` | Kill server processes | +| `./tools/agent.sh mcp call ` | Call an MCP tool | +| `./tools/agent.sh mcp quick [tool]` | Smoke test | +| `./tools/agent.sh mcp shell` | Interactive REPL | +| `./tools/agent.sh api [body]` | Generic HTTP call | +| `./tools/agent.sh --verbose ` | Verbose output (see Verbose Flag above for `-v`/`-V`/env var) | +| `sqlite3 data/photofield.cache.db ...` | Inspect the database | diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 00000000..f632e516 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,9 @@ +{ + "mcpServers": { + "photofield": { + "url": "http://localhost:8080/mcp", + "transport": "http", + "directTools": true + } + } +} diff --git a/Taskfile.yml b/Taskfile.yml index f5f2337b..86ae7e22 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -48,6 +48,7 @@ tasks: - "echo ' task docs: Run the docs frontend in development mode'" - "echo ' task e2e: Run end-to-end tests in watch mode'" - "echo ' task release:local: Build, package, and create a local Docker image'" + silent: true commit:analyze: diff --git a/api.yaml b/api.yaml index dcc509b9..e0cbd4db 100644 --- a/api.yaml +++ b/api.yaml @@ -47,6 +47,62 @@ paths: schema: $ref: "#/components/schemas/Problem" + /collections/{id}/events: + get: + description: Split a collection's photos into time-bounded events based on time and location proximity. + tags: ["Display"] + parameters: + - name: id + in: path + required: true + description: Collection ID + schema: + $ref: "#/components/schemas/CollectionId" + responses: + "200": + description: List of events + content: + application/json: + schema: + $ref: "#/components/schemas/EventsList" + + /collections/{id}/files: + get: + description: Search a collection's photos by text or structured query. + tags: ["Source"] + parameters: + - name: id + in: path + required: true + schema: + $ref: "#/components/schemas/CollectionId" + + - name: search + in: query + description: Natural language or structured search query + schema: + $ref: "#/components/schemas/Search" + + - name: sort + in: query + description: Sort order. Prefix with `-` for descending or `+` for ascending, e.g. `-date` (newest first), `+date` (oldest first), `-similarity` (best matches first). Multiple values allowed. + schema: + $ref: "#/components/schemas/Sort" + + - name: limit + in: query + description: Maximum number of results + schema: + $ref: "#/components/schemas/Limit" + + responses: + "200": + description: Search results + content: + application/json: + schema: + $ref: "#/components/schemas/FileList" + /scenes: post: description: Create a new scene using the provided parameters @@ -1465,4 +1521,65 @@ components: $ref: "#/components/schemas/Color" text: type: string - description: Feature text to be displayed on the map \ No newline at end of file + description: Feature text to be displayed on the map + + EventSummary: + type: object + properties: + index: + type: integer + created_after: + type: string + format: date-time + created_before: + type: string + format: date-time + photo_count: + type: integer + location_count: + type: integer + locations: + type: array + items: + type: string + + EventsList: + type: object + properties: + items: + type: array + items: + $ref: "#/components/schemas/EventSummary" + + FileInfo: + type: object + properties: + id: + type: integer + file_name: + type: string + datetime: + type: string + format: date-time + width: + type: integer + height: + type: integer + color: + type: string + location: + type: string + similarity: + type: number + tags: + type: array + items: + type: string + + FileList: + type: object + properties: + items: + type: array + items: + $ref: "#/components/schemas/FileInfo" \ No newline at end of file diff --git a/dimensions_test.go b/dimensions_test.go new file mode 100644 index 00000000..627c094a --- /dev/null +++ b/dimensions_test.go @@ -0,0 +1,124 @@ +package main + +import "testing" + +func TestParsePreviewDimensions_Clamp(t *testing.T) { + tests := []struct { + name string + origW int + origH int + reqW *int + reqH *int + wantW int + wantH int + wantErr bool + }{ + { + name: "both specified exceed clamp", + origW: 8000, + origH: 6000, + reqW: intPtr(100000), + reqH: intPtr(100000), + wantW: 4096, + wantH: 4096, + wantErr: false, + }, + { + name: "both specified within limits", + origW: 8000, + origH: 6000, + reqW: intPtr(1000), + reqH: intPtr(2000), + wantW: 1000, + wantH: 2000, + wantErr: false, + }, + { + name: "width exceeds clamp, height within", + origW: 8000, + origH: 6000, + reqW: intPtr(5000), + reqH: intPtr(100), + wantW: 4096, + wantH: 100, + wantErr: false, + }, + { + name: "height exceeds clamp, width within", + origW: 8000, + origH: 6000, + reqW: intPtr(100), + reqH: intPtr(5000), + wantW: 100, + wantH: 4096, + wantErr: false, + }, + { + name: "only width specified exceeds clamp", + origW: 8000, + origH: 6000, + reqW: intPtr(10000), + reqH: nil, + wantW: 4096, + wantH: 3072, + wantErr: false, + }, + { + name: "only height specified exceeds clamp", + origW: 8000, + origH: 6000, + reqW: nil, + reqH: intPtr(10000), + wantW: 4096, + wantH: 3072, + wantErr: false, + }, + { + name: "neither specified, original within limits", + origW: 4000, + origH: 3000, + reqW: nil, + reqH: nil, + wantW: 4000, + wantH: 3000, + wantErr: false, + }, + { + name: "neither specified, original exceeds clamp", + origW: 8000, + origH: 6000, + reqW: nil, + reqH: nil, + wantW: 4096, + wantH: 3072, + wantErr: false, + }, + { + name: "zero dimensions rejected", + origW: 8000, + origH: 6000, + reqW: intPtr(0), + reqH: intPtr(100), + wantW: 0, + wantH: 0, + wantErr: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + gotW, gotH, err := parsePreviewDimensions(tt.origW, tt.origH, tt.reqW, tt.reqH) + if (err != nil) != tt.wantErr { + t.Errorf("parsePreviewDimensions() error = %v, wantErr %v", err, tt.wantErr) + return + } + if gotW != tt.wantW || gotH != tt.wantH { + t.Errorf("parsePreviewDimensions() = (%d, %d), want (%d, %d)", gotW, gotH, tt.wantW, tt.wantH) + } + }) + } +} + +func intPtr(i int) *int { + return &i +} diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index 242c64d3..d535e269 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -43,6 +43,7 @@ export default defineConfig({ { text: 'Search', link: '/features/search' }, { text: 'Tags', link: '/features/tags' }, { text: 'Reverse Geolocation', link: '/features/geolocation' }, + { text: 'MCP Server', link: '/mcp-server' }, ] }, { diff --git a/docs/mcp-server.md b/docs/mcp-server.md new file mode 100644 index 00000000..64d0fe54 --- /dev/null +++ b/docs/mcp-server.md @@ -0,0 +1,64 @@ +# MCP Server + +The Photofield MCP ([Model Context Protocol](https://modelcontextprotocol.io)) server lets LLM agents search and retrieve photos from your Photofield instance through a simple, agent-friendly interface that avoids manual browser navigation and raw API calls. It runs on the same port as the main server at `/mcp` by default. + +> **Experimental.** The MCP integration is early and not yet fully explored. It is a read-only interface, so feel free to experiment without worry. + +## Use Cases + +- **Find specific moments**: "Find the selfie I made in front of a brachiosaurus at the zoo" or "Show me photos where I'm wearing the red hat." The agent uses semantic search to locate exactly what you're thinking of. +- **Trip exploration**: "What did I do in Barcelona on the 14th of July?" The agent groups photos into chronological events, extracts location names, and walks you through your day. +- **Photo retrieval**: "Show me that picture of the sunset behind the lighthouse from the Sicily trip." The agent finds and displays the actual image. +- **Collection overview**: "How many photos do I have in total?" or "What collections are available?" The agent inspects collection metadata without user intervention. + +## Client Configuration + +Point your MCP client at the Photofield MCP endpoint: + +```json +{ + "mcpServers": { + "photofield": { + "url": "http://localhost:8080/mcp", + "transport": "http", + "directTools": true + } + } +} +``` + +## What It Does + +Five tools are available: + +| Tool | Purpose | +|---|---| +| `list_collections` | List all photo collections, their IDs, and indexed photo counts | +| `events` | Split a collection into chronological events (by day and location) | +| `search_photos` | Search photos using natural language, image similarity, or face similarity | +| `get_photo_metadata` | Retrieve structured metadata for a single photo: tags, faces, GPS coordinates, dimensions | +| `get_photo` | Retrieve an actual photo image as a base64-encoded image, with options for resizing and cropping | + +## AI / Semantic Search + +Text-based semantic search requires the [photofield-ai](https://github.com/SmilyOrg/photofield-ai) server to be running and configured in `configuration.yaml`: + +```yaml +ai: + textual: + host: http://localhost:8081 + visual: + host: http://localhost:8081 + faces: + host: http://localhost:8081 +``` + +Without the AI server, `search_photos` still works for tag, date, and filename filters, but text-based semantic search and face/image similarity searches will return no results. + +## Health Check + +The server exposes a health check endpoint at `/health`: + +| Path | Method | Description | +|------|--------|-------------| +| `/health` | GET | Returns `{"status": "ok"}` when healthy | diff --git a/go.mod b/go.mod index 3dc89ea9..7ec465fe 100644 --- a/go.mod +++ b/go.mod @@ -29,6 +29,7 @@ require ( github.com/karrick/godirwalk v1.15.6 github.com/lpar/gzipped v1.1.0 github.com/matoous/go-nanoid/v2 v2.0.0 + github.com/modelcontextprotocol/go-sdk v1.6.1 github.com/mostlygeek/go-exiftool v0.0.0-20190130212521-a0e5de16f760 github.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 github.com/peterstace/simplefeatures v0.44.0 @@ -63,6 +64,7 @@ require ( github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect github.com/golang/gddo v0.0.0-20210115222349-20d68f94ee1f // indirect github.com/golang/protobuf v1.5.2 // indirect + github.com/google/jsonschema-go v0.4.3 // indirect github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e // indirect github.com/google/uuid v1.6.0 // indirect github.com/gosimple/unidecode v1.0.0 // indirect @@ -84,12 +86,16 @@ require ( github.com/prometheus/common v0.26.0 // indirect github.com/prometheus/procfs v0.6.0 // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + github.com/segmentio/asm v1.1.3 // indirect + github.com/segmentio/encoding v0.5.4 // indirect github.com/tdewolff/minify/v2 v2.7.1-0.20200112204046-70870d25a935 // indirect github.com/tdewolff/parse/v2 v2.4.2 // indirect github.com/tetratelabs/wazero v1.9.0 // indirect github.com/wcharczuk/go-chart v2.0.2-0.20191206192251-962b9abdec2b+incompatible // indirect + github.com/yosida95/uritemplate/v3 v3.0.2 // indirect go.uber.org/atomic v1.7.0 // indirect - golang.org/x/sys v0.35.0 // indirect + golang.org/x/oauth2 v0.35.0 // indirect + golang.org/x/sys v0.41.0 // indirect golang.org/x/text v0.38.0 // indirect golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect gonum.org/v1/plot v0.0.0-20190410204940-3a5f52653745 // indirect diff --git a/go.sum b/go.sum index f48b6caf..62d7b7fb 100644 --- a/go.sum +++ b/go.sum @@ -259,6 +259,8 @@ github.com/gofrs/uuid v4.0.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRx github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY= +github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= github.com/golang-migrate/migrate/v4 v4.15.0-beta.1 h1:3iUwrd6V9oIzNc6TQdp4SLYNjQV1DXOK/E7cjaq7zbo= github.com/golang-migrate/migrate/v4 v4.15.0-beta.1/go.mod h1:QOmbm9b62AcsxBz7VbwJf+3mqgAyVrdKx7AQ8T9m5og= github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0= @@ -320,11 +322,13 @@ github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= -github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/go-github/v35 v35.2.0/go.mod h1:s0515YVTI+IMrDoy9Y4pHt9ShGpzHvHO8rZ7L7acgvs= github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0= +github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= @@ -532,6 +536,8 @@ github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0j github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/mitchellh/mapstructure v0.0.0-20170523030023-d0303fe80992/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v0.0.0-20180220230111-00c29f56e238/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU= +github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= @@ -613,6 +619,10 @@ github.com/rs/zerolog v1.15.0/go.mod h1:xYTKnLHcpfU2225ny5qZjxnj9NvkumZYjJHlAThC github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd h1:CmH9+J6ZSsIjUK3dcGsnCnO41eRBOnY12zwkn5qVwgc= github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk= github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0= +github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc= +github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg= +github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0= +github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0= github.com/sheerun/queue v1.0.1 h1:TIAQyN0aRRvrJcNa2beZFfxwuxrfXBc9Mj+UWDNH7Ao= github.com/sheerun/queue v1.0.1/go.mod h1:YtjrWT5jymvCLo/lEWDk3sv7A1Kgj0qcl3SZx7Zmcfo= github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24/go.mod h1:M+9NzErvs504Cn4c5DxATwIqPbtswREoFCre64PpcG4= @@ -674,6 +684,8 @@ github.com/xanzy/go-gitlab v0.15.0/go.mod h1:8zdQa/ri1dfn8eS3Ir1SyfvOKlw7WBJ8DVT github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI= github.com/xdg-go/scram v1.0.2/go.mod h1:1WAq6h33pAW+iRreB34OORO2Nf7qel3VV3fjBj+hCSs= github.com/xdg-go/stringprep v1.0.2/go.mod h1:8F9zXuvzgwmyT5DUm4GUfZGDdT3W+LCvS6+da4O5kxM= +github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4= +github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4= github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= @@ -822,6 +834,8 @@ golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210413134643-5e61552d6c78/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ= +golang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/sync v0.0.0-20170517211232-f52d1811a629/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -907,8 +921,8 @@ golang.org/x/sys v0.0.0-20210521090106-6ca3eb03dfc2/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= -golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= +golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= diff --git a/internal/collection/collection.go b/internal/collection/collection.go index 6fe5f768..92c447a2 100644 --- a/internal/collection/collection.go +++ b/internal/collection/collection.go @@ -15,12 +15,12 @@ import ( type Collection struct { Id string `json:"id"` Name string `json:"name"` - Layout string `json:"layout"` - Sort string `json:"sort"` - Limit int `json:"limit"` - IndexLimit int `json:"index_limit"` - ExpandSubdirs bool `json:"expand_subdirs"` - ExpandSort string `json:"expand_sort"` + Layout string `json:"layout,omitempty"` + Sort string `json:"sort,omitempty"` + Limit int `json:"limit,omitempty"` + IndexLimit int `json:"index_limit,omitempty"` + ExpandSubdirs bool `json:"expand_subdirs,omitempty"` + ExpandSort string `json:"expand_sort,omitempty"` Dirs []string `json:"dirs"` IndexedAt *time.Time `json:"indexed_at,omitempty"` IndexedCount int `json:"indexed_count"` diff --git a/internal/collection/events.go b/internal/collection/events.go new file mode 100644 index 00000000..57697217 --- /dev/null +++ b/internal/collection/events.go @@ -0,0 +1,127 @@ +package collection + +import ( + "context" + "time" + + "github.com/golang/geo/s2" + + "photofield/internal/image" +) + +const ( + eventGapTime = time.Hour + locationGapTime = 15 * 60 // seconds + locationGapDist = 1.0 // km +) + +// EventSummary represents a time-bounded event within a collection. +type EventSummary struct { + Index int `json:"index,omitempty"` + CreatedAfter string `json:"created_after"` + CreatedBefore string `json:"created_before"` + PhotoCount int `json:"photo_count"` + LocationCount int `json:"location_count,omitempty"` + Locations []string `json:"locations,omitempty"` +} + +// SplitIntoEvents splits a collection's photos into events based on time and +// location proximity. Photos are grouped when they fall within the same day +// and are no more than eventGapTime apart. Reverse-geocoding is applied to +// photo locations that are >1km apart and >15 minutes apart. +func (collection *Collection) SplitIntoEvents(ctx context.Context, source *image.Source) ([]EventSummary, error) { + infos, _ := collection.GetInfos(source, image.ListOptions{}) + + var events []EventSummary + var current *EventSummary + var lastPhotoTime time.Time + var lastLocTime time.Time + var lastLatLng s2.LatLng + locations := make(map[string]struct{}) + + for info := range infos { + photoTime := info.DateTime + if photoTime.IsZero() { + continue + } + + if current == nil { + current = &EventSummary{ + CreatedAfter: photoTime.Format(time.RFC3339), + } + } else { + elapsed := photoTime.Sub(lastPhotoTime) + if elapsed < 0 { + elapsed = -elapsed + } + if elapsed > eventGapTime || !sameDay(lastPhotoTime, photoTime) { + // Finalize previous event + current.CreatedBefore = lastPhotoTime.Format(time.RFC3339) + current.Locations = make([]string, 0, len(locations)) + for loc := range locations { + current.Locations = append(current.Locations, loc) + } + events = append(events, *current) + + // Start new event + current = &EventSummary{ + CreatedAfter: photoTime.Format(time.RFC3339), + } + locations = make(map[string]struct{}) + lastLatLng = s2.LatLng{} // reset reference point for new event + } + } + + current.PhotoCount++ + lastPhotoTime = photoTime + + // Location tracking + if source.Geo != nil && source.Geo.Available() { + lastLocCheck := lastLocTime.Sub(photoTime) + if lastLocCheck < 0 { + lastLocCheck = -lastLocCheck + } + queryLocation := lastLocTime.IsZero() || lastLocCheck > time.Duration(locationGapTime)*time.Second + if queryLocation && image.IsValidLatLng(info.LatLng) { + lastLocTime = photoTime + dist := image.AngleToKm(lastLatLng.Distance(info.LatLng)) + if dist > locationGapDist { + location, err := source.Geo.ReverseGeocode(ctx, info.LatLng) + if err == nil { + locations[location] = struct{}{} + lastLatLng = info.LatLng + } + } + } + } + } + + // Finalize last event + if current != nil && current.PhotoCount > 0 { + current.CreatedBefore = lastPhotoTime.Format(time.RFC3339) + current.Locations = make([]string, 0, len(locations)) + for loc := range locations { + current.Locations = append(current.Locations, loc) + } + events = append(events, *current) + } + + // Assign indices + for i := range events { + events[i].Index = i + 1 + events[i].LocationCount = len(events[i].Locations) + } + + if events == nil { + events = make([]EventSummary, 0) + } + + return events, nil +} + +// sameDay reports whether a and b are on the same calendar day. +func sameDay(a, b time.Time) bool { + y1, m1, d1 := a.Date() + y2, m2, d2 := b.Date() + return y1 == y2 && m1 == m2 && d1 == d2 +} diff --git a/internal/collection/search.go b/internal/collection/search.go new file mode 100644 index 00000000..00cf90e1 --- /dev/null +++ b/internal/collection/search.go @@ -0,0 +1,229 @@ +package collection + +import ( + "context" + "fmt" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/golang/geo/s2" + + "photofield/internal/ai" + "photofield/internal/image" + "photofield/internal/layout" + "photofield/internal/search" +) + +// SortType represents a sort specification for search results. +// Supports +/- prefixes for descending/ascending, and supports +// multiple sort fields (e.g. "-similarity,+date"). +type SortType string + +// SortOrder maps a SortType to an image.ListOrder. +// Returns the first recognized order, or DateDesc as default. +func SortOrder(s SortType) image.ListOrder { + if s == "" { + return image.DateDesc + } + lo := layout.OrderFromSort(string(s)) + return image.ListOrder(lo) +} + +// SortOrders parses a sort string and returns a slice of ListOrders. +// The primary sort is the first element; secondary sorts are appended. +// Supports formats like "-similarity,+date" or just "-date". +func SortOrders(s SortType) []image.ListOrder { + if s == "" { + return []image.ListOrder{image.DateDesc} + } + orders := make([]image.ListOrder, 0) + parts := strings.Split(string(s), ",") + for _, part := range parts { + part = strings.TrimSpace(part) + if part == "" { + continue + } + lo := layout.OrderFromSort(part) + if lo != layout.None { + orders = append(orders, image.ListOrder(lo)) + } + } + if len(orders) == 0 { + return []image.ListOrder{image.DateDesc} + } + return orders +} + +// SortPrimaryOrder returns the primary sort order. +func SortPrimaryOrder(s SortType) image.ListOrder { + orders := SortOrders(s) + if len(orders) == 0 { + return image.DateDesc + } + return orders[0] +} + +// SortIsSimilarity returns true if the primary sort is similarity-based. +func SortIsSimilarity(s SortType) bool { + lo := SortPrimaryOrder(s) + return lo == image.SimilarityDesc || lo == image.SimilarityAsc +} + +// SearchOptions holds the parameters for a collection search. +type SearchOptions struct { + QueryStr string + Sort SortType + Limit int +} + +// SearchResult represents a single search result with metadata. +type SearchResult struct { + Id int32 `json:"id"` + FileName string `json:"file_name"` + DateTime string `json:"datetime,omitempty"` + Width int `json:"width,omitempty"` + Height int `json:"height,omitempty"` + Color string `json:"color,omitempty"` // hex color + Location string `json:"location,omitempty"` // reverse-geocoded, if geo available + Similarity float32 `json:"similarity,omitempty"` + Tags []string `json:"tags,omitempty"` // tags on this photo +} + +// Search searches a collection's photos by text, image reference, face reference, +// or structured qualifiers. Returns metadata and similarity scores. +func (collection *Collection) Search( + ctx context.Context, + source *image.Source, + opts SearchOptions, +) ([]SearchResult, []search.Token, []search.FieldMeta, error) { + // 1. Parse query + var tokens []search.Token + var expr search.Expression + var parseErr error + + if opts.QueryStr != "" { + q, err := search.Parse(opts.QueryStr) + if err != nil { + return nil, nil, nil, fmt.Errorf("parse failed: %w", err) + } + tokens = q.Tokens() + expr, parseErr = q.Expression() + } + + // 2. Resolve embeddings + var imageEmbedding ai.Embedding + var faceEmbedding ai.Embedding + + if parseErr == nil && expr.Image.Present { + emb, err := source.GetImageEmbedding(image.ImageId(expr.Image.Value)) + if err != nil { + return nil, tokens, expr.Errors, fmt.Errorf("image embed failed: %w", err) + } + imageEmbedding = emb + } + + if parseErr == nil && imageEmbedding == nil && expr.Face.Present { + emb, err := source.GetFaceEmbedding(int(expr.Face.Value)) + if err != nil { + return nil, tokens, expr.Errors, fmt.Errorf("face embed failed: %w", err) + } + faceEmbedding = emb + } + + if parseErr == nil && imageEmbedding == nil && expr.Text != "" { + emb, err := source.Clip.EmbedText(expr.Text) + if err != nil { + return nil, tokens, expr.Errors, fmt.Errorf("text embed failed: %w", err) + } + imageEmbedding = emb + } + + // 3. Determine sort order + order := SortPrimaryOrder(opts.Sort) + + // No default threshold — let all results through and let the caller + // control filtering via `t:X` if desired. (The original loadScene used + // 0.262 for non-similarity sorts but that was too aggressive for some + // datasets.) + + // 4. Query DB + limit := opts.Limit + if limit <= 0 { + limit = 50 + } + + infos, _ := collection.GetInfos(source, image.ListOptions{ + OrderBy: order, + Limit: limit, + Expression: expr, + ImageEmbedding: imageEmbedding, + FaceEmbedding: faceEmbedding, + }) + + // 5. Collect results + results := make([]SearchResult, 0) + var lastLocTime time.Time + var lastLatLng s2.LatLng + + for info := range infos { + res := SearchResult{ + Id: int32(info.Id), + FileName: filepath.Base(getImagePath(source, info.Id)), + Similarity: info.Similarity, + } + + if !info.DateTime.IsZero() { + res.DateTime = info.DateTime.Format(time.RFC3339) + } + res.Width = info.Width + res.Height = info.Height + + // Color as hex string + if info.Color != 0 { + res.Color = fmt.Sprintf("#%06x", info.Color&0xFFFFFF) + } + + // Reverse-geocode (same logic as events: 15 min / 1 km gap) + if source.Geo != nil && source.Geo.Available() && image.IsValidLatLng(info.LatLng) { + lastLocCheck := lastLocTime.Sub(info.DateTime) + if lastLocCheck < 0 { + lastLocCheck = -lastLocCheck + } + queryLocation := lastLocTime.IsZero() || lastLocCheck > 15*time.Minute + if queryLocation { + lastLocTime = info.DateTime + dist := image.AngleToKm(lastLatLng.Distance(info.LatLng)) + if dist > 1.0 { + location, err := source.Geo.ReverseGeocode(ctx, info.LatLng) + if err == nil { + lastLatLng = info.LatLng + res.Location = location // assign to current photo + } + } + } + } + + // Tags + tagNames := make([]string, 0) + for t := range source.ListImageTags(info.Id) { + tagNames = append(tagNames, t.Name) + } + sort.Strings(tagNames) + res.Tags = tagNames + + results = append(results, res) + } + + if results == nil { + results = make([]SearchResult, 0) + } + + return results, tokens, expr.Errors, parseErr +} + +func getImagePath(source *image.Source, id image.ImageId) string { + path, _ := source.GetImagePath(id) + return path +} diff --git a/internal/mcp/mcp.go b/internal/mcp/mcp.go new file mode 100644 index 00000000..9fe7c790 --- /dev/null +++ b/internal/mcp/mcp.go @@ -0,0 +1,364 @@ +// Package mcp provides a Model Context Protocol (MCP) server for the +// photofield application, mounted onto the chi HTTP router. +package mcp + +import ( + "context" + "fmt" + "net" + "net/http" + "os" + "sync/atomic" + + "github.com/modelcontextprotocol/go-sdk/mcp" + + "photofield/internal/collection" + "photofield/internal/image" +) + +// Server holds the MCP server instance and its chi-mountable HTTP handler. +type Server struct { + srv *mcp.Server + handler http.Handler + baseURL atomic.Value // set from request Host header per request (stores string) + apiPrefix string // e.g. "/api" — used for constructing file URLs +} + +// New creates a new MCP server for photofield with the given data sources +// and registers all available tools. The base URL is derived at request time +// from the incoming request's Host header, with `addr` used as a fallback +// default (derived from the listener address). `apiPrefix` is the HTTP route +// prefix for file endpoints (e.g. "/api"). Callers should mount handler() +// on a chi router, e.g.: +// +// r.Mount("/mcp", s.handler()) +func New(collections *[]collection.Collection, imageSource *image.Source, addr, apiPrefix string) (*Server, error) { + sdkSrv := mcp.NewServer(&mcp.Implementation{ + Name: "photofield", + Version: "dev", + }, nil) + + // Handler closures capture collections, imageSource, and a pointer to this Server + // so they can read the current base URL at request time. + srv := &Server{srv: sdkSrv, apiPrefix: apiPrefix} + + mcp.AddTool(sdkSrv, &mcp.Tool{ + Name: "list_collections", + Description: "List all photo collections. Use this first — the collection_id from the response is required by all other tools. " + + "If indexed_count is 0 or indexed_at is missing, the collection has not been indexed yet.", + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{}, + }, + }, listCollections(collections, imageSource)) + + mcp.AddTool(sdkSrv, &mcp.Tool{ + Name: "events", + Description: "Split a collection's photos into chronological events. Returns metadata summaries (photo count, date ranges, location count) — NOT the photo images themselves. " + + "Use after list_collections, before search_photos, to get high-level context about where and when photos were taken.", + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{ + "collection_id": map[string]any{"type": "string", "description": "The collection ID from list_collections. Use the 'id' field from the collection object returned by list_collections."}, + }, + "required": []string{"collection_id"}, + }, + }, eventsHandler(collections, imageSource)) + + mcp.AddTool(sdkSrv, &mcp.Tool{ + Name: "search_photos", + Description: "Search a collection's photos by text, image reference (img:ID), or face reference (face:ID). Returns metadata summaries — NOT the image data. " + + "⚠️ Use get_photo(file_id) to verify key results visually before showing photos to the user — metadata and similarity scores can be misleading. " + + "Use your judgment on when to verify: for single specific results, definitely check; for browsing large sets, verify only the top matches.\n\n" + + "Results beyond the limit are silently discarded. Use get_photo_metadata on results to get preview_url for markdown embedding, or get_photo for the full image.\n\n" + + "QUERY TYPES:\n" + + "- Text search: e.g. 'red car on highway' — uses CLIP embeddings, sorted by match quality\n" + + "- Image similarity: e.g. 'img:1234' — finds photos visually similar to the given image ID\n" + + "- Face similarity: e.g. 'face:5678' — finds photos containing similar faces\n\n" + + "COMBINABLE QUALIFIERS (mix with text or use standalone):\n" + + "- tag:name — filter by tag (e.g. 'vacation', 'fav')\n" + + "- filename:text — filter by filename (supports * and ? wildcards, e.g. 'filename:*.png')\n" + + "- created:YYYY-MM-DD — filter by date (supports ranges like 'created:2023-01-01..2023-12-31', wildcards like 'created:*-12-25', and operators like 'created:>=2023-06-15')\n" + + "- t:X — similarity threshold filter (0.15-0.30, where higher = more strict; e.g. 'beach sunset t:0.25')\n" + + "- dedup:X — filter duplicates by similarity (0-1, e.g. 'dedup:0.9' keeps only photos <90% similar to each other)\n\n" + + "SORT OPTIONS (passed as the 'sort' parameter):\n" + + "- -date (default) — newest first\n" + + "- +date — oldest first\n" + + "- -similarity — best matches first\n" + + "- +similarity — worst matches first\n" + + "- +shuffle-hourly — random within each hour\n" + + "- +shuffle-daily — random within each day\n" + + "- +shuffle-weekly — random within each week\n" + + "- +shuffle-monthly — random within each month\n" + + "- Multiple fields: e.g. '-similarity,+date' (sort by similarity, break ties newest first)\n\n" + + "EXAMPLES:\n" + + "- 'beach sunset' — semantically search for beach/sunset photos\n" + + "- 'beach sunset t:0.25' — find beach sunsets with at least 0.25 similarity\n" + + "- 'created:2023-06..2023-08 tag:vacation' — vacation photos from summer 2023\n" + + "- 'img:100 tag:fav' — favorited photos similar to image 100\n" + + "- 'dog' sort:-similarity — dogs sorted by relevance\n" + + "- 'dog' sort:+shuffle-daily — random order, but grouped by day\n" + + "- 'portrait' sort:-similarity,+date — best portraits first, newest tiebreak\n" + + "- 'filename:IMG_*.jpg' — all IMG_ photos, oldest first\n\n" + + "VERIFICATION: For single specific results, call get_photo on top matches to confirm content. For large browse results, verify the top 1-3 matches before presenting.", + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{ + "collection_id": map[string]any{"type": "string", "description": "The collection ID from list_collections."}, + "query": map[string]any{"type": "string", "description": "Search query: natural language text, image similarity (img:ID), or face similarity (face:ID)."}, + "sort": map[string]any{"type": [2]string{"null", "string"}, "description": "Sort order. '-date' (newest) by default. Options: +date, -similarity, +similarity, +shuffle-hourly, +shuffle-daily, +shuffle-weekly, +shuffle-monthly."}, + "limit": map[string]any{"type": [2]string{"null", "integer"}, "description": "Max results. Default 50. Results beyond limit are silently discarded."}, + }, + "required": []string{"collection_id", "query"}, + }, + }, searchPhotosHandler(collections, imageSource)) + + mcp.AddTool(sdkSrv, &mcp.Tool{ + Name: "get_photo_metadata", + Description: "Retrieve structured photo metadata (dimensions, path, dates, tags, faces, location, URLs). " + + "⚠️ Metadata alone may not be reliable — if you're about to show a photo to the user based on metadata or search results, call get_photo(file_id) to visually confirm it actually contains what you claim. " + + "Tags, location, and similarity scores can be wrong or misleading. " + + "Returns preview_url and original_url fields. " + + "Do NOT output raw HTML () or bare URLs to display photos — the MCP client will not render them. " + + "Use preview_url directly in markdown syntax (![alt](url)). " + + "Use after list_collections, events, or search_photos to inspect details on specific file_ids.", + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{ + "file_id": map[string]any{"type": "integer", "description": "The photo file ID (required). Obtain from search_photos results."}, + }, + "required": []string{"file_id"}, + }, + }, getPhotoMetadataHandler(imageSource, srv)) + + mcp.AddTool(sdkSrv, &mcp.Tool{ + Name: "get_photo", + Description: "Retrieve a photo as a base64-encoded image. This is the only tool that returns actual image data. " + + "Use this as a verification tool — call get_photo(file_id) on search results to visually confirm the photo contains what you expect before showing it to the user. " + + "Metadata and search scores can be misleading, so visual confirmation is recommended for key results.\n\n" + + "⚠️ DO NOT output raw HTML () or bare image URLs — the MCP client will not render them. Always call get_photo(file_id) instead. " + + "Default (file_id only): 256x256 JPEG thumbnail — fast and usually sufficient for verification. " + + "Format: jpeg (default), png, webp. " + + "Crop params (crop_x/y/w/h) are in original image pixel coordinates; all four must be specified together. " + + "Only pass optional parameters (w, h, crop) when the thumbnail is too small to verify details.\n\n" + + "KEY RULE: Always call get_photo on any photo you're about to show the user or confirm in your response. " + + "Use your judgment on when to verify for intermediate/browsing results — check top matches, but don't feel you must check every single result.", + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{ + "file_id": map[string]any{"type": "integer", "description": "The photo file ID."}, + "w": map[string]any{"type": [2]string{"null", "integer"}, "description": "Target width in pixels (1-4096). Omit unless investigating details — always start with the default 256x256 thumbnail."}, + "h": map[string]any{"type": [2]string{"null", "integer"}, "description": "Target height in pixels (1-4096). Omit unless investigating details — always start with the default 256x256 thumbnail."}, + "format": map[string]any{"type": [2]string{"null", "string"}, "description": "Output format. Default: jpeg. Options: jpeg, png, webp."}, + "crop_x": map[string]any{"type": [2]string{"null", "integer"}, "description": "Crop left edge in original image pixels. Must specify all four crop params together."}, + "crop_y": map[string]any{"type": [2]string{"null", "integer"}, "description": "Crop top edge in original image pixels."}, + "crop_w": map[string]any{"type": [2]string{"null", "integer"}, "description": "Crop width in original image pixels."}, + "crop_h": map[string]any{"type": [2]string{"null", "integer"}, "description": "Crop height in original image pixels."}, + }, + "required": []string{"file_id"}, + }, + }, getPhotoHandler(imageSource, srv)) + + h := mcp.NewStreamableHTTPHandler(func(_ *http.Request) *mcp.Server { + return sdkSrv + }, nil) + + // Derive a default base URL from the listener address for fallback when + // the Host header is absent (e.g. behind certain reverse proxies). + var fallbackAddr string + if addr != "" { + _, p, err := net.SplitHostPort(addr) + if err != nil { + p = addr // might be a bare port like "8080" + } + if p == "" { + p = "8080" + } + fallbackAddr = net.JoinHostPort("localhost", p) + } else { + fallbackAddr = "localhost:8080" + } + + // Wrap with panic recovery to prevent server crashes from tool handler panics. + // Also extract the Host header from each request and store it in srv.baseURL + // so that tool handlers can construct absolute image URLs. + wrappedHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + host := r.Host + if host == "" { + host = fallbackAddr + } + scheme := "http" + if r.TLS != nil { + scheme = "https" + } + srv.baseURL.Store(scheme + "://" + host) + + var written bool + // Wrap ResponseWriter to detect if WriteHeader was called + wrappedW := &responseWriterWrapper{ResponseWriter: w, wroteHeader: &written} + defer func() { + if rec := recover(); rec != nil { + fmt.Fprintln(os.Stderr, "MCP handler recovered from panic:", rec) + // Try to write an error response if not already written + if !written { + wrappedW.Header().Set("Content-Type", "application/json") + wrappedW.WriteHeader(http.StatusInternalServerError) + } + } + }() + h.ServeHTTP(wrappedW, r) + }) + + // Initialize baseURL with the fallback default; the wrappedHandler + // overwrites it per-request. + srv.baseURL.Store("http://" + fallbackAddr) + + srv.handler = wrappedHandler + return srv, nil +} + +// --- list_collections --- + +type collectionsInput struct{} + +type collectionsOutput struct { + Items []collection.Collection `json:"items"` +} + +func listCollections(collections *[]collection.Collection, imageSource *image.Source) mcp.ToolHandlerFor[collectionsInput, collectionsOutput] { + return func(_ context.Context, _ *mcp.CallToolRequest, _ collectionsInput) (*mcp.CallToolResult, collectionsOutput, error) { + var items []collection.Collection + for i := range *collections { + c := &(*collections)[i] + c.UpdateIndexedAt(imageSource) + items = append(items, *c) + } + if items == nil { + items = make([]collection.Collection, 0) + } + return nil, collectionsOutput{Items: items}, nil + } +} + +// --- events --- + +type eventsInput struct { + CollectionId string `json:"collection_id" jsonschema:"The collection ID from list_collections."` +} + +type eventsOutput struct { + Events []collection.EventSummary `json:"events"` +} + +func eventsHandler(collections *[]collection.Collection, imageSource *image.Source) mcp.ToolHandlerFor[eventsInput, eventsOutput] { + return func(ctx context.Context, _ *mcp.CallToolRequest, input eventsInput) (*mcp.CallToolResult, eventsOutput, error) { + defer func() { + if r := recover(); r != nil { + fmt.Fprintln(os.Stderr, "events handler recovered from panic:", r) + } + }() + // Find the collection + var coll *collection.Collection + for i := range *collections { + if (*collections)[i].Id == input.CollectionId { + coll = &(*collections)[i] + break + } + } + if coll == nil { + return nil, eventsOutput{}, fmt.Errorf("collection not found: %s", input.CollectionId) + } + + // Delegate to collection method + events, err := coll.SplitIntoEvents(ctx, imageSource) + if err != nil { + return nil, eventsOutput{}, err + } + return nil, eventsOutput{Events: events}, nil + } +} + +// --- search_photos --- + +type searchPhotosInput struct { + CollectionId string `json:"collection_id" jsonschema:"The collection ID from list_collections."` + Query string `json:"query" jsonschema:"Search query: natural language text, image similarity (img:ID), or face similarity (face:ID)."` + Sort *string `json:"sort" jsonschema:"Sort order. '-date' (newest) by default. Options: +date, -similarity, +similarity, +shuffle-hourly, +shuffle-daily, +shuffle-weekly, +shuffle-monthly, or comma-joined like '-similarity,+date'."` + Limit *int `json:"limit" jsonschema:"Max results. Default 50. Results beyond limit are silently discarded."` +} + +type searchPhotosOutput struct { + Items []collection.SearchResult `json:"items"` +} + +func searchPhotosHandler(collections *[]collection.Collection, imageSource *image.Source) mcp.ToolHandlerFor[searchPhotosInput, searchPhotosOutput] { + return func(ctx context.Context, _ *mcp.CallToolRequest, input searchPhotosInput) (*mcp.CallToolResult, searchPhotosOutput, error) { + defer func() { + if r := recover(); r != nil { + fmt.Fprintln(os.Stderr, "search_photos handler recovered from panic:", r) + } + }() + // Find the collection + var coll *collection.Collection + for i := range *collections { + if (*collections)[i].Id == input.CollectionId { + coll = &(*collections)[i] + break + } + } + if coll == nil { + return nil, searchPhotosOutput{}, fmt.Errorf("collection not found: %s", input.CollectionId) + } + + limit := 50 + if input.Limit != nil && *input.Limit > 0 { + limit = *input.Limit + } + + sort := collection.SortType("") + if input.Sort != nil && *input.Sort != "" { + sort = collection.SortType(*input.Sort) + } + + opts := collection.SearchOptions{ + QueryStr: input.Query, + Sort: sort, + Limit: limit, + } + + items, _, _, err := coll.Search(ctx, imageSource, opts) + if err != nil { + return nil, searchPhotosOutput{}, err + } + if items == nil { + items = make([]collection.SearchResult, 0) + } + return nil, searchPhotosOutput{Items: items}, nil + } +} + +// responseWriterWrapper wraps http.ResponseWriter to track if WriteHeader was called. +type responseWriterWrapper struct { + http.ResponseWriter + wroteHeader *bool +} + +func (w *responseWriterWrapper) WriteHeader(statusCode int) { + if !*w.wroteHeader { + *w.wroteHeader = true + w.ResponseWriter.WriteHeader(statusCode) + } +} + +func (w *responseWriterWrapper) Write(b []byte) (int, error) { + if !*w.wroteHeader { + *w.wroteHeader = true + w.ResponseWriter.WriteHeader(http.StatusOK) + } + return w.ResponseWriter.Write(b) +} + +// Handler returns the http.Handler for mounting onto a chi router. +func (s *Server) Handler() http.Handler { + return s.handler +} diff --git a/internal/mcp/photo.go b/internal/mcp/photo.go new file mode 100644 index 00000000..ed996de7 --- /dev/null +++ b/internal/mcp/photo.go @@ -0,0 +1,551 @@ +package mcp + +import ( + "bytes" + "context" + "fmt" + goimage "image" + "image/color" + "image/draw" + "image/png" + "os" + "path/filepath" + "runtime" + "strings" + + "sync" + + "github.com/modelcontextprotocol/go-sdk/mcp" + "github.com/tdewolff/canvas" + "github.com/tdewolff/canvas/rasterizer" + + "photofield/internal/codec" + jpegcodec "photofield/internal/codec/jpeg" + webpjack "photofield/internal/codec/webp/jack" + webpjackdyn "photofield/internal/codec/webp/jack/dynamic" + webpjacktra "photofield/internal/codec/webp/jack/transpiled" + "photofield/internal/image" + "photofield/internal/io" + "photofield/internal/render" +) + +// imagePoolKey is a key for the image pool cache. +type imagePoolKey struct { + Width int + Height int + Mem codec.ImageMem +} + +// tilePools caches image pools by dimensions to avoid recreating them. +var tilePools sync.Map + +// getPool creates or retrieves an image pool for the given dimensions. +func getPool(config *render.Render) *sync.Pool { + key := imagePoolKey{ + Width: config.ImageWidth, + Height: config.ImageHeight, + Mem: config.ImageMem, + } + if pool, ok := tilePools.Load(key); ok { + return pool.(*sync.Pool) + } + + pool := &sync.Pool{} + switch config.ImageMem { + case codec.ImageMemNRGBA: + pool.New = func() interface{} { + return goimage.NewNRGBA(goimage.Rect(0, 0, config.ImageWidth, config.ImageHeight)) + } + case codec.ImageMemPaletted: + pool.New = func() interface{} { + return goimage.NewPaletted( + goimage.Rect(0, 0, config.ImageWidth, config.ImageHeight), + color.Palette{ + color.RGBA{0x00, 0x00, 0x00, 0x00}, + color.RGBA{0xFF, 0xFF, 0xFF, 0xFF}, + }, + ) + } + default: + pool.New = func() interface{} { + return goimage.NewRGBA(goimage.Rect(0, 0, config.ImageWidth, config.ImageHeight)) + } + } + stored, _ := tilePools.LoadOrStore(key, pool) + return stored.(*sync.Pool) +} + +// getPoolImage gets an image from the pool and creates a canvas context. +func getPoolImage(config *render.Render) (draw.Image, *canvas.Context) { + pool := getPool(config) + img := pool.Get().(draw.Image) + r := rasterizer.New(img, 1.0) + c := canvas.NewContext(r) + c.SetView(canvas.Identity) + return img, c +} + +// putPoolImage returns an image to the pool. +func putPoolImage(config *render.Render, img draw.Image) { + pool := getPool(config) + pool.Put(img) +} + +// getPhotoInput contains the parameters for the get_photo MCP tool. +type getPhotoInput struct { + FileId int `json:"file_id" jsonschema:"The photo file ID to retrieve"` + + // Dimensions - if omitted, defaults to thumbnail (256x256 max) + TargetW *int `json:"w" jsonschema:"Target width in pixels (1-4096). Omitted = auto thumbnail"` + TargetH *int `json:"h" jsonschema:"Target height in pixels (1-4096). Omitted = auto thumbnail"` + + // Format - if omitted, defaults to JPEG + Format *string `json:"format" jsonschema:"Output format: jpeg, png, or webp (default jpeg)"` + + // Cropping - all in original image pixel coordinates + CropX *int `json:"crop_x" jsonschema:"Crop left edge in original image pixels"` + CropY *int `json:"crop_y" jsonschema:"Crop top edge in original image pixels"` + CropW *int `json:"crop_w" jsonschema:"Crop width in original image pixels"` + CropH *int `json:"crop_h" jsonschema:"Crop height in original image pixels"` +} + +// getPhotoMetadataInput contains the parameters for the get_photo_metadata MCP tool. +type getPhotoMetadataInput struct { + FileId int `json:"file_id" jsonschema:"The photo file ID to retrieve metadata for"` +} + +// getPhotoOutput is the empty output type for get_photo — this tool returns only +// the image (as MCP ImageContent), no structured metadata. Metadata is available +// via the separate get_photo_metadata tool. +type getPhotoOutput struct{} + +// getPhotoMetadataOutput contains the structured metadata for the get_photo_metadata MCP tool. +type getPhotoMetadataOutput struct { + Width int `json:"width"` // original image width in pixels + Height int `json:"height"` // original image height in pixels + Path string `json:"path"` // original file path + Video bool `json:"video,omitempty"` // true if the file is a video + CreatedAt string `json:"created_at"` // file creation time in RFC 3339 + Tags []SimpleTag `json:"tags,omitempty"` + Faces []FaceInfo `json:"faces,omitempty"` + Location string `json:"location,omitempty"` // reverse-geocoded location + LatLng *LatLng `json:"latlng,omitempty"` // GPS coordinates + PreviewUrl string `json:"preview_url"` // absolute URL to a ~400px wide preview image (for direct markdown embedding: ![name](preview_url)) + OriginalUrl string `json:"original_url"` // absolute URL to the original image (full-resolution variant) +} + +// FaceInfo represents detected face data for a photo. +type FaceInfo struct { + Id int `json:"id"` + X int `json:"x"` + Y int `json:"y"` + W int `json:"w"` + H int `json:"h"` + Confidence int `json:"confidence"` + PreviewUrl string `json:"preview_url,omitempty"` // absolute URL to cropped face preview image +} + +// LatLng holds GPS coordinates. +type LatLng struct { + Lat float64 `json:"lat"` + Lng float64 `json:"lng"` +} + +// SimpleTag represents a tag with a string ID. +type SimpleTag struct { + Id string `json:"id"` + Name string `json:"name"` + FileCount int `json:"file_count"` +} + +// getPhotoMetadataHandler handles the get_photo_metadata tool request. +// Returns all photo metadata without the image data — useful for inspecting +// tags, faces, location, thumbnails, and dimensions without downloading the image. +func getPhotoMetadataHandler(imageSource *image.Source, srv *Server) mcp.ToolHandlerFor[getPhotoMetadataInput, getPhotoMetadataOutput] { + return func(ctx context.Context, _ *mcp.CallToolRequest, input getPhotoMetadataInput) (*mcp.CallToolResult, getPhotoMetadataOutput, error) { + // Ensure we have a valid context - fall back to Background if nil. + if ctx == nil { + ctx = context.Background() + } + // Get file info to validate existence + info := imageSource.GetInfo(image.ImageId(input.FileId)) + if info.Width == 0 || info.Height == 0 { + return nil, getPhotoMetadataOutput{}, fmt.Errorf("file not found: %d", input.FileId) + } + + // Gather metadata using the same logic as get_photo + var metadata photoMetadata + var metaErr error + func() { + defer func() { + if r := recover(); r != nil { + metaErr = fmt.Errorf("internal error reading photo metadata: %v", r) + fmt.Fprintf(os.Stderr, "get_photo_metadata handler recovered from panic: %v\n%s", r, stackTrace()) + } + }() + metadata = gatherPhotoMetadata(ctx, imageSource, input.FileId, info, srv.baseURL.Load().(string), srv.apiPrefix) + }() + if metaErr != nil { + return nil, getPhotoMetadataOutput{}, metaErr + } + + // Return only structured metadata — no image content block. + // Leave Content nil so the SDK auto-populates it with JSON text + // from StructuredContent (required for MCP clients that only read content). + return nil, getPhotoMetadataOutput{ + PreviewUrl: metadata.PreviewUrl, + OriginalUrl: metadata.OriginalUrl, + Width: info.Width, + Height: info.Height, + Path: metadata.Path, + Video: metadata.Video, + CreatedAt: metadata.CreatedAt, + Tags: metadata.Tags, + Faces: metadata.Faces, + Location: metadata.Location, + LatLng: metadata.LatLng, + }, nil + } +} + +// getPhotoHandler handles the get_photo MCP tool request. +func getPhotoHandler(imageSource *image.Source, srv *Server) mcp.ToolHandlerFor[getPhotoInput, getPhotoOutput] { + return func(ctx context.Context, _ *mcp.CallToolRequest, input getPhotoInput) (*mcp.CallToolResult, getPhotoOutput, error) { + // Ensure we have a valid context - fall back to Background if nil. + if ctx == nil { + ctx = context.Background() + } + // Get file info to validate existence + info := imageSource.GetInfo(image.ImageId(input.FileId)) + if info.Width == 0 || info.Height == 0 { + return nil, getPhotoOutput{}, fmt.Errorf("file not found: %d", input.FileId) + } + + // Determine target dimensions + targetW, targetH := input.TargetW, input.TargetH + if targetW == nil || targetH == nil { + // Default to small thumbnail: 256x256 max (like thumbnail sources) + w := 256 + h := 256 + targetW = &w + targetH = &h + } + + // Parse and validate format + formatStr := "jpeg" + if input.Format != nil && *input.Format != "" { + formatStr = *input.Format + switch formatStr { + case "jpeg", "jpg": + formatStr = "jpeg" + case "png": + case "webp": + default: + return nil, getPhotoOutput{}, fmt.Errorf("unsupported format: %s (use jpeg, png, or webp)", *input.Format) + } + } + + // Encode image data + var imageData []byte + var encodeErr error + func() { + defer func() { + if r := recover(); r != nil { + encodeErr = fmt.Errorf("internal error rendering photo: %v", r) + fmt.Fprintf(os.Stderr, "get_photo handler recovered from panic: %v\n%s", r, stackTrace()) + } + }() + imageData, encodeErr = encodePhoto(ctx, imageSource, image.ImageId(input.FileId), *targetW, *targetH, formatStr, + input.CropX, input.CropY, input.CropW, input.CropH) + }() + if encodeErr != nil { + return nil, getPhotoOutput{}, encodeErr + } + + mime := "image/jpeg" + if formatStr == "png" { + mime = "image/png" + } else if formatStr == "webp" { + mime = "image/webp" + } + + // Return only the image as MCP ImageContent — no structured metadata. + // Use get_photo_metadata(file_id) to retrieve tags, faces, dimensions, URLs, etc. + // The SDK handles base64 encoding for the JSON wire format. + res := &mcp.CallToolResult{ + Content: []mcp.Content{ + &mcp.ImageContent{ + Data: imageData, + MIMEType: mime, + }, + }, + } + return res, getPhotoOutput{}, nil + } +} + +// encodePhoto renders and encodes a photo to the specified format. +func encodePhoto(ctx context.Context, source *image.Source, fileId image.ImageId, targetW, targetH int, format string, + cropX, cropY, cropW, cropH *int) ([]byte, error) { + + // Get file info + info := source.GetInfo(fileId) + if info.Width == 0 || info.Height == 0 { + return nil, fmt.Errorf("file not found: %d", fileId) + } + + // Validate dimensions + if targetW < 1 || targetW > 4096 || targetH < 1 || targetH > 4096 { + return nil, fmt.Errorf("dimensions %dx%d out of range (1-4096)", targetW, targetH) + } + + // Validate crop bounds + if cropW != nil && cropH != nil && *cropW > 0 && *cropH > 0 { + cx := 0 + cy := 0 + if cropX != nil { + cx = *cropX + } + if cropY != nil { + cy = *cropY + } + if cx < 0 || cy < 0 || cx+*cropW > info.Width || cy+*cropH > info.Height { + return nil, fmt.Errorf("crop bounds (%d,%d)+(%dx%d) exceed image size (%dx%d)", + cx, cy, *cropW, *cropH, info.Width, info.Height) + } + } + + // Create render config (similar to defaultSceneConfig.Render) + rn := render.Render{ + TileSize: 256, + ImageWidth: targetW, + ImageHeight: targetH, + MaxSolidPixelArea: 0, // Force full render, no solid color optimization + BackgroundColor: color.RGBA{0, 0, 0, 0}, + CoverFit: true, + ImageMem: codec.ImageMemRGBA, + QualityPreset: render.QualityPresetFast, + } + + // Get pooled image and canvas context + img, c := getPoolImage(&rn) + defer putPoolImage(&rn, img) + + rn.CanvasImage = img + + // Reset view and setup coordinates + c.ResetView() + c.SetView(canvas.Identity.Translate(0, float64(targetH))) + + // Draw background + draw.Draw(img, img.Bounds(), &goimage.Uniform{rn.BackgroundColor}, goimage.Point{}, draw.Src) + + // Setup photo with full content area bounds (no border) + photo := &render.Photo{ + Id: fileId, + } + photo.Sprite.Rect = render.Rect{ + X: 0, + Y: 0, + W: float64(targetW), + H: float64(targetH), + } + + // Build optional crop rect + var crop render.Rect + if cropW != nil && cropH != nil && *cropW > 0 && *cropH > 0 { + cx := 0 + cy := 0 + if cropX != nil { + cx = *cropX + } + if cropY != nil { + cy = *cropY + } + crop = render.Rect{ + X: float64(cx), + Y: float64(cy), + W: float64(*cropW), + H: float64(*cropH), + } + } + + // Draw photo using existing rendering logic + photo.Draw(ctx, &rn, nil, c, render.Scales{Tile: 1.0}, source, false, crop) + + // Encode to requested format + var buf bytes.Buffer + quality := 80 + if rn.QualityPreset == render.QualityPresetHigh { + quality = 100 + } + + switch format { + case "jpeg": + if err := jpegcodec.Encode(&buf, img, quality); err != nil { + return nil, fmt.Errorf("error encoding JPEG: %w", err) + } + case "png": + if err := png.Encode(&buf, img); err != nil { + return nil, fmt.Errorf("error encoding PNG: %w", err) + } + case "webp": + // WebP encoding - try the available encoders + encoders := []func(w *bytes.Buffer, img goimage.Image, quality int) error{ + func(w *bytes.Buffer, img goimage.Image, quality int) error { + return webpjack.Encode(w, img, quality) + }, + func(w *bytes.Buffer, img goimage.Image, quality int) error { + return webpjackdyn.Encode(w, img, quality) + }, + func(w *bytes.Buffer, img goimage.Image, quality int) error { + return webpjacktra.Encode(w, img, quality) + }, + } + encoded := false + for _, enc := range encoders { + err := enc(&buf, img, quality) + if err == nil { + encoded = true + break + } + buf.Reset() + } + if !encoded { + // Fallback to JPEG if no WebP encoder works + if err := jpegcodec.Encode(&buf, img, quality); err != nil { + return nil, fmt.Errorf("error encoding JPEG (WebP fallback): %w", err) + } + } + default: + return nil, fmt.Errorf("unsupported format: %s", format) + } + + return buf.Bytes(), nil +} + +// photoMetadata holds all metadata fields for a photo. +type photoMetadata struct { + Path string + Video bool + CreatedAt string + PreviewUrl string + OriginalUrl string + Tags []SimpleTag + Faces []FaceInfo + Location string + LatLng *LatLng +} + +// fileURL builds an absolute URL to a file endpoint, handling an empty or root apiPrefix. +func fileURL(serverBaseURL, apiPrefix, path string) string { + if apiPrefix == "" || apiPrefix == "/" { + return serverBaseURL + path + } + return serverBaseURL + apiPrefix + path +} + +// gatherPhotoMetadata collects all metadata for a photo by file ID. +// serverBaseURL is the absolute API base URL (e.g. "http://localhost:8080"). +// apiPrefix is the HTTP route prefix for file endpoints (e.g. "/api" or ""). +func gatherPhotoMetadata(ctx context.Context, source *image.Source, fileId int, info image.Info, serverBaseURL, apiPrefix string) photoMetadata { + originalPath, _ := source.GetImagePath(image.ImageId(fileId)) + location := "" + var latlng *LatLng + if image.IsValidLatLng(info.LatLng) { + latlng = &LatLng{ + Lat: info.LatLng.Lat.Degrees(), + Lng: info.LatLng.Lng.Degrees(), + } + if source.Geo != nil && source.Geo.Available() { + location, _ = source.Geo.ReverseGeocode(ctx, info.LatLng) + } + } + + isVideo := source.IsSupportedVideo(originalPath) + filename := filepath.Base(originalPath) + previewFilename := strings.TrimSuffix(filename, filepath.Ext(filename)) + "_preview.jpg" + + // Build preview URL: use /api/files/{id}/previews/{filename} with ~400px width for direct markdown embedding + var previewUrl string + if originalPath != "" { + previewUrl = fileURL(serverBaseURL, apiPrefix, "/files/"+fmt.Sprintf("%d", fileId)+"/previews/"+previewFilename+"?w=400") + } + + // Build original image URL: use the 'original' variant (full-resolution source copy) + var originalUrl string + for _, s := range source.Sources { + if s.Name() != "original" { + continue + } + if !s.Exists(ctx, io.ImageId(fileId), originalPath) { + continue + } + originalUrl = fileURL(serverBaseURL, apiPrefix, "/files/"+fmt.Sprintf("%d", fileId)+"/variants/"+s.Name()+"/"+filename) + break + } + + // Gather tags + tags := make([]SimpleTag, 0) + for t := range source.ListImageTags(image.ImageId(fileId)) { + tags = append(tags, SimpleTag{ + Id: t.Name, + Name: t.Name, + FileCount: t.FileCount, + }) + } + + // Gather face detections + faceInfos := source.GetFacesByFileId(image.ImageId(fileId)) + faces := make([]FaceInfo, 0, len(faceInfos)) + for _, f := range faceInfos { + // Build face preview URL: crop a square around the face + faceCropSize := f.W + if f.H > faceCropSize { + faceCropSize = f.H + } + // Center the crop on the face + cropX := f.X + (f.W-faceCropSize)/2 + if cropX < 0 { + cropX = 0 + } + cropY := f.Y + (f.H-faceCropSize)/2 + if cropY < 0 { + cropY = 0 + } + faceFilename := fmt.Sprintf("face_%d.jpg", f.Id) + faces = append(faces, FaceInfo{ + Id: f.Id, + X: f.X, + Y: f.Y, + W: f.W, + H: f.H, + Confidence: f.Confidence, + PreviewUrl: fileURL(serverBaseURL, apiPrefix, "/files/"+fmt.Sprintf("%d", fileId)+"/previews/"+faceFilename+"?w=200&h=200&crop_x="+fmt.Sprintf("%d", cropX)+"&crop_y="+fmt.Sprintf("%d", cropY)+"&crop_w="+fmt.Sprintf("%d", faceCropSize)+"&crop_h="+fmt.Sprintf("%d", faceCropSize)), + }) + } + + if originalUrl == "" { + originalUrl = fileURL(serverBaseURL, apiPrefix, "/files/"+fmt.Sprintf("%d", fileId)+"/variants/"+filename) + } + + return photoMetadata{ + Path: originalPath, + Video: isVideo, + CreatedAt: info.DateTime.Format("2006-01-02T15:04:05Z07:00"), + PreviewUrl: previewUrl, + OriginalUrl: originalUrl, + Tags: tags, + Faces: faces, + Location: location, + LatLng: latlng, + } +} + +// stackTrace returns a formatted stack trace for logging panics. +func stackTrace() string { + const maxStack = 32 + buf := make([]byte, 4096) + n := runtime.Stack(buf, false) + return string(buf[:n]) +} diff --git a/internal/openapi/api.gen.go b/internal/openapi/api.gen.go index 83e05e6e..a7b873f0 100644 --- a/internal/openapi/api.gen.go +++ b/internal/openapi/api.gen.go @@ -103,6 +103,21 @@ type DocsCapability struct { Url string `json:"url"` } +// EventSummary defines model for EventSummary. +type EventSummary struct { + CreatedAfter *time.Time `json:"created_after,omitempty"` + CreatedBefore *time.Time `json:"created_before,omitempty"` + Index *int `json:"index,omitempty"` + LocationCount *int `json:"location_count,omitempty"` + Locations *[]string `json:"locations,omitempty"` + PhotoCount *int `json:"photo_count,omitempty"` +} + +// EventsList defines model for EventsList. +type EventsList struct { + Items *[]EventSummary `json:"items,omitempty"` +} + // A validated and typed search query expression, types omitted as this is subject to many changes. type Expression map[string]interface{} @@ -112,6 +127,24 @@ type FileBinary string // FileId defines model for FileId. type FileId int +// FileInfo defines model for FileInfo. +type FileInfo struct { + Color *string `json:"color,omitempty"` + Datetime *time.Time `json:"datetime,omitempty"` + FileName *string `json:"file_name,omitempty"` + Height *int `json:"height,omitempty"` + Id *int `json:"id,omitempty"` + Location *string `json:"location,omitempty"` + Similarity *float32 `json:"similarity,omitempty"` + Tags *[]string `json:"tags,omitempty"` + Width *int `json:"width,omitempty"` +} + +// FileList defines model for FileList. +type FileList struct { + Items *[]FileInfo `json:"items,omitempty"` +} + // GeoJSON FeatureCollection type GeoJSON struct { // Array of GeoJSON features representing photos in the scene @@ -352,6 +385,18 @@ type TagIdPathParam TagId // TaskIdPathParam defines model for TaskIdPathParam. type TaskIdPathParam TaskId +// GetCollectionsIdFilesParams defines parameters for GetCollectionsIdFiles. +type GetCollectionsIdFilesParams struct { + // Natural language or structured search query + Search *Search `json:"search,omitempty"` + + // Sort order. Prefix with `-` for descending or `+` for ascending, e.g. `-date` (newest first), `+date` (oldest first), `-similarity` (best matches first). Multiple values allowed. + Sort *Sort `json:"sort,omitempty"` + + // Maximum number of results + Limit *Limit `json:"limit,omitempty"` +} + // GetFilesIdPreviewsFilenameParams defines parameters for GetFilesIdPreviewsFilename. type GetFilesIdPreviewsFilenameParams struct { // Target width in pixels. If omitted, uses original width or scales proportionally with height. @@ -522,6 +567,12 @@ type ServerInterface interface { // (GET /collections/{id}) GetCollectionsId(w http.ResponseWriter, r *http.Request, id CollectionId) + // (GET /collections/{id}/events) + GetCollectionsIdEvents(w http.ResponseWriter, r *http.Request, id CollectionId) + + // (GET /collections/{id}/files) + GetCollectionsIdFiles(w http.ResponseWriter, r *http.Request, id CollectionId, params GetCollectionsIdFilesParams) + // (GET /files/{id}) GetFilesId(w http.ResponseWriter, r *http.Request, id FileIdPathParam) @@ -650,6 +701,94 @@ func (siw *ServerInterfaceWrapper) GetCollectionsId(w http.ResponseWriter, r *ht handler(w, r.WithContext(ctx)) } +// GetCollectionsIdEvents operation middleware +func (siw *ServerInterfaceWrapper) GetCollectionsIdEvents(w http.ResponseWriter, r *http.Request) { + ctx := r.Context() + + var err error + + // ------------- Path parameter "id" ------------- + var id CollectionId + + err = runtime.BindStyledParameter("simple", false, "id", chi.URLParam(r, "id"), &id) + if err != nil { + http.Error(w, fmt.Sprintf("Invalid format for parameter id: %s", err), http.StatusBadRequest) + return + } + + var handler = func(w http.ResponseWriter, r *http.Request) { + siw.Handler.GetCollectionsIdEvents(w, r, id) + } + + for _, middleware := range siw.HandlerMiddlewares { + handler = middleware(handler) + } + + handler(w, r.WithContext(ctx)) +} + +// GetCollectionsIdFiles operation middleware +func (siw *ServerInterfaceWrapper) GetCollectionsIdFiles(w http.ResponseWriter, r *http.Request) { + ctx := r.Context() + + var err error + + // ------------- Path parameter "id" ------------- + var id CollectionId + + err = runtime.BindStyledParameter("simple", false, "id", chi.URLParam(r, "id"), &id) + if err != nil { + http.Error(w, fmt.Sprintf("Invalid format for parameter id: %s", err), http.StatusBadRequest) + return + } + + // Parameter object where we will unmarshal all parameters from the context + var params GetCollectionsIdFilesParams + + // ------------- Optional query parameter "search" ------------- + if paramValue := r.URL.Query().Get("search"); paramValue != "" { + + } + + err = runtime.BindQueryParameter("form", true, false, "search", r.URL.Query(), ¶ms.Search) + if err != nil { + http.Error(w, fmt.Sprintf("Invalid format for parameter search: %s", err), http.StatusBadRequest) + return + } + + // ------------- Optional query parameter "sort" ------------- + if paramValue := r.URL.Query().Get("sort"); paramValue != "" { + + } + + err = runtime.BindQueryParameter("form", true, false, "sort", r.URL.Query(), ¶ms.Sort) + if err != nil { + http.Error(w, fmt.Sprintf("Invalid format for parameter sort: %s", err), http.StatusBadRequest) + return + } + + // ------------- Optional query parameter "limit" ------------- + if paramValue := r.URL.Query().Get("limit"); paramValue != "" { + + } + + err = runtime.BindQueryParameter("form", true, false, "limit", r.URL.Query(), ¶ms.Limit) + if err != nil { + http.Error(w, fmt.Sprintf("Invalid format for parameter limit: %s", err), http.StatusBadRequest) + return + } + + var handler = func(w http.ResponseWriter, r *http.Request) { + siw.Handler.GetCollectionsIdFiles(w, r, id, params) + } + + for _, middleware := range siw.HandlerMiddlewares { + handler = middleware(handler) + } + + handler(w, r.WithContext(ctx)) +} + // GetFilesId operation middleware func (siw *ServerInterfaceWrapper) GetFilesId(w http.ResponseWriter, r *http.Request) { ctx := r.Context() @@ -1790,6 +1929,12 @@ func HandlerWithOptions(si ServerInterface, options ChiServerOptions) http.Handl r.Group(func(r chi.Router) { r.Get(options.BaseURL+"/collections/{id}", wrapper.GetCollectionsId) }) + r.Group(func(r chi.Router) { + r.Get(options.BaseURL+"/collections/{id}/events", wrapper.GetCollectionsIdEvents) + }) + r.Group(func(r chi.Router) { + r.Get(options.BaseURL+"/collections/{id}/files", wrapper.GetCollectionsIdFiles) + }) r.Group(func(r chi.Router) { r.Get(options.BaseURL+"/files/{id}", wrapper.GetFilesId) }) diff --git a/main.go b/main.go index db95a1ae..dfd75a7e 100644 --- a/main.go +++ b/main.go @@ -62,6 +62,7 @@ import ( pfio "photofield/internal/io" "photofield/internal/io/bench" "photofield/internal/layout" + "photofield/internal/mcp" "photofield/internal/metrics" "photofield/internal/openapi" "photofield/internal/render" @@ -267,6 +268,23 @@ func getCollectionById(id string) *collection.Collection { return nil } +func ptr[T any](v T) *T { return &v } + +func parseTime(s string) *time.Time { + t, err := time.Parse(time.RFC3339, s) + if err != nil { + return nil + } + return &t +} + +func ptrToStrSlice(s []string) *[]string { + if len(s) == 0 { + return nil + } + return &s +} + func pushApiRequest(request ApiRequest) { requestsMutex.Lock() requests = append(requests, request) @@ -505,7 +523,98 @@ func (*Api) GetCollectionsId(w http.ResponseWriter, r *http.Request, id openapi. } } - problem(w, r, http.StatusNotFound, "Scene not found") + problem(w, r, http.StatusNotFound, "Collection not found") +} + +func (*Api) GetCollectionsIdEvents(w http.ResponseWriter, r *http.Request, id openapi.CollectionId) { + coll := getCollectionById(string(id)) + if coll == nil { + problem(w, r, http.StatusBadRequest, "Collection not found") + return + } + + items, err := coll.SplitIntoEvents(r.Context(), imageSource) + if err != nil { + problem(w, r, http.StatusInternalServerError, err.Error()) + return + } + + apiItems := make([]openapi.EventSummary, len(items)) + for i, e := range items { + apiItems[i] = openapi.EventSummary{ + Index: ptr(e.Index), + CreatedAfter: parseTime(e.CreatedAfter), + CreatedBefore: parseTime(e.CreatedBefore), + PhotoCount: ptr(e.PhotoCount), + LocationCount: ptr(e.LocationCount), + Locations: ptrToStrSlice(e.Locations), + } + } + + respond(w, r, http.StatusOK, openapi.EventsList{Items: &apiItems}) +} + +func (*Api) GetCollectionsIdFiles(w http.ResponseWriter, r *http.Request, id openapi.CollectionId, params openapi.GetCollectionsIdFilesParams) { + coll := getCollectionById(string(id)) + if coll == nil { + problem(w, r, http.StatusBadRequest, "Collection not found") + return + } + + limit := 50 + if params.Limit != nil && int(*params.Limit) > 0 { + limit = int(*params.Limit) + } + + opts := collection.SearchOptions{ + Limit: limit, + } + if params.Search != nil { + opts.QueryStr = string(*params.Search) + } + if params.Sort != nil { + opts.Sort = collection.SortType(string(*params.Sort)) + } + + items, _, _, err := coll.Search(r.Context(), imageSource, opts) + if err != nil { + problem(w, r, http.StatusInternalServerError, err.Error()) + return + } + + if items == nil { + items = make([]collection.SearchResult, 0) + } + + apiItems := make([]openapi.FileInfo, len(items)) + for i, item := range items { + apiItems[i] = openapi.FileInfo{ + Id: ptr(int(item.Id)), + FileName: &item.FileName, + Similarity: ptr(item.Similarity), + } + if item.DateTime != "" { + t := parseTime(item.DateTime) + apiItems[i].Datetime = t + } + if item.Width != 0 { + apiItems[i].Width = ptr(item.Width) + } + if item.Height != 0 { + apiItems[i].Height = ptr(item.Height) + } + if item.Color != "" { + apiItems[i].Color = &item.Color + } + if item.Location != "" { + apiItems[i].Location = &item.Location + } + if item.Tags != nil { + apiItems[i].Tags = ptrToStrSlice(item.Tags) + } + } + + respond(w, r, http.StatusOK, openapi.FileList{Items: &apiItems}) } func taskDisplayOrder(taskType string) int { @@ -1507,8 +1616,18 @@ func (*Api) GetFilesIdPreviewsFilename(w http.ResponseWriter, r *http.Request, i return } - img, c := getPoolImage(&rn) - defer putPoolImage(&rn, img) + var img draw.Image + var c *canvas.Context + if params.W != nil && params.H != nil && rn.ImageWidth == rn.ImageHeight { + img, c = getPoolImage(&rn) + defer putPoolImage(&rn, img) + } else { + img = goimage.NewRGBA( + goimage.Rect(0, 0, rn.ImageWidth, rn.ImageHeight), + ) + renderer := rasterizer.New(img, 1.0) + c = canvas.NewContext(renderer) + } rn.CanvasImage = img rn.MaxSolidPixelArea = 0 // Force full render, no solid color optimization @@ -1624,9 +1743,56 @@ func parsePreviewDimensions(origW, origH int, reqW, reqH *int) (w, h int, err er w, h = origW, origH } + // Clamp to maximum allowed dimension (prevents DoS via huge allocations) + // Clamp the source dimension first, then the derived dimension, and + // re-balance aspect ratio if needed. + const maxPreviewDim = 4096 + if reqW != nil && reqH != nil { + // Both specified — clamp independently (no aspect ratio to preserve) + if w > maxPreviewDim { + w = maxPreviewDim + } + if h > maxPreviewDim { + h = maxPreviewDim + } + } else if reqW != nil { + // Only width — clamp it, then derive height; re-balance if height exceeds + if w > maxPreviewDim { + w = maxPreviewDim + } + h = int(float64(origH) * float64(w) / float64(origW)) + if h < 1 { + h = 1 + } + if h > maxPreviewDim { + h = maxPreviewDim + w = int(float64(origW) * float64(h) / float64(origH)) + } + } else if reqH != nil { + // Only height — clamp it, then derive width; re-balance if width exceeds + if h > maxPreviewDim { + h = maxPreviewDim + } + w = int(float64(origW) * float64(h) / float64(origH)) + if w < 1 { + w = 1 + } + if w > maxPreviewDim { + w = maxPreviewDim + h = int(float64(origH) * float64(w) / float64(origW)) + } + } else { + // Neither — use original dimensions; scale proportionally if either exceeds + if w > maxPreviewDim || h > maxPreviewDim { + scale := float64(maxPreviewDim) / float64(max(w, h)) + w = int(float64(w) * scale) + h = int(float64(h) * scale) + } + } + // Validate - if w < 1 || w > 4096 || h < 1 || h > 4096 { - return 0, 0, fmt.Errorf("dimensions %dx%d out of range (1-4096)", w, h) + if w < 1 || h < 1 { + return 0, 0, fmt.Errorf("invalid dimensions: width and height must be positive") } return w, h, nil @@ -2331,11 +2497,29 @@ func main() { var api Api r.Mount("/", openapi.Handler(&api)) r.Mount("/metrics", promhttp.Handler()) + r.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + fmt.Fprintf(w, `{"status":"ok"}`) + }) }) r.Mount("/debug", middleware.Profiler()) r.Handle("/debug/fgprof", fgprof.Handler()) + // MCP server — base URL is derived from request Host header at runtime, + // falling back to the listener address if the Host header is absent. + srv, err := mcp.New(&collections, imageSource, addr, apiPrefix) + if err != nil { + log.Fatalf("failed to create MCP server: %v", err) + } + mcpPrefix := os.Getenv("PHOTOFIELD_MCP_PREFIX") + if mcpPrefix == "" { + mcpPrefix = "/mcp" + } + r.Mount(mcpPrefix, srv.Handler()) + log.Printf("MCP server mounted at %s", mcpPrefix) + msg := "" if apiPrefix != "/" { // Hardcode well-known mime types, see https://github.com/golang/go/issues/32350 diff --git a/tools/agent.sh b/tools/agent.sh new file mode 100755 index 00000000..bc7eca59 --- /dev/null +++ b/tools/agent.sh @@ -0,0 +1,658 @@ +#!/bin/bash +# agent.sh — Unified harness for testing the photofield server +# +# Covers server lifecycle, generic HTTP API calls, and tool calls. +# +# USAGE: +# agent.sh --help Print this help +# agent.sh --verbose Verbose output (must precede subcommand; env override: AGT_VERBOSE=1) +# +# agent.sh server start Start server (auto-detect / launch) +# agent.sh server stop Stop via PID file +# agent.sh server restart Stop + start +# agent.sh server status Check if running +# agent.sh server kill Aggressive pkill (photofield + exiftool) +# +# agent.sh api [body] Generic HTTP call +# agent.sh api --key val Named-arg body +# +# agent.sh mcp call Tool call (JSON or named args) +# agent.sh mcp call --key val Tool call (named args) +# agent.sh mcp quick [tool args] Smoke test +# agent.sh mcp shell Interactive REPL +# +# ENV: +# AGT_PORT — Server port (default: 8080) +# AGT_BIN — Path to photofield binary +# AGT_DATA_DIR — Path to data directory +# AGT_START — Auto-start server (default: true) +# AGT_URL — Full endpoint URL (overrides PORT) +# AGT_API_BASE — API base URL (default: http://localhost:$PORT) + +set -uo pipefail + +# ─── Config ─── +PORT="${AGT_PORT:-8080}" +API_BASE="${AGT_API_BASE:-http://localhost:${PORT}}" +ENDPOINT_URL="${AGT_URL:-${API_BASE}/mcp}" +BIN="${AGT_BIN:-$(cd "$(dirname "$0")/.." && pwd)/photofield}" +DATA_DIR="${AGT_DATA_DIR:-$(pwd)/data}" +AUTO_START="${AGT_START:-true}" +VERBOSE=${AGT_VERBOSE:-0} +_SERVER_MANAGED=false + +# ─── Paths ─── +_pid_file="${DATA_DIR}/agent.pid" +_headers_file="${DATA_DIR}/agent-headers-$$" + +# ─── Colors ─── +if [[ -t 1 ]]; then + COL_GREEN=$'\033[0;32m'; COL_RED=$'\033[0;31m'; COL_CYAN=$'\033[0;36m' + COL_BOLD=$'\033[1m'; COL_RESET=$'\033[0m' +else + COL_GREEN=''; COL_RED=''; COL_CYAN=''; COL_BOLD=''; COL_RESET='' +fi + +log_ok() { printf '%s\n' "${COL_GREEN}✓${COL_RESET} $*" >&2; } +log_fail() { printf '%s\n' "${COL_RED}✗${COL_RESET} $*" >&2; } +log_info() { printf '%s\n' "${COL_CYAN}ℹ${COL_RESET} $*" >&2; } +log_step() { printf '%s\n' "${COL_CYAN}▶${COL_RESET} $*" >&2; } + +# ─── Server Management ─── +server_is_running() { + # Check PID file first + if [[ -f "$_pid_file" ]]; then + local pid + pid=$(cat "$_pid_file" 2>/dev/null) + if [[ -n "$pid" ]] && kill -0 "$pid" 2>/dev/null; then + return 0 + fi + rm -f "$_pid_file" + fi + # Fall back to port check + curl -s --max-time 1 "${ENDPOINT_URL}" &>/dev/null +} + +server_start() { + if server_is_running; then + log_info "Server already running on port ${PORT}" + _SERVER_MANAGED=false + return 0 + fi + + if [[ ! -x "$BIN" ]]; then + log_fail "Server binary not found: ${BIN}" + log_info "Set AGT_BIN=/path/to/photofield to override" + return 1 + fi + + log_step "Starting server..." + mkdir -p "$DATA_DIR" + export PHOTOFIELD_ADDRESS=":$(echo "$PORT" | sed 's/.*://')" + export PHOTOFIELD_DATA_DIR="$DATA_DIR" + nohup "$BIN" > "${DATA_DIR}/agent.log" 2>&1 & + _SERVER_MANAGED=true + local pid=$! + printf '%s\n' "$pid" > "$_pid_file" + log_info "PID: ${pid} (log: ${DATA_DIR}/agent.log)" + + local waited=0 + while (( waited < 30 )); do + if curl -s --max-time 2 "${ENDPOINT_URL}" &>/dev/null; then + log_ok "Server is ready" + return 0 + fi + sleep 0.5 + waited=$((waited + 1)) + done + + log_fail "Server failed to start within 30s" + tail -20 "${DATA_DIR}/agent.log" >&2 + return 1 +} + +server_stop() { + if [[ -f "$_pid_file" ]]; then + local pid + pid=$(cat "$_pid_file" 2>/dev/null) + if [[ -n "$pid" ]] && kill -0 "$pid" 2>/dev/null; then + log_info "Stopping server (PID ${pid})..." + kill "$pid" 2>/dev/null || true + # Wait for it to die + local w=0 + while (( w < 10 )); do + kill -0 "$pid" 2>/dev/null || break + sleep 0.3 + w=$((w + 1)) + done + kill -0 "$pid" 2>/dev/null && kill -9 "$pid" 2>/dev/null || true + log_ok "Server stopped" + fi + rm -f "$_pid_file" + else + log_info "No PID file found" + # Port may still be in use + if server_is_running; then + log_info "Server running but no PID — use 'server kill' for aggressive stop" + return 1 + fi + log_ok "Nothing to stop" + fi +} + +server_restart() { + server_stop + sleep 1 + server_start +} + +server_status() { + if server_is_running; then + local pid="" + if [[ -f "$_pid_file" ]]; then + pid=$(cat "$_pid_file" 2>/dev/null) + fi + log_ok "Server running on port ${PORT}" + [[ -n "$pid" ]] && log_info "PID: ${pid}" + # Check which processes are using the port + local procs + procs=$(ss -tlnp "sport = :${PORT}" 2>/dev/null | grep -oP 'pid=\K[0-9]+' | head -5 | tr '\n' ',' | sed 's/,$//') + [[ -n "$procs" ]] && log_info "Port listeners: ${procs}" + return 0 + else + log_info "Server not running on port ${PORT}" + return 1 + fi +} + +server_kill() { + log_step "Killing server processes on port ${PORT}..." + local count=0 + # Kill by PID file first (most targeted) + if [[ -f "$_pid_file" ]]; then + local pid + pid=$(cat "$_pid_file" 2>/dev/null) + if [[ -n "$pid" ]]; then + kill -9 "$pid" 2>/dev/null || true + count=$((count + 1)) + fi + rm -f "$_pid_file" + fi + # Kill anything listening on our port (catches server children, exiftool, etc.) + local p + for p in $(ss -tlnp "sport = :${PORT}" 2>/dev/null | grep -oP 'pid=\K[0-9]+' || true); do + kill -9 "$p" 2>/dev/null || true + count=$((count + 1)) + done + sleep 0.3 + # Double-check (safety net) + for p in $(ss -tlnp "sport = :${PORT}" 2>/dev/null | grep -oP 'pid=\K[0-9]+' || true); do + kill -9 "$p" 2>/dev/null || true + count=$((count + 1)) + done + log_ok "Killed ${count} process(es) on port ${PORT}" +} + +# ─── Generic API Calls ─── +api_call() { + local method="${1^^}" # uppercase + local url="$2" + shift 2 + + local body="" hdrs=(-X "$method") + local named_mode=false + + # Check if remaining args look like named --key val pairs + if [[ $# -gt 0 && "$1" == "--" ]]; then + shift + named_mode=true + elif [[ $# -gt 0 && "$1" =~ ^--[a-zA-Z] ]]; then + named_mode=true + fi + + if [[ "$named_mode" == "true" ]]; then + body=$(build_args "$@") + elif [[ $# -gt 0 ]]; then + body="$*" + fi + + [[ "$method" == "GET" || "$method" == "HEAD" ]] && body="" + + log_info "→ ${method} ${url}" + [[ -n "$body" ]] && log_info "Body: ${body:0:200}" + + local status_code tmpfile + tmpfile=$(mktemp "${DATA_DIR}/agent-raw-XXXXXX") + status_code=$(curl -s -o "$tmpfile" -w "%{http_code}" \ + "${hdrs[@]}" \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -d "$body" \ + "$url" 2>/dev/null) + + local raw + raw=$(cat "$tmpfile"; rm -f "$tmpfile") + + log_info "HTTP ${status_code}" + + # Try to pretty-print as JSON + local pretty + pretty=$(echo "$raw" | jq '.' 2>/dev/null) + if [[ -n "$pretty" ]]; then + echo "$pretty" + else + if [[ ${#raw} -lt 500 ]]; then + echo "$raw" + else + printf '%s\n' "${raw:0:500}... [${#raw} chars total]" + fi + fi +} + +# ─── Session & Requests ─── +_SESSION_ID="" +_REQUEST_ID=0 + +session_init() { + local resp + resp=$(curl -s -D "$_headers_file" \ + -X POST "${ENDPOINT_URL}" \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{ + "protocolVersion":"2024-11-05", + "capabilities":{}, + "clientInfo":{"name":"agent","version":"0.1"} + }}' 2>/dev/null) + + _SESSION_ID=$(grep -i "Mcp-Session-Id" "$_headers_file" 2>/dev/null | head -1 | tr -d '\r' | sed 's/.*[Mm]cp-[Ss]ession-[Ii]d:[[:space:]]*//') + rm -f "$_headers_file" + + if [[ -z "$_SESSION_ID" ]]; then + log_info "No session ID (server may not require one)" + else + log_info "Session: ${_SESSION_ID}" + fi + + # Send initialized notification + local hdrs=(-H "Content-Type: application/json") + [[ -n "$_SESSION_ID" ]] && hdrs+=(-H "Mcp-Session-Id: ${_SESSION_ID}") + curl -s -X POST "${ENDPOINT_URL}" \ + "${hdrs[@]}" \ + -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' 2>/dev/null || true +} + +# Parse SSE response: extract JSON from "data: {...}" lines +_sse_parse() { + local raw="$1" + echo "$raw" | sed -n 's/^data: //p' | tail -1 +} + +# Call a tool, return cleaned JSON on stdout +call_tool() { + local tool="$1" + shift + local args="$*" + [[ -z "$args" ]] && args="{}" + _REQUEST_ID=$((_REQUEST_ID + 1)) + + local hdrs=(-H "Content-Type: application/json" -H "Accept: application/json, text/event-stream") + [[ -n "$_SESSION_ID" ]] && hdrs+=(-H "Mcp-Session-Id: ${_SESSION_ID}") + + local raw + raw=$(curl -s --max-time 30 -X POST "${ENDPOINT_URL}" \ + "${hdrs[@]}" \ + -d "{\"jsonrpc\":\"2.0\",\"id\":${_REQUEST_ID},\"method\":\"tools/call\",\"params\":{\"name\":\"${tool}\",\"arguments\":${args}}}" \ + 2>/dev/null) + + _sse_parse "$raw" +} + +# Build JSON args from --key val pairs +build_args() { + local result="{" + local first=true + while [[ $# -gt 0 ]]; do + local key="$1" val="$2" + shift 2 + key="${key#--}" + [[ "$first" != "true" ]] && result+="," + first=false + if [[ "$val" =~ ^[0-9]+(\.[0-9]+)?$ ]]; then + result+="\"${key}\":${val}" + elif [[ "$val" == "true" || "$val" == "false" ]]; then + result+="\"${key}\":${val}" + elif [[ "$val" == "null" ]]; then + result+="\"${key}\":null" + else + val=$(printf '%s' "$val" | sed 's/\\/\\\\/g; s/"/\\"/g; s/\t/\\t/g') + result+="\"${key}\":\"${val}\"" + fi + done + result+="}" + echo "$result" +} + +# ─── Output formatting ─── +print_result() { + local tool="$1" resp="$2" + + # Check for JSON-RPC error field + local rpc_error + rpc_error=$(echo "$resp" | jq -r '.error.message // empty' 2>/dev/null) + if [[ -n "$rpc_error" ]]; then + printf '%s\n' "${COL_RED}✗${COL_RESET} ${COL_BOLD}${tool}${COL_RESET} — ${rpc_error}" + [[ "$VERBOSE" == "1" ]] && echo "$resp" | jq '.' 2>/dev/null + return 1 + fi + + # Check for result.isError (tools can return errors in result) + local is_error + is_error=$(echo "$resp" | jq -r '.result.isError // false' 2>/dev/null) + + # Check for content in result + local content_text + content_text=$(echo "$resp" | jq -r '.result.content[0].text // empty' 2>/dev/null) + + if [[ -n "$content_text" ]]; then + if [[ "$is_error" == "true" ]]; then + printf '%s\n' "${COL_RED}✗${COL_RESET} ${COL_BOLD}${tool}${COL_RESET} — ${content_text}" + [[ "$VERBOSE" == "1" ]] && echo "$resp" | jq '.' 2>/dev/null + return 1 + fi + + if [[ "$VERBOSE" == "1" ]]; then + local inner_json + inner_json=$(echo "$content_text" | jq '.' 2>/dev/null) + if [[ -n "$inner_json" ]]; then + echo "$inner_json" + else + echo "$content_text" + fi + return 0 + fi + + # Normal response — try to parse content as JSON for a clean summary + local inner_json + inner_json=$(echo "$content_text" | jq '.' 2>/dev/null) + if [[ -n "$inner_json" ]]; then + # Find top-level array keys and report their counts + local summary_label + summary_label=$(echo "$inner_json" | jq -r ' + [ to_entries[] | select(.value | type == "array") ] | + if length > 0 then + (map("\(.key): \(.value | length)") | join(", ")) + else + "ok" + end + ' 2>/dev/null) + printf '%s\n' "${COL_GREEN}✓${COL_RESET} ${COL_BOLD}${tool}${COL_RESET} — ${summary_label}" + echo "$inner_json" | jq '.' 2>/dev/null || echo "$inner_json" + else + # Plain text response + if [[ ${#content_text} -lt 500 ]]; then + printf '%s\n' "${COL_GREEN}✓${COL_RESET} ${COL_BOLD}${tool}${COL_RESET}" + echo "$content_text" + else + printf '%s\n' "${COL_GREEN}✓${COL_RESET} ${COL_BOLD}${tool}${COL_RESET} — ${#content_text} chars" + echo "$content_text" | head -c 500 + echo "..." + fi + fi + return 0 + fi + + # Check for structured content + local has_structured + has_structured=$(echo "$resp" | jq -e '.result.structuredContent' &>/dev/null && echo yes || echo no) + if [[ "$has_structured" == "yes" ]]; then + local sc_json + sc_json=$(echo "$resp" | jq -r '.result.structuredContent' 2>/dev/null) + if [[ -n "$sc_json" ]]; then + printf '%s\n' "${COL_GREEN}✓${COL_RESET} ${COL_BOLD}${tool}${COL_RESET}" + echo "$sc_json" | jq '.' 2>/dev/null || echo "$sc_json" + return 0 + fi + fi + + # No result at all + printf '%s\n' "${COL_RED}✗${COL_RESET} ${COL_BOLD}${tool}${COL_RESET} — no result" + [[ "$VERBOSE" == "1" ]] && echo "$resp" | jq '.' 2>/dev/null || echo "$resp" + return 1 +} + +# ─── REPL ─── +run_repl() { + echo "" + printf '%s\n' "${COL_CYAN}Agent Test Shell${COL_RESET} (type 'help' for commands, 'quit' to exit)" + echo "" + + session_init + + while true; do + printf "${COL_CYAN}>${COL_RESET} " + read -r line + [[ -z "$line" ]] && continue + [[ "$line" == "quit" || "$line" == "exit" ]] && break + [[ "$line" == "help" ]] && { + echo " call Call a tool with JSON args" + echo " call --key val ... Call with named args" + echo " api [body] Generic HTTP call" + echo " quit Exit" + continue + } + + local tool args + if [[ "$line" =~ ^api[[:space:]]+ ]]; then + local rest="${line#api }" + local method url body + if [[ "$rest" =~ ^([A-Z]+)[[:space:]]+([^[:space:]]+)(.*)$ ]]; then + method="${BASH_REMATCH[1]}" + url="${BASH_REMATCH[2]}" + body="${BASH_REMATCH[3]}" + [[ -z "$body" ]] && body="{}" + echo "$(api_call "$method" "$url" "$body")" + else + echo "Usage: api [body]" + fi + elif [[ "$line" =~ ^call[[:space:]]+ ]]; then + local rest="${line#call }" + if [[ "$rest" =~ ^([^[:space:]]+)[[:space:]]+(--.*)$ ]]; then + tool="${BASH_REMATCH[1]}" + args=$(build_args ${BASH_REMATCH[2]}) + elif [[ "$rest" =~ ^([^[:space:]]+)[[:space:]]+(.*)$ ]]; then + tool="${BASH_REMATCH[1]}" + args="${BASH_REMATCH[2]}" + [[ -z "$args" ]] && args="{}" + else + tool="$rest" + args="{}" + fi + local resp + resp=$(call_tool "$tool" "$args") + print_result "$tool" "$resp" + else + echo "Unknown command: $line (type 'help')" + fi + done +} + +# ─── Help ─── +print_help() { + cat <<'EOF' +agent.sh — Unified harness for testing the photofield server + +Usage: + agent.sh [options] [args...] + +Options: + --verbose, -v Global verbosity flag (must precede subcommand; also AGT_VERBOSE=1) + --help, -h, -- Print this help + +Server commands: + server start Start server (auto-detect running or launch) + server stop Stop via PID file (graceful) + server restart Stop + start + server status Show PID and port status + server kill Kill PID file + port listeners + +API commands: + api [body] Generic HTTP call (GET/POST/PUT/DELETE) + api --key val Named-arg body construction + +Tool commands: + mcp call Call a tool with JSON args + mcp call --key val Call a tool with named args + mcp quick [tool args] Smoke test (default: list_collections) + mcp shell Interactive REPL + +Environment variables: + AGT_PORT — Server port (default: 8080) + AGT_BIN — Path to photofield binary + AGT_DATA_DIR — Path to data directory + AGT_START — Auto-start server (default: true) + AGT_URL — Full endpoint URL + AGT_API_BASE — API base URL (default: http://localhost:$PORT) + AGT_VERBOSE — Verbose output (1 = yes) +EOF +} + +# ─── CLI ─── +# Top-level flags: --verbose/--v before subcommand +cmd="help" +subcmd="" +verbose_override=0 + +while [[ $# -gt 0 ]]; do + case "$1" in + --verbose|-v|-V) VERBOSE=1; verbose_override=1; shift ;; + --help|-h|--) cmd="help"; shift ;; + server) + cmd="server" + shift + # Detect misplaced --verbose (position 2) and fail loudly + if [[ $# -gt 0 && "$1" == "--verbose" ]]; then + echo "Error: --verbose must come before subcommand. Use: agent.sh --verbose server " >&2 + exit 1 + fi + subcmd="${1:-help}" + shift + ;; + api) cmd="api"; shift; break ;; + mcp) + cmd="mcp" + shift + # Detect misplaced --verbose (position 2) and fail loudly + if [[ $# -gt 0 && "$1" == "--verbose" ]]; then + echo "Error: --verbose must come before subcommand. Use: agent.sh --verbose mcp " >&2 + exit 1 + fi + break + ;; + *) + # If we got here with cmd="help" and no prior match, treat as unknown + if [[ "$cmd" == "help" ]]; then + cmd="help" + fi + break + ;; + esac +done + +# ─── Execute ─── +# For mcp/server, subcmd is the first remaining arg after the main loop +[[ -z "$subcmd" && $# -gt 0 && "$cmd" != "api" ]] && subcmd="$1" && shift + +case "$cmd" in + help) + print_help + exit 0 + ;; + + server) + case "$subcmd" in + start) server_start ;; + stop) server_stop ;; + restart) server_restart ;; + status) server_status ;; + kill) server_kill ;; + *) + echo "Usage: agent.sh server " >&2 + exit 1 + ;; + esac + ;; + + api) + if [[ $# -lt 2 ]]; then + echo "Usage: agent.sh api [body]" >&2 + exit 1 + fi + api_call "$@" + ;; + + mcp) + case "$subcmd" in + call) + # Parse: call_tool [-- key val ...] or call_tool + call_tool="" + named_mode=false + if [[ $# -eq 0 ]]; then + echo "Usage: agent.sh mcp call " >&2 + exit 1 + fi + call_tool="$1" + shift + if [[ $# -gt 0 && "$1" == "--" ]]; then + shift + named_mode=true + elif [[ $# -gt 0 && "$1" =~ ^--[a-zA-Z] ]]; then + named_mode=true + fi + + if [[ "$named_mode" == "true" ]]; then + args_json=$(build_args "$@") + elif [[ $# -gt 0 ]]; then + args_json="$*" + else + args_json="{}" + fi + + [[ "$AUTO_START" == "true" ]] && server_start + session_init + resp=$(call_tool "$call_tool" "$args_json") + print_result "$call_tool" "$resp" + ;; + quick) + [[ "$AUTO_START" == "true" ]] && server_start + session_init + quick_arg="${1:-list_collections}" + quick_tool="${quick_arg%% *}" + quick_args="${quick_arg#* }" + [[ "$quick_tool" == "$quick_args" ]] && quick_args="{}" + [[ -z "$quick_tool" ]] && quick_tool="list_collections" && quick_args="{}" + resp=$(call_tool "$quick_tool" "$quick_args") + if print_result "$quick_tool" "$resp"; then + log_ok "Quick test passed" + else + log_fail "Quick test failed" + exit 1 + fi + ;; + shell) + [[ "$AUTO_START" == "true" ]] && server_start + run_repl + ;; + *) + echo "Usage: agent.sh mcp [args...]" >&2 + exit 1 + ;; + esac + ;; + + *) + echo "Unknown command: $cmd" >&2 + print_help >&2 + exit 1 + ;; +esac