Using the AprilCam MCP Server

The MCP server (aprilcam mcp) exposes AprilCam to AI agents over the Model Context Protocol. You open cameras, build playfields, read tags and objects, capture frames, draw paths and overlays, and look up playfield features — all through tools, using opaque handles. You never need, and do not have, filesystem access: camera_id and playfield_id are handles, not paths.

The server is a thin client. All vision (AprilTag/ArUco detection, homography, deskew, object detection) runs in the daemon; the MCP server forwards perception results and raw frames. The live-camera flow needs no OpenCV in the MCP server’s environment.

Running it

The MCP server speaks stdio. Register it with your MCP client (e.g. an .mcp.json):

{
  "mcpServers": {
    "aprilcam": { "command": "aprilcam", "args": ["mcp"] }
  }
}

aprilcam init writes that block into .mcp.json and .vscode/mcp.json for you. (A second console entry point, aprilcam-mcp, runs the same server.)

The MCP server is a thin client: it connects to a running daemon — it does not start one. By default it finds the daemon the way every client does (a local Unix socket, or mDNS on the LAN). To pin it to a specific daemon, register it with a --daemon-host (or set APRILCAM_DAEMON_HOST):

{
  "mcpServers": {
    "aprilcam": {
      "command": "aprilcam",
      "args": ["mcp", "--daemon-host", "vidar.local"]
    }
  }
}

You can also switch daemons at runtime without restarting — call the connect_daemon tool (below). If no daemon is reachable, tools return a clear No aprilcam daemon found error; start one on the camera host (aprilcam daemon start, or systemd) — see Operating the Daemon.

Base install is enough — pipx install aprilcam ships the MCP SDK; OpenCV is not required (see install tiers).

The server keeps no state across restarts and auto-opens nothing. Every session must call open_camera first — it is the gate that registers the handle every downstream tool needs.

The golden path

1. open_camera(pattern="<name>")  ->  camera_id  (+ playfield_id if configured)
2. (only if no playfield_id came back)  create_playfield(camera_id)
3. stream_tags(playfield_id)      ->  start the detection loop
4. get_tags / get_objects / where / create_path  using the playfield_id
5. stop_stream(playfield_id)      ->  when done

• open_camera returns a playfield_id and playfield_name immediately when the camera is already configured and calibrated — skip step 2. A "calibration_stale": true flag means recalibration is needed.
• Pass the playfield_id as the source_id to stream_tags / get_tags / get_objects / where so that world_xy populates. Passing the camera_id also works — the server resolves the camera’s playfield.
• Discover the field with get_playfield() / list_playfields(); search it with where(). Call get_robot_api_guide() for the high-rate Python API.

Camera numbers are the persistent enumeration handle

list_cameras returns each camera’s persistent enumeration number — a stable handle that does not change when cameras are plugged/unplugged — as index. open_camera(index=N) accepts that same number (it is resolved to the live OS device index via the daemon). Prefer pattern= (a name substring) when you can; it is unambiguous.

Coordinate system

When the playfield is calibrated, every tag/object carries world_xy in centimetres, A1-centred: the origin is AprilTag 1 (playfield centre), +x is east, +y is north. Angles (orientation_yaw, heading_rad) are radians, 0 = +x, counter-clockwise positive — so a tag’s forward direction is (cos yaw, sin yaw). Parallax correction and the A1 origin shift are applied by the daemon. get_objects reports object world_xy in the same A1-centred frame as get_tags.

Image return format

Every tool that returns an image takes a format: "base64" (inline in the response, default) or "file" (a JPEG is written to a temp file and its path returned). Choose per request.

Tool reference

Version & connection

Cameras

Playfields

Detection, tags & objects

Frame capture

Live view, paths & overlays

Frame registry & static images ¹

¹ OpenCV note. Most tools are cv2-free, but a few do pixel work inside the MCP server and therefore need OpenCV in the MCP server’s environment: create_playfield (live corner detection), calibrate_playfield, and the frame-registry / static-image tools. For an already-calibrated camera the golden path never hits these — open_camera returns the playfield_id directly. Calibration is normally done once on the daemon host with aprilcam calibrate. Moving this residual pixel work into the daemon is tracked work; until then these tools raise a clear install-hint error on a base (no-OpenCV) install.

Robot hand-off

For a robot control loop that needs tags at 5–50 Hz or live overlay drawing, don’t drive it through MCP at runtime — call get_robot_api_guide() and hand the program the DaemonControl Python API, which talks to the daemon directly over gRPC.