AnnotateKit

AnnotateKit turns drawings and comments inside your iOS app into prompts for your coding agent. Tap, draw, and send to Claude, Codex, Cursor, or any MCP client.

Works with SwiftUI, UIKit, React Native and Expo, on device and in the simulator. Everything is behind #if DEBUG, so Release builds carry none of it.

Install

Tell your agent:

Set up AnnotateKit from github.com/ooolzhas/annotatekit in this app.

The annotate-setup skill handles Xcode projects, XcodeGen, Tuist, SwiftPM, React Native and Expo. In Claude Code, one plugin brings the skill, the MCP server and the CLI:

/plugin install annotate --marketplace ooolzhas/annotatekit

import AnnotateKit

@main
struct MyApp: App {
    init() { Annotate.install() }   // no-op in Release
    var body: some Scene { WindowGroup { ContentView() } }
}

brew install ooolzhas/tap/annotate   # or: npm install -g annotate-ios · mint install ooolzhas/annotatekit

React Native and Expo

iOS only. The annotate-ios npm package carries the pod, and autolinking picks it up.

npm install -D annotate-ios

Expo: add the plugin to app.json, then run a development build (npx expo run:ios or expo-dev-client). Expo Go can't load native code.

{ "expo": { "plugins": ["annotate-ios"] } }

Bare React Native: add two lines to ios/<App>/AppDelegate, then cd ios && pod install.

import AnnotateKit                // Swift (RN 0.77+), at the top
Annotate.install()                // first line of application(_:didFinishLaunchingWithOptions:)

#import "AnnotateKit-Swift.h"                  // Obj-C (AppDelegate.mm), with the imports
[AnnotateBridge installWithFile:@__FILE__];    // first line of application:didFinishLaunchingWithOptions:

The app's iOS deployment target must be 17.0 or later (Expo: ios.deploymentTarget in expo-build-properties). The annotate CLI works the same as for native apps.

Usage

1. Tap Annotate to freeze the screen
2. Draw, tap an element, or hold on a spot
3. Add a note, attach an image
4. Tap ↑ to send
5. Follow the agent's progress on the pill

Connect your agent

Channels are a Claude Code research preview, which is why the flag is needed.

// Cursor: .cursor/mcp.json
{ "mcpServers": { "annotate": { "command": "annotate", "args": ["mcp"] } } }

// No CLI installed? npx fetches it on first run
{ "mcpServers": { "annotate": { "command": "npx", "args": ["-y", "annotate-ios", "mcp"] } } }

// VS Code: .vscode/mcp.json
{ "servers": { "annotate": { "type": "stdio", "command": "annotate", "args": ["mcp"] } } }

# Codex: ~/.codex/config.toml
[mcp_servers.annotate]
command = "annotate"
args = ["mcp"]

What agents get

Each note arrives as a specific component to check, not a description to interpret.

• Screenshots at 1x (pixel = point), an annotated copy, and a crop per pin
• Drawings as images and vectors
• Coordinates and the element under each pin, with nearby text to grep for
• Device metadata: model, iOS version, size, appearance, Dynamic Type
• Your notes, each tagged Fix or Question

Instead of "the blue button near the bottom is too small", the agent reads Button "Get started" at (24, 300, 120×44) and edits that view directly.

UI annotations from the running app: 2 pins on 1 screen. Please make these changes in the code.

Device: iPhone 17 Pro · iOS 26.0 · 402×874 pt @3x · dark · Dynamic Type L
Coordinates are iOS points, origin top-left. Screen images are 1x, so pixel = point.

Screen 1 (402×874 pt) — annotated …/screens/1-annotated.png · clean …/screens/1.png

1. Button "Get started" at (24, 300, 120×44) — tapped (84, 322), 21% across, 37% down
   bigger, tighter
   Nearby text: "Light that follows your day"
   Crop: …/crops/1.png

2. Sketch over Image "Halo Pendant" at (190, 520, 180×140) — center (280, 590), 70% across, 68% down
   make this card a carousel
   Crop: …/crops/2.png

Full detail (all frames, visible text per screen, sketch vectors): …/session.json

Tracking status

The pill narrates delivery (received → working → all changes applied), and each pin's badge turns into a check when the agent resolves it. Agents report per pin with MCP tools or the CLI:

The plugin's Stop hook keeps Claude from ending its turn while a send still shows as in progress.

Gestures

To annotate another screen, leave annotate mode, navigate, and come back. One send carries every screen.

How it works

• Selection. A tap hit-tests UIKit, then walks the accessibility tree (switched on the first time you annotate) to find SwiftUI Text, Button and Image. No view tagging needed; .annotatable() optionally adds the view's file:line.
• Delivery. The app writes a session bundle to .annotate/ in your project. On a physical device, the running annotate process copies it over devicectl, with no network permission or Info.plist keys. The bundle format is in SCHEMA.md.
• Cost. Nothing in Release and no dependencies. In DEBUG, near-zero until you enter annotate mode.

Limitations

• iOS 17+ only. Frames assume a full-screen window; Stage Manager windows aren't handled.
• No hover highlight: the Simulator doesn't deliver pointer hover to apps.
• Channels are a research preview, so their flags may change.

Contributing

See CONTRIBUTING.md and AGENTS.md.

License

MIT · annotatekit.com · @ooolzhas