Control AKARI with AI (Codex / Claude)

Every AKARI app — Video, SVG, Sheets, Stage, Design and 3D — can be driven directly by AI tools like Codex and Claude. Ask "cut this video," "draw this icon," or "turn this table into a chart," and the AI actually performs the edit. The AI can even decide what's needed and open or switch apps on its own (for example: "make a cover icon from this video" → it opens the SVG app and draws it). This works through MCP (Model Context Protocol) — a shared standard that lets AI tools talk to apps. Both Codex and Claude support it.

You → AI (Codex / Claude) • (MCP :47628) AKARI shell → open / switch apps (os_open_app) • (MCP :each port) each app → control that app (video_*, svg_*, sheets_* ...) • (MCP) Akari Pool → shared memory between apps (assets, analysis) Each app has a small built-in "window" (an MCP server) that starts automatically when you open the app inside AKARI. The AKARI shell itself also has a window, so the AI can open other apps for you. Everything stays on your own computer — nothing is exposed externally.

AKARI shell (open / switch apps): http://127.0.0.1:47628/mcp → os_* Video: http://127.0.0.1:47616/mcp → video_* SVG: http://127.0.0.1:47618/mcp → svg_* Sheets: http://127.0.0.1:47620/mcp → sheets_* Stage: http://127.0.0.1:47622/mcp → stage_* Design: http://127.0.0.1:47624/mcp → design_* 3D: http://127.0.0.1:47626/mcp → 3d_* You don't have to connect all of them — just the apps you use, plus the AKARI shell.

1. Launch AKARI Desktop. This starts the AKARI shell window (:47628). 2. Open the app you want to use inside it. This starts that app's window. 3. Open the project (Work) you want to edit. If you skip this, the AI will say "not connected" or time out. Just open the app in AKARI to fix it.

Add the windows you use to ~/.codex/config.toml (at least the shell plus the apps you want): [mcp_servers.akari-shell] url = "http://127.0.0.1:47628/mcp" [mcp_servers.akari-video] url = "http://127.0.0.1:47616/mcp" # Add the others the same way: # akari-svg=47618 / akari-sheets=47620 / akari-stage=47622 / akari-design=47624 / akari-3d=47626 Restart Codex and you'll have os_* and video_* (and more) tools.

Run these in your terminal: claude mcp add --transport http akari-shell http://127.0.0.1:47628/mcp claude mcp add --transport http akari-video http://127.0.0.1:47616/mcp claude mcp add --transport http akari-svg http://127.0.0.1:47618/mcp claude mcp add --transport http akari-sheets http://127.0.0.1:47620/mcp claude mcp add --transport http akari-stage http://127.0.0.1:47622/mcp claude mcp add --transport http akari-design http://127.0.0.1:47624/mcp claude mcp add --transport http akari-3d http://127.0.0.1:47626/mcp Add -s user to register for every project. Check it worked with: claude mcp list If an app isn't open yet it shows "Failed to connect" — that's normal. Open the app in AKARI and it connects automatically.

Once the AKARI shell window (:47628) is connected, the AI can open apps as needed: • "Make a cover icon from this video" → the AI opens SVG (os_open_app) and draws it • "Switch to Stage and place that video on the screen" → it opens Stage and places it • "Which app is open right now?" → os_current_app Right after opening an app the screen takes a moment to start up, so the AI waits until it can connect before acting.

Just ask in plain language. The trick is to ask the AI to "look at the state first" so it understands things before editing. Examples: • "Look at the app I have open" • Video: "Cut the silence at the beginning" • SVG: "Draw an icon: a rounded square with a star on top" • Sheets: "Turn this table into a bar chart, in blue" • Stage: "Place an iPhone in the center and rotate it slowly" • Design: "Resize this design for an Instagram post" • 3D: "Make a rounded box and fuse a sphere on top" Everything the AI does is mirrored on screen, and you can undo it.

A "skill" is a one-file Markdown manual that teaches the AI how to use an app properly (read state → propose → apply). With a skill installed, the AI follows the right workflow every time. Where skills go (Claude Code): • For all your projects: ~/.claude/skills/<skill-name>/SKILL.md • For one project only: <project>/.claude/skills/<skill-name>/SKILL.md Installing a skill from a URL: 1. Give the AI the URL of a SKILL.md. 2. Ask it to "save this skill to ~/.claude/skills/ and enable it." 3. The AI saves the file and the skill becomes active. 4. Type /<skill-name>, or just make a related request, and it activates automatically.

"Not connected" / timeout / "Failed to connect" → Check the app is open inside AKARI (its window must be running). The os_* window is ready as long as AKARI is running. Tool not found / outdated → Update or reload the app, then reconnect MCP on the AI side. Not sure what you can do → Ask the AI to "list the available tools" — it always returns the latest. Not sure which app maps to which window → See the Ports list above, or run os_list_apps.