vr-orchestrator

This came out of a simple annoyance. I’d be in the headset working on a VR scene, spot something wrong, and have to take the whole thing off to go type at Claude. So now I just talk. I look at the scene, say what’s bugging me, say “go ahead”, and it packages that up and hands it to an agent sitting in the right repo.

There’s a panel that floats in the world next to whatever I’m building (that’s it in the image above), so I can see what the agents are up to without leaving VR. Because it’s a SteamVR overlay rather than part of any scene, it survives scene reloads and works on top of whatever VR app is running.

What I’m poking at:

• Voice as the whole interface: no push-to-talk, it just listens, and “go ahead” is the thing that fires off a task
• A fleet, not one agent: one session I’m talking to, dispatching separate long-lived agents into each project rather than trusting a single giant context
• One event-driven session: the orchestrator isn’t polling anything, it gets woken by whichever happens next, a finished voice turn or a finished agent, and both land in the same loop
• The panel as a dashboard: each agent’s context-window percentage, whether it’s busy, how far along it is, all readable from inside the scene
• There’s a --preview flag that draws the panel to a PNG, so I can fiddle with the UI without putting the headset on

As of July 2026 the whole thing is open source (MIT, Windows + SteamVR + Claude Code specific). Getting it there forced the fun kind of cleanup: machine paths moved into a gitignored config, the scene list became auto-discovered from whatever Godot OpenXR projects live under your workspace, and the agent launcher grew a config-driven adapter so it can drive agent CLIs other than Claude Code.