I got tired of keeping a Telegram tab open all day just to talk to my local agent, so I built a monitor-resident avatar that fronts it instead — a video-call-style face that performs the replies instead of me reading a wall of text.
A few things that might interest this crowd:
No runtime GPU for the avatar. Instead of live-animating a face, the agent’s reply carries inline emotion-beat tags — [], [] deploy?, [] — and a player selects and blends between ~30 pre-rendered clips. The GPU stays 100% on the model. Code and logs render as chat cards instead of being read aloud, and [] pops a human-in-the-loop approve/cancel that blocks the agent until you answer.
It hooks into a real agent as a connector. The agent’s gateway dials out to a local WebSocket the app hosts, so from the agent’s side the avatar is just “another channel,” indistinguishable from a messenger. Adapters for Hermes and OpenClaw are included, plus a demo mode that runs with zero setup.
Local voice both ways. Edge TTS out (swappable to Qwen3-TTS / MeloTTS / Piper), Silero VAD + faster-whisper in.
Open-core, MIT. The engine is fully usable on its own: drop in a folder of clips named by emotion, point it at your agent. Avatars are pure-data bundles — no code runs when you install one — so you can build your own. The repo ships the engine only; there’s no bundled character.
The engine just plays video clips — it doesn’t care what produced the pixels. So the avatar can be photoreal, 2D anime, a 3D render, pixel art, anything. You can even use a Live2D or VRM model: pre-render its expressions into clips and it becomes a Ghost Vessel preset. A Live2D shell can’t do the reverse — it can’t take live-action footage. Clips are a superset. The trade-off is that you give up real-time rig interactivity (no dragging the head around) and you pay a one-time render step.
Repo + 45s demo: https://github.com/ghdtjrtka/ghost-vessel
The part I’d most like feedback on is the emotion-beat output format — the contract that turns model text into a UI performance. Has anyone here tried similar output contracts to drive an interface from LLM output? Curious what broke on your setup, and whether smaller local models keep the tags straight.
Yeah. Just add creating the example avatars to the project’s ROADMAP or TODO list. It’d be nice to have that from user perspective, just to get started. And after that people can create their own avatars. But you need to make the first step easy for them.
And if I might add: Skip the nonsense with the different licenses. Currently you have a correct LICENSE file with the MIT license. But then also a NOTICE file which says the MIT license doesn’t apply to all the project. And a different notice with different wording in the README.md. That’s kinda convoluted. And unnecessary complex since you don’t even provide avatars and/or presets… Just do the entire project under MIT license. Batteries included. That’ll aid with distribution.
Circling back since you called it: the NOTICE file is gone. Everything in the repo is MIT now — engine, tools, docs, and the bundled starter avatar — and the README says that in one line instead of three places contradicting each other.
The carve-out only ever existed because the photoreal avatar is a commissioned likeness I can’t relicense, and that one isn’t in the repo anyway. So it’s named where it’s sold rather than sitting in a legal side-file scaring people off the code. Thanks for pushing on it.
Awesome. Great work 👍
Better than the TODO list — it’s in the repo now.
git clonegives you a working avatar: 6 emotion beats, a blink-aligned idle, mood-based resting idles for when it’s been praised or scolded, and a talking loop.Only one avatar rather than the 4-5 you suggested, so “pick one you like” isn’t solved yet. But the bigger blocker was that producing one wasn’t documented honestly, and that part is fixed: she was built entirely with free local tools — Animagine XL for the neutral source still, HunyuanVideo 1.5 (480p i2v, step-distilled) for the clips, then the repo’s own scripts to loop-cut and validate them. One 12GB card, ~170s per clip, no paid service anywhere in the chain.
Worth flagging something I got wrong in my earlier reply: I’d assumed subtle expressions were just hard for video models in general. They’re not — Wan 2.2 5B was rendering “concerned” and “angry” as a blank neutral face, and swapping to Hunyuan 1.5 fixed it on the same hardware, faster. If you go down this path, model size matters more than prompt tuning does.
On the licensing: you’re right that it’s convoluted, and I’ve been sitting with that. The engine and every script and the starter avatar are all MIT, so for anyone cloning the repo it already is “batteries included, one license”. The NOTICE exists because the commissioned photoreal avatar is someone’s likeness and I can’t hand that out under MIT. But you’re correct that spelling that out in three places makes it read as if there are strings attached to the code, when there aren’t. I’ll collapse it into one clear line in the README instead of a separate file.
Still no Colab notebook. Plain Python + ffmpeg + MediaPipe.