AI Co-host Guide

What It Does

There are two co-host workflows: a continuous voice companion in cohost.html, and manual dock actions that send approved lines to cohost-overlay.html.

Dock Right-click Co-host Setup

1. Open the Streaming Chat dock with your session: dock.html?session=YOUR_SESSION_ID.
2. Add the AI stage overlay to OBS: cohost-overlay.html?session=YOUR_SESSION_ID. Add &tts only for a text-response workflow.
3. For AI answers, configure an LLM provider in the popup. For quick testing, choose SSN Hosted Trial LLM, which uses llm.socialstream.ninja.
4. Enable Chat Bots and AI services > Chat Bot - Private Interface > Enable private chat bot option.
5. Right-click a dock message and select Co-host. The menu is hidden until the stage overlay is detected. Dock message rows do not yet expose this context menu to the keyboard; keyboard users can use the typed message field or voice controls on cohost.html.

Simple AI Option

llm.socialstream.ninja is available through SSN Hosted Trial LLM as a free simple option for testing while the trial is available.

• Open Chat Bots and AI services > Configure LLM Service Provider.
• Select SSN Hosted Trial LLM (experimental).
• Leave the endpoint, token, and model blank for the default trial setup.
• For longer-term use, use your own token or local provider if the trial service is disabled or rate-limited.

Voice Conversation Support

• OpenAI Realtime: Streams the microphone selected on cohost.html directly to OpenAI for one native speech-to-speech conversation. The spoken conversation and live-chat context stay in the same co-host session.
• SSN Desktop: Uses local Whisper with the microphone selected on cohost.html. The first use downloads about 42 MB; later transcription runs offline.
• Chrome / Chrome extension link: The extension opens the hosted co-host page in a Chrome tab. OpenAI Realtime uses WebRTC with a short-lived client secret; text-response providers use Chrome Speech Recognition and may require internet.
• Other browsers: Voice recognition is not guaranteed. Typed co-host chat, live-chat input, camera/screen input, and configured LLM responses can still work without speech recognition.

OpenAI Realtime Setup

1. In the SSN popup, choose ChatGPT API, add your OpenAI project API key, and use Test selected chat bot.
2. Open the generated cohost.html?session=YOUR_SESSION_ID link from the popup. Its private, expiring co-host capability authorizes Realtime without enabling the separate Private Chat Bot option.
3. Select OpenAI Realtime. The standard key stays in the SSN extension/Desktop background; the co-host page receives only a short-lived Realtime client secret.
4. Explicitly choose the microphone you talk into, keep Audio Response selected, and press Start Co-host.
5. Optional: leave Video source on No Video (the default), or explicitly choose Screen Share or a camera. OpenAI receives one current image with each direct spoken or typed request rather than continuous video; this can add API cost and latency.
6. Leave Live Chat on Context only - answer when asked. Ask “What is chat saying?” to have the same spoken AI discuss recent viewer messages.
7. Capture the audio from the co-host browser/app in OBS. The native OpenAI voice plays only from cohost.html; cohost-overlay.html receives the avatar and text, not that same audio stream. When supported, choose a virtual cable under Co-host output for isolated OBS routing.
8. Keep the stage overlay's speech bubble visible, or provide captions, so the AI response is not available through audio alone.

OpenAI Speed, Quality, And Cost

• Model: Mini is normally faster and costs less. Full quality is more capable and costs more.
• Reasoning effort: Minimal or Low normally lowers response delay and billed output usage. High or Extra high can improve complex answers but may respond more slowly and cost more. Changes apply while connected.
• Turn-taking speed: Fast waits up to about 2 seconds for the end of a thought; Balanced about 4 seconds; Patient about 8 seconds. Fast feels more responsive but can interrupt a natural pause.
• Diagnostics: Latency reports the time from the end of the turn to first text or audio, and separates model delay when possible. Network conditions and provider load can still vary between otherwise similar requests.

Optional Stream Controls

OpenAI Realtime supports the Spotify, OBS scene, and featured-chat tools below. SSN Configured LLM currently supports Spotify only; other co-host providers do not yet expose these tools.

1. In the SSN popup, enable only the co-host tools you want: Spotify, OBS scenes, or featured chat.
2. For OBS, enter the exact allowed scene names as a comma-separated list and keep actions.html connected to OBS, or use an OBS browser source with Full Permissions.
3. On cohost.html, open Co-host Stream Controls and arm the same tools for that controller.
4. Ask directly, for example “switch to BRB,” “feature that last message,” or “clear the featured chat.” Viewer messages cannot authorize tools, and only one stream-changing tool runs per direct streamer request.

How The Pages Talk

The dock and overlay use the existing Social Stream session bridge. Messages are sent as overlayNinja payloads and targeted by label.

{
  "action": "cohostOverlay",
  "target": "cohost-overlay",
  "meta": {
    "command": "say",
    "text": "The line the avatar should say",
    "speak": true,
    "emotion": "happy"
  }
}

Available Actions

Status And Troubleshooting

• If the Co-host menu is missing, the co-host overlay is not detected in the same session.
• If Answer or Light Roast is disabled, the SSN bridge or Private Chat Bot is not available.
• The primary chat bot overlay toggle is optional and does not enable dock right-click co-host actions.
• If cohost.html is not seeing chat, confirm the URL has the same session as the popup and that Live Chat is set to Context, Questions, or All.
• Context only currently shares chat with the AI only for OpenAI Realtime. Other providers monitor the feed without injecting it into the model.
• Mute microphone stops audio sent to the co-host. Mute co-host voice, the volume slider, output-device selector, and Stop speaking control playout. Mute shared system audio is separate and appears for screen-capable providers.
• If a stream-control tool is unavailable, enable it in both the SSN popup and Co-host Stream Controls. OBS also needs at least one exact scene name in its allowlist.
• Use Test overlay to send a harmless visible test line, then confirm it appears in the AI Stage Overlay in OBS.
• If the co-host knows chat but stays quiet, that is expected in Context only: ask it what chat is saying. Questions or All speak automatic replies from the co-host page; they do not post those replies into YouTube, Twitch, or Kick chat.
• If Desktop shows Listening (Desktop Whisper) but never fills Heard, confirm the selected microphone is not muted and allow the first model download to finish.
• If Chrome never fills Heard, allow microphone permission, check the operating system's default microphone, confirm internet access, and verify that Chrome Speech Recognition is available.
• If OpenAI Realtime text appears but no audio plays, verify Audio Response, browser audio permissions, and OBS browser/app audio capture. Use &tts only for the separate text-overlay workflow.
• If OpenAI Realtime replies are slow, try Mini, Low reasoning, and Fast turn-taking, then compare the Latency line in Diagnostics. Higher reasoning and Patient turn-taking deliberately trade speed for depth or longer pauses.
• If OpenAI Realtime disconnects, leave the co-host running while it retries. Diagnostics identifies the data, peer, provider, or health-check failure. A recovered session keeps the selected system instructions and performance settings but cannot recover the previous spoken conversation.
• If an Answer, Light Roast, or SSN Configured LLM request times out, check that Social Stream is on, the session matches, Private Chat Bot is enabled, and the selected LLM provider is reachable. OpenAI Realtime does not require Private Chat Bot.
• If Local Gemma cannot download its model files from the default asset host, set the asset host to your own mirrored Gemma folder.
• If the hosted trial AI stops responding, it may be disabled or rate-limited; switch to your own token, Ollama, or Custom API.
• If multiple overlays are open, make sure the overlay label matches the dock target. The default label is cohost-overlay.

Useful URL Parameters

• session=YOUR_SESSION_ID - required for the dock and overlay to share a room.
• tts or speak=1 - lets the overlay speak with browser TTS.
• forcetts - allows overlay TTS even when the co-host controller is already playing audio; this intentionally creates a second voice.
• label=cohost-overlay - sets the overlay target label.
• name=NinjaBot - sets the display name on the overlay.
• avatar=https://... - uses a custom avatar image.
• position=bottom-right - controls stage placement.
• scale=1.2 - scales the overlay.
• status - shows connection/status text on the overlay.

Design Notes

• For dock/manual text workflows, the overlay is the visual and optional TTS playout source. For native OpenAI voice, capture cohost.html audio separately; the stage overlay does not replay that same WebRTC audio.
• The dock is the control surface, so streamer approval stays off-stream.
• Generated responses are not auto-spoken; the streamer must approve them first.
• Non-standard command details live inside meta, keeping payloads predictable for overlays and automations.