ai-integrations-openai

Provision the Replit AI Integrations proxy by calling setupReplitAIIntegrations with the openai provider slug. It automatically sets AI_INTEGRATIONS_OPENAI_BASE_URL and AI_INTEGRATIONS_OPENAI_API_KEY environment variables, and usage is billed to your Replit credits.

Use the voiceChatStream server function with gpt-audio for speech-to-speech responses, and the useVoiceRecorder and useVoiceStream React hooks on the client. Convert browser WebM recordings to WAV with ffmpeg before sending, and stream results over SSE.

Supported models include gpt-5.2 for general tasks, gpt-5.3-codex for coding, gpt-5-mini and gpt-5-nano for high-volume work, o4-mini and o3 for reasoning, gpt-image-1 for images, gpt-audio for speech-to-speech, and gpt-4o-mini-transcribe for transcription.

The integration does not support the embeddings API, fine-tuning API, files API, image variations, video inputs or outputs, and the speech API. For those use cases, use OpenAI directly with your own API key or another integration such as Gemini for video.

Use ElevenLabs for interactive, low-latency voice apps that require STT/TTS chaining with a separate text model. Use OpenAI gpt-audio for direct speech-to-speech chat and non-latency-sensitive tasks like batch transcription or audiobook generation.

Orval cannot generate typed hooks for SSE streaming endpoints, so the response type is unknown. Consume the text and voice streams manually with fetch and ReadableStream parsing, or use the provided React voice stream hooks which handle SSE parsing.