## Basic model info - Model name: openai/openai whisper - Model description: Whisper large-v3 by OpenAI - speech-to-text transcription and translation with word-level timestamps and optional speaker diarization - Endpoint name: speech-to-text ## Model schema The model schema is defined in the OpenAPI schema: [OpenAPI Schema](https://oapi.sunra.ai/main/openai/whisper/latest.json) ### Model input schema The model input schema is: ```json { "description": "Input model for Whisper speech-to-text transcription.", "properties": { "prompt": { "default": "", "description": "Prompt to use for generation. Defaults to an empty string.", "title": "Prompt", "type": "string", "x-sr-order": 201 }, "audio_url": { "anyOf": [ { "format": "uri", "maxLength": 2083, "minLength": 1, "type": "string" }, { "type": "string" } ], "description": "URL of the audio (or video) file to transcribe. Supported formats: mp3, mp4, mpeg, mpga, m4a, wav or webm.", "title": "Audio Url", "x-sr-order": 301 }, "batch_size": { "default": 64, "description": "Batch size for the transcription.", "maximum": 64, "minimum": 1, "title": "Batch Size", "type": "integer", "x-sr-order": 405 }, "chunk_level": { "default": "segment", "description": "Level of the timestamp chunks to return. 'word' returns one chunk per spoken word with start/end timestamps; 'none' returns a single chunk with the whole transcription (usually slightly better quality).", "enum": [ "none", "segment", "word" ], "title": "Chunk Level", "type": "string", "x-sr-order": 404 }, "diarize": { "default": false, "description": "Whether to diarize the audio file (label speakers).", "title": "Diarize", "type": "boolean", "x-sr-order": 403 }, "language": { "anyOf": [ { "enum": [ "af", "am", "ar", "as", "az", "ba", "be", "bg", "bn", "bo", "br", "bs", "ca", "cs", "cy", "da", "de", "el", "en", "es", "et", "eu", "fa", "fi", "fo", "fr", "gl", "gu", "ha", "haw", "he", "hi", "hr", "ht", "hu", "hy", "id", "is", "it", "ja", "jw", "ka", "kk", "km", "kn", "ko", "la", "lb", "ln", "lo", "lt", "lv", "mg", "mi", "mk", "ml", "mn", "mr", "ms", "mt", "my", "ne", "nl", "nn", "no", "oc", "pa", "pl", "ps", "pt", "ro", "ru", "sa", "sd", "si", "sk", "sl", "sn", "so", "sq", "sr", "su", "sv", "sw", "ta", "te", "tg", "th", "tk", "tl", "tr", "tt", "uk", "ur", "uz", "vi", "yi", "yo", "zh" ], "type": "string" }, { "type": "null" } ], "description": "Language of the audio file. If not set, the language is detected automatically. If translate is selected as the task, the audio will be translated to English, regardless of the language selected.", "title": "Language", "x-sr-order": 402 }, "num_speakers": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "description": "Number of speakers in the audio file. If not set, the number of speakers is detected automatically.", "title": "Num Speakers", "x-sr-order": 406 }, "task": { "default": "transcribe", "description": "Task to perform on the audio file. Either transcribe or translate.", "enum": [ "transcribe", "translate" ], "title": "Task", "type": "string", "x-sr-order": 401 } }, "required": [ "audio_url" ], "title": "SpeechToTextInput", "type": "object" } ``` ### Model output schema The model output schema is: ```json { "description": "Output model for Whisper transcription results.", "properties": { "chunks": { "anyOf": [ { "items": { "description": "A timestamped chunk of the transcription (one word when chunk_level is 'word').", "properties": { "speaker": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "description": "Speaker ID of the chunk. Only present if diarization is enabled.", "title": "Speaker" }, "text": { "description": "Transcription of the chunk", "title": "Text", "type": "string" }, "timestamp": { "description": "Start and end timestamp of the chunk in seconds", "items": { "anyOf": [ { "type": "number" }, { "type": "null" } ] }, "title": "Timestamp", "type": "array" } }, "required": [ "timestamp", "text" ], "title": "WhisperChunk", "type": "object" }, "type": "array" }, { "type": "null" } ], "description": "Timestamp chunks of the audio file", "title": "Chunks" }, "diarization_segments": { "anyOf": [ { "items": { "description": "A speaker diarization segment.", "properties": { "speaker": { "description": "Speaker ID of the segment", "title": "Speaker", "type": "string" }, "timestamp": { "description": "Start and end timestamp of the segment in seconds", "items": { "type": "number" }, "title": "Timestamp", "type": "array" } }, "required": [ "timestamp", "speaker" ], "title": "DiarizationSegment", "type": "object" }, "type": "array" }, { "type": "null" } ], "description": "Speaker diarization segments of the audio file. Only present if diarization is enabled.", "title": "Diarization Segments" }, "inferred_languages": { "description": "List of languages that the audio file is inferred to be", "items": { "type": "string" }, "title": "Inferred Languages", "type": "array" }, "input_audio_duration": { "description": "Duration of the input media in seconds (rounded up)", "title": "Input Audio Duration", "type": "integer" }, "text": { "description": "Transcription of the audio file", "title": "Text", "type": "string" } }, "required": [ "text", "inferred_languages", "input_audio_duration" ], "title": "SpeechToTextOutput", "type": "object" } ``` ## Example inputs and outputs Use the following example inputs and outputs to understand the model. ### Input example ```json { "prompt": "", "audio_url": "", "batch_size": 64, "chunk_level": "segment", "diarize": false, "language": null, "num_speakers": null, "task": "transcribe" } ``` ### Output example ```json { } ``` ## Model code examples ### JavaScript ```javascript import { sunra } from "@sunra/client"; const result = await sunra.subscribe("openai/whisper/speech-to-text", { input: { prompt: '', audio_url: '', task: 'transcribe', language: null, diarize: false, chunk_level: 'segment', batch_size: 64, num_speakers: null }, logs: true, onQueueUpdate: (update) => { console.log(`Status Update: ${update.status}, Request ID: ${update.request_id}`); }, }); console.log(result.data); console.log(result.requestId); ``` ### Python ```python import sunra_client result = sunra_client.subscribe( "openai/whisper/speech-to-text", arguments={ "prompt": "", "audio_url": "", "task": "transcribe", "language": None, "diarize": False, "chunk_level": "segment", "batch_size": 64, "num_speakers": None }, with_logs=True, on_enqueue=print, on_queue_update=print, ) print(result) ``` ### Java ```java import ai.sunra.client.*; import java.util.Map; import com.google.gson.JsonObject; var client = SunraClient.withEnvCredentials(); var response = client.subscribe( "openai/whisper/speech-to-text", SubscribeOptions.builder() .input(Map.of( "prompt", "", "audio_url", "", "task", "transcribe", "language", null, "diarize", false, "chunk_level", "segment", "batch_size", 64, "num_speakers", null)) .resultType(JsonObject.class) .onQueueUpdate(update -> System.out.printf( "\nStatus Update: %s, Request ID: %s%n", update.getStatus(), update.getRequestId() )) .logs(true) .build() ); System.out.println("Completed!"); System.out.println(response.getData()); ``` ### Kotlin ```kotlin import ai.sunra.client.kt.* import com.google.gson.JsonObject val client = createSunraClient() val response = client.subscribe( endpointId = "openai/whisper/speech-to-text", input = mapOf( "prompt" to "", "audio_url" to "", "task" to "transcribe", "language" to null, "diarize" to false, "chunk_level" to "segment", "batch_size" to 64, "num_speakers" to null), options = ai.sunra.client.kt.SubscribeOptions(logs = true), onUpdate = { update -> println("\nStatus Update: ${update.status}, Request ID: ${update.requestId}") } ) println("Completed!") println(response.data) ``` ### Curl ```bash curl --request POST \ --url https://api.sunra.ai/v1/queue/openai/whisper/speech-to-text \ --header "Authorization: Key $SUNRA_KEY" \ --header "Content-Type: application/json" \ --data '{"prompt":"","audio_url":"","task":"transcribe","language":null,"diarize":false,"chunk_level":"segment","batch_size":64,"num_speakers":null}' ``` ## Model readme undefined