com.alibaba.dashscope.audio.omni

类 OmniRealtimeConversation

java.lang.Object

okhttp3.WebSocketListener

com.alibaba.dashscope.audio.omni.OmniRealtimeConversation

public class OmniRealtimeConversation
extends okhttp3.WebSocketListener

作者:

lengjiayi

构造器概要

构造器

构造器和说明
OmniRealtimeConversation(OmniRealtimeParam param, OmniRealtimeCallback callback) Constructor

方法概要

限定符和类型	方法和说明
void	appendAudio(String audioBase64) send audio in base64 format
void	appendVideo(String videoBase64) send one image frame in video in base64 format
void	cancelResponse() cancel the current response
void	checkStatus() Omni APIs
void	clearAppendedAudio() clear the audio sent to server before.
void	close() close the connection to server
void	close(int code, String reason) close the connection to server
void	commit() Commit the audio and video sent before.
void	connect() Connect to server, create session and return default session configuration
void	createResponse(String instructions, List<OmniRealtimeModality> modalities) create response, use audio and video commited before to request llm.
long	getFirstAudioDelay()
long	getFirstTextDelay()
String	getResponseId()
String	getSessionId()
void	onClosed(okhttp3.WebSocket webSocket, int code, String reason)
void	onClosing(okhttp3.WebSocket webSocket, int code, String reason)
void	onFailure(okhttp3.WebSocket webSocket, Throwable t, okhttp3.Response response)
void	onMessage(okhttp3.WebSocket webSocket, String text)
void	onOpen(okhttp3.WebSocket webSocket, okhttp3.Response response) WebSocket callbacks
void	sendRaw(String rawData) send raw data to server
void	updateSession(OmniRealtimeConfig config) Update session configuration, should be used before create response

从类继承的方法 okhttp3.WebSocketListener

onMessage

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

OmniRealtimeConversation

public OmniRealtimeConversation(OmniRealtimeParam param,
                                OmniRealtimeCallback callback)

Constructor

参数:

param - apikey, model, url, etc.

callback - callback

方法详细资料

checkStatus

public void checkStatus()

Omni APIs

connect

public void connect()
             throws NoApiKeyException,
                    InterruptedException

Connect to server, create session and return default session configuration

抛出:

NoApiKeyException

InterruptedException

updateSession

public void updateSession(OmniRealtimeConfig config)

Update session configuration, should be used before create response

参数:

config - session configuration

appendAudio

public void appendAudio(String audioBase64)

send audio in base64 format

参数:

audioBase64 - base64 audio string

appendVideo

public void appendVideo(String videoBase64)

send one image frame in video in base64 format

参数:

videoBase64 - base64 image string

commit

public void commit()

Commit the audio and video sent before. When in Server VAD mode, the client does not need to use this method, the server will commit the audio automatically after detecting vad end.

clearAppendedAudio

public void clearAppendedAudio()

clear the audio sent to server before.

createResponse

public void createResponse(String instructions,
                           List<OmniRealtimeModality> modalities)

create response, use audio and video commited before to request llm. When in Server VAD mode, the client does not need to use this method, the server will create response automatically after detecting vad and sending commit.

参数:

instructions - instructions to llm

modalities - omni output modalities to be used in session

cancelResponse

public void cancelResponse()

cancel the current response

close

public void close()

close the connection to server

close

public void close(int code,
                  String reason)

close the connection to server

参数:

code - websocket close code

reason - websocket close reason

sendRaw

public void sendRaw(String rawData)

send raw data to server

参数:

rawData - raw data

getSessionId

public String getSessionId()

getResponseId

public String getResponseId()

getFirstAudioDelay

public long getFirstAudioDelay()

getFirstTextDelay

public long getFirstTextDelay()

onOpen

public void onOpen(okhttp3.WebSocket webSocket,
                   okhttp3.Response response)

WebSocket callbacks

覆盖:

onOpen 在类中 okhttp3.WebSocketListener

onMessage

public void onMessage(okhttp3.WebSocket webSocket,
                      String text)

覆盖:

onMessage 在类中 okhttp3.WebSocketListener

onClosed

public void onClosed(okhttp3.WebSocket webSocket,
                     int code,
                     String reason)

覆盖:

onClosed 在类中 okhttp3.WebSocketListener

onFailure

public void onFailure(okhttp3.WebSocket webSocket,
                      Throwable t,
                      okhttp3.Response response)

覆盖:

onFailure 在类中 okhttp3.WebSocketListener

onClosing

public void onClosing(@NotNull
                      okhttp3.WebSocket webSocket,
                      int code,
                      @NotNull
                      String reason)

覆盖:

onClosing 在类中 okhttp3.WebSocketListener