LLM Module

Micropython Example:

from module import LlmModule

def on_keyword_detected():
    print("[Keyword] detected")

def on_asr_data_input(data: str, finish: bool, index: int):
    print(f"[ASR data] {data}")

def on_llm_data_input(data: str, finish: bool, index: int):
    print(f"[LLM data] {data}")

module_llm = LlmModule()
module_llm.begin_voice_assistant()

module_llm.set_voice_assistant_on_keyword_detected_callback(on_keyword_detected)
module_llm.set_voice_assistant_on_asr_data_input_callback(on_asr_data_input)
module_llm.set_voice_assistant_on_llm_data_input_callback(on_llm_data_input)

while True:
    module_llm.update()

UIFLOW2 Example:

example.png

llm_voice_assistant.m5f2

class LlmModule

Constructors

class LlmModule

Initialize LlmModule and set up UART communication based on board type.

UIFLOW2:

init.png

Methods

LlmModule.update() None

Update ModuleLLM, receive response message.

UIFLOW2:

update.png

LlmModule.check_connection() bool

Check if the module connection is working properly.

Returns:

True if module connection is OK, False otherwise.

Return type:

bool

UIFLOW2:

check_connection.png

LlmModule.get_response_msg_list() list

Get the list of module’s response messages.

Returns:

List of response messages as dictionaries.

Return type:

list

UIFLOW2:

get_response_msg_list.png

LlmModule.clear_response_msg_list() None

Clear the module’s response message list.

UIFLOW2:

clear_response_msg_list.png

LlmModule.sys_ping() int

Send a ping to the system and get the response code.

UIFLOW2:

sys_ping.png

LlmModule.sys_reset(wait_reset_finish=True) int

Reset the system.

Parameters:

wait_reset_finish (bool) – Whether to wait for reset completion.

Returns:

Result of the reset command.

Return type:

int

UIFLOW2:

sys_reset.png

LlmModule.sys_reboot() int

Reboot the system.

Returns:

Result of the reboot command.

Return type:

int

UIFLOW2:

sys_reboot.png

LlmModule.llm_setup(prompt='', model='qwen2.5-0.5b', response_format='llm.utf-8.stream', input='llm.utf-8.stream', enoutput=True, enkws=True, max_token_len=127, request_id='llm_setup') str

Set up the LLM module.

Parameters:
  • prompt (str) – The prompt text.

  • model (str) – The model name.

  • response_format (str) – The response format.

  • input (str) – The input format.

  • enoutput (bool) – Enable output.

  • enkws (bool) – Enable keyword spotting.

  • max_token_len (int) – Maximum token length.

  • request_id (str) – Request ID.

Returns:

Result of the setup command.

Return type:

str

UIFLOW2:

llm_setup.png

LlmModule.llm_inference(work_id, input_data, request_id='llm_inference') str

Perform inference with the LLM module.

Parameters:
  • work_id – The work ID.

  • input_data – The input data.

  • request_id (str) – Request ID.

Returns:

Result of the inference command.

Return type:

str

UIFLOW2:

llm_inference.png

LlmModule.audio_setup(capcard=0, capdevice=0, cap_volume=0.5, playcard=0, playdevice=1, play_volume=0.15, request_id='audio_setup') str

Set up the audio module.

Parameters:
  • capcard (int) – Capture card index.

  • capdevice (int) – Capture device index.

  • cap_volume (float) – Capture volume.

  • playcard (int) – Playback card index.

  • playdevice (int) – Playback device index.

  • play_volume (float) – Playback volume.

  • request_id (str) – Request ID.

Returns:

Result of the setup command.

Return type:

str

UIFLOW2:

audio_setup.png

LlmModule.tts_setup(model='single_speaker_english_fast', response_format='tts.base64.wav', input='tts.utf-8.stream', enoutput=True, enkws=True, request_id='tts_setup') str

Set up the TTS module.

Parameters:
  • model (str) – TTS model name.

  • response_format (str) – The response format.

  • input (str) – The input format.

  • enoutput (bool) – Enable output.

  • enkws (bool) – Enable keyword spotting.

  • request_id (str) – Request ID.

Returns:

Result of the setup command.

Return type:

str

UIFLOW2:

tts_setup.png

LlmModule.kws_setup(kws='HELLO', model='sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01', response_format='kws.bool', input='sys.pcm', enoutput=True, request_id='kws_setup') str

Set up the KWS module.

Parameters:
  • kws (str) – Keyword to detect.

  • model (str) – KWS model name.

  • response_format (str) – The response format.

  • input (str) – The input format.

  • enoutput (bool) – Enable output.

  • request_id (str) – Request ID.

Returns:

Result of the setup command.

Return type:

str

UIFLOW2:

kws_setup.png

LlmModule.asr_setup(model='sherpa-ncnn-streaming-zipformer-20M-2023-02-17', response_format='asr.utf-8.stream', input='sys.pcm', enoutput=True, enkws=True, rule1=2.4, rule2=1.2, rule3=30.0, request_id='asr_setup') str

Set up the ASR module.

Parameters:
  • model (str) – ASR model name.

  • response_format (str) – The response format.

  • input (str) – The input format.

  • enoutput (bool) – Enable output.

  • enkws (bool) – Enable keyword spotting.

  • rule1 (float) – Rule 1 value.

  • rule2 (float) – Rule 2 value.

  • rule3 (float) – Rule 3 value.

  • request_id (str) – Request ID.

Returns:

Result of the setup command.

Return type:

str

UIFLOW2:

asr_setup.png

LlmModule.get_latest_llm_work_id() str

Get latest LLM module work id.

Returns:

Latest LLM module work id.

Return type:

str

UIFLOW2:

get_latest_llm_work_id.png

LlmModule.get_latest_audio_work_id() str

Get latest Audio module work id.

Returns:

Latest Audio module work id.

Return type:

str

UIFLOW2:

get_latest_audio_work_id.png

LlmModule.get_latest_tts_work_id() str

Get latest TTS module work id.

Returns:

Latest TTS module work id.

Return type:

str

UIFLOW2:

get_latest_tts_work_id.png

LlmModule.get_latest_kws_work_id() str

Get latest KWS module work id.

Returns:

Latest KWS module work id.

Return type:

str

UIFLOW2:

get_latest_kws_work_id.png

LlmModule.get_latest_asr_work_id() str

Get latest ASR module work id.

Returns:

Latest ASR module work id.

Return type:

str

UIFLOW2:

get_latest_asr_work_id.png

LlmModule.get_latest_error_code() int

Get latest ModuleLLM response error code.

Returns:

Latest ModuleLLM response error code.

Return type:

int

UIFLOW2:

get_latest_error_code.png

LlmModule.begin_voice_assistant(wake_up_keyword='HELLO', prompt='') bool

Begin the voice assistant.

Parameters:
  • wake_up_keyword (str) – The wake-up keyword.

  • prompt (str) – The assistant prompt.

Returns:

True if the voice assistant began successfully, False otherwise.

Return type:

bool

UIFLOW2:

begin_voice_assistant.png

LlmModule.set_voice_assistant_on_keyword_detected_callback(on_keyword_detected) None

Set the callback for when the wake-up keyword is detected.

Parameters:

on_keyword_detected – Callback function to be executed on keyword detection.

UIFLOW2:

set_voice_assistant_on_keyword_detected_callback.png

LlmModule.set_voice_assistant_on_asr_data_input_callback(on_asr_data_input) None

Set the callback for when ASR data is input.

Parameters:

on_asr_data_input – Callback function to handle ASR data input.

UIFLOW2:

set_voice_assistant_on_asr_data_input_callback.png

LlmModule.set_voice_assistant_on_llm_data_input_callback(on_llm_data_input) None

Set the callback for when LLM data is input.

Parameters:

on_llm_data_input – Callback function to handle LLM data input.

UIFLOW2:

set_voice_assistant_on_llm_data_input_callback.png