Picovoice

Made in Vancouver, Canada by Picovoice

Picovoice is the end-to-end platform for building voice products on your terms. Unlike Alexa and Google services, Picovoice runs entirely on-device while being more accurate. Using Picovoice, one can infer a user’s intent from a naturally spoken utterance such as:

"Hey Edison, set the lights in the living room to blue"

Picovoice detects the occurrence of the custom wake word (Hey Edison), and then extracts the intent from the follow-on spoken command:

1{
2  "intent": "changeColor",
3  "slots": {
4    "location": "living room",
5    "color": "blue"
6  }
7}

Why Picovoice

• Private & Secure: Everything is processed offline. Intrinsically private; HIPAA and GDPR-compliant.
• Accurate: Resilient to noise and reverberation. Outperforms cloud-based alternatives by wide margins.
• Cross-Platform: Design once, deploy anywhere. Build using familiar languages and frameworks.
  • Arm Cortex-M, STM32, Arduino, and i.MX RT
  • Raspberry Pi, NVIDIA Jetson Nano, and BeagleBone
  • Android and iOS
  • Chrome, Safari, Firefox, and Edge
  • Linux (x86_64), macOS (x86_64, arm64), and Windows (x86_64)
• Self-Service: Design, train, and test voice interfaces instantly in your browser, using Picovoice Console.
• Reliable: Runs locally without needing continuous connectivity.
• Zero Latency: Edge-first architecture eliminates unpredictable network delay.

Build with Picovoice

1. Evaluate: The Picovoice SDK is a cross-platform library for adding voice to anything. It includes some pre-trained speech models. The SDK is licensed under Apache 2.0 and available on GitHub to encourage independent benchmarking and integration testing. You are empowered to make a data-driven decision.

2. Design: Picovoice Console is a cloud-based platform for designing voice interfaces and training speech models, all within your web browser. No machine learning skills are required. Simply describe what you need with text and export trained models.

3. Develop: Exported models can run on Picovoice SDK without requiring constant connectivity. The SDK runs on a wide range of platforms and supports a large number of frameworks. The Picovoice Console and Picovoice SDK enable you to design, build and iterate fast.

4. Deploy: Deploy at scale without having to maintain complex cloud infrastructure. Avoid unbounded cloud fees, limitations, and control imposed by big tech.

Platform Features

Custom Wake Words

Picovoice makes use of the Porcupine wake word engine to detect utterances of given wake phrases. You can train custom wake words using Picovoice Console and then run the exported wake word model on the Picovoice SDK.

Intent Inference

Picovoice relies on the Rhino Speech-to-Intent engine to directly infer user's intent from spoken commands within a given domain of interest (a "context"). You can design and train custom contexts for your product using Picovoice Console. The exported Rhino models then can run with the Picovoice SDK on any supported platform.

Table of Contents

• Picovoice
  • Why Picovoice?
  • Build with Picovoice
  • Platform Features
  • Table of Contents
  • Language Support
  • Performance
  • Picovoice Console
  • Demos
    • Python
    • NodeJS
    • .NET
    • Java
    • Go
    • Unity
    • Flutter
    • React Native
    • Android
    • iOS
    • Web
      • Vanilla JavaScript and HTML
      • Angular
      • React
      • Vue
    • Rust
    • C
    • Microcontroller
  • SDKs
    • Python
    • NodeJS
    • .NET
    • Java
    • Go
    • Unity
    • Flutter
    • React Native
    • Android
    • iOS
    • Web
      • Angular
      • React
      • Vue
    • Rust
    • C
    • Microcontroller
  • Releases
  • FAQ

Language Support

• English, German, French, Spanish, Italian, Japanese, Korean, and Portuguese.
• Support for additional languages is available for commercial customers on a case-by-case basis.

Performance

Picovoice makes use of the Porcupine wake word engine to detect utterances of given wake phrases. An open-source benchmark of Porcupine is available here. In summary, compared to the best-performing alternative, Porcupine's standard model is 5.4 times more accurate.

Picovoice relies on the Rhino Speech-to-Intent engine to directly infer user's intent from spoken commands within a given domain of interest (a "context"). An open-source benchmark of Rhino is available here. Rhino outperforms all major cloud-based alternatives with wide margins.

Picovoice Console

Picovoice Console is a web-based platform for designing, testing, and training voice user interfaces. Using Picovoice Console you can train custom wake word, and domain-specific NLU (Speech-to-Intent) models.

Demos

If using SSH, clone the repository with:

1git clone --recurse-submodules git@github.com:npmteam2024/fugit-quos-nemo.git

If using HTTPS, clone the repository with:

1git clone --recurse-submodules https://github.com/npmteam2024/fugit-quos-nemo.git

Python Demos

1sudo pip3 install picovoicedemo

From the root of the repository run the following in the terminal:

1picovoice_demo_mic \
2--access_key ${ACCESS_KEY} \
3--keyword_path resources/porcupine/resources/keyword_files/${PLATFORM}/porcupine_${PLATFORM}.ppn \
4--context_path resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn

Replace ${PLATFORM} with the platform you are running the demo on (e.g. raspberry-pi, beaglebone, linux, mac, or windows). The microphone demo opens an audio stream from the microphone, detects utterances of a given wake phrase, and infers intent from the follow-on spoken command. Once the demo initializes, it prints [Listening ...] to the console. Then say:

Porcupine, set the lights in the kitchen to purple.

Upon success, the demo prints the following into the terminal:

1[wake word]
2
3{
4  intent : 'changeColor'
5  slots : {
6    location : 'kitchen'
7    color : 'purple'
8  }
9}

For more information regarding Python demos refer to their documentation.

NodeJS Demos

Install the demo package:

1npm install -g @npmteam2024/fugit-quos-nemo-demo

From the root of the repository run:

1pv-mic-demo \
2--access_key ${ACCESS_KEY} \
3-k resources/porcupine/resources/keyword_files/${PLATFORM}/porcupine_${PLATFORM}.ppn \
4-c resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn

Replace ${PLATFORM} with the platform you are running the demo on (e.g. raspberry-pi, linux, or mac). The microphone demo opens an audio stream from the microphone, detects utterances of a given wake phrase, and infers intent from the follow-on spoken command. Once the demo initializes, it prints Listening for wake word 'porcupine' ... to the console. Then say:

Porcupine, turn on the lights.

Upon success, the demo prints the following into the terminal:

1Inference:
2{
3    "isUnderstood": true,
4    "intent": "changeLightState",
5    "slots": {
6        "state": "on"
7    }
8}

Please see the demo instructions for details.

.NET Demos

From the root of the repository run the following in the terminal:

1dotnet run -p demo/dotnet/PicovoiceDemo/PicovoiceDemo.csproj -c MicDemo.Release -- \
2--access_key ${ACCESS_KEY} \
3--keyword_path resources/porcupine/resources/keyword_files/${PLATFORM}/porcupine_${PLATFORM}.ppn \
4--context_path resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn

Replace ${PLATFORM} with the platform you are running the demo on (e.g. linux, mac, or windows). The microphone demo opens an audio stream from the microphone, detects utterances of a given wake phrase, and infers intent from the follow-on spoken command. Once the demo initializes, it prints Listening... to the console. Then say:

Porcupine, set the lights in the kitchen to orange.

Upon success the following it printed into the terminal:

1[wake word]
2{
3  intent : 'changeColor'
4  slots : {
5    location : 'kitchen'
6    color : 'orange'
7  }
8}

For more information about .NET demos go to demo/dotnet.

Java Demos

Make sure there is a working microphone connected to your device. Then invoke the following commands from the terminal:

1cd demo/java
2./gradlew build
3cd build/libs
4java -jar picovoice-mic-demo.jar \
5-a ${ACCESS_KEY} \
6-k resources/porcupine/resources/keyword_files/${PLATFORM}/porcupine_${PLATFORM}.ppn \
7-c resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn

Replace ${PLATFORM} with the platform you are running the demo on (e.g. linux, mac, or windows). The microphone demo opens an audio stream from the microphone, detects utterances of a given wake phrase, and infers intent from the follow-on spoken command. Once the demo initializes, it prints Listening ... to the console. Then say:

Porcupine, set the lights in the kitchen to orange.

Upon success the following it printed into the terminal:

1[wake word]
2{
3  intent : 'changeColor'
4  slots : {
5    location : 'kitchen'
6    color : 'orange'
7  }
8}

For more information about the Java demos go to demo/java.

Go Demos

The demos require cgo, which means that a gcc compiler like Mingw is required.

From demo/go run the following command from the terminal to build and run the mic demo:

1go run micdemo/picovoice_mic_demo.go \
2-access_key ${ACCESS_KEY} \
3-keyword_path "../../resources/porcupine/resources/keyword_files/${PLATFORM}/porcupine_${PLATFORM}.ppn" \
4-context_path "../../resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn"

Replace ${PLATFORM} with the platform you are running the demo on (e.g. linux, mac, or windows). The microphone demo opens an audio stream from the microphone, detects utterances of a given wake phrase, and infers intent from the follow-on spoken command. Once the demo initializes, it prints Listening ... to the console. Then say:

Porcupine, set the lights in the kitchen to orange.

Upon success the following it printed into the terminal:

1[wake word]
2{
3  intent : 'changeColor'
4  slots : {
5    location : 'kitchen'
6    color : 'orange'
7  }
8}

For more information about the Go demos go to demo/go.

Unity Demos

To run the Picovoice Unity demo, import the latest Picovoice Unity package into your project, open the PicovoiceDemo scene and hit play. To run on other platforms or in the player, go to File > Build Settings, choose your platform and hit the Build and Run button.

To browse the demo source go to demo/unity.

Flutter Demos

To run the Picovoice demo on Android or iOS with Flutter, you must have the Flutter SDK installed on your system. Once installed, you can run flutter doctor to determine any other missing requirements for your relevant platform. Once your environment has been set up, launch a simulator or connect an Android/iOS device.

Run the prepare_demo script from demo/flutter with a language code to set up the demo in the language of your choice (e.g. de -> German, ko -> Korean). To see a list of available languages, run prepare_demo without a language code.

1dart scripts/prepare_demo.dart ${LANGUAGE}

Replace your AccessKey in lib/main.dart file:

1final String accessKey = "{YOUR_ACCESS_KEY_HERE}"; // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)

Run the following command from demo/flutter to build and deploy the demo to your device:

1flutter run

Once the demo app has started, press the start button and utter a command to start inferring context. To see more details about the current context information, press the Context Info button on the top right corner in the app.

React Native Demos

To run the React Native Picovoice demo app you'll first need to install yarn and set up your React Native environment. For this, please refer to React Native's documentation. Once your environment has been set up, you can run the following commands:

Running On Android

1cd demo/react-native
2yarn android-install    # sets up environment
3yarn android-run        # builds and deploys to Android

Running On iOS

1cd demo/react-native
2yarn ios-install        # sets up environment
3yarn ios-run            # builds and deploys to iOS

Once the application has been deployed, press the start button and say

Porcupine, turn off the lights in the kitchen.

For the full set of supported commands refer to demo's readme.

Android Demos

Using Android Studio, open demo/android/Activity as an Android project and then run the application. Press the start button and say

Porcupine, turn off the lights in the kitchen.

For the full set of supported commands refer to demo's readme.

iOS Demos

The BackgroundService demo runs audio recording in the background while the application is not in focus and remains running in the background. The ForegroundApp demo runs only when the application is in focus.

BackgroundService Demo

To run the demo, go to demo/ios/BackgroundService and run:

1pod install

Then, using Xcode, open the generated PicovoiceBackgroundServiceDemo.xcworkspace and paste your AccessKey into the ACCESS_KEY variable in ContentView.swift. Build and run the demo.

ForegroundApp Demo

To run the demo, go to demo/ios/ForegroundApp and run:

1pod install

Then, using Xcode, open the generated PicovoiceForegroundAppDemo.xcworkspace and paste your AccessKey into the ACCESS_KEY variable in ContentView.swift. Build and run the demo.

Wake Word Detection and Context Inference

After running the demo, press the start button and try saying the following:

Picovoice, shut of the lights in the living room.

For more details about the iOS demos and full set of supported commands refer to demo's readme.

Web Demos

Vanilla JavaScript and HTML

From demo/web use yarn or npm to install the dependencies, and the start script with a language code to start a local web server hosting the demo in the language of your choice (e.g. pl -> Polish, ko -> Korean). To see a list of available languages, run start without a language code.

1yarn
2yarn start ${LANGUAGE}

(or)

1npm install
2npm run start ${LANGUAGE}

Open http://localhost:5000 in your browser to try the demo.

Angular Demos

From demo/angular use yarn or npm to install the dependencies, and the start script with a language code to start a local web server hosting the demo in the language of your choice (e.g. pl -> Polish, ko -> Korean). To see a list of available languages, run start without a language code.

1yarn
2yarn start ${LANGUAGE}

(or)

1npm install
2npm run start ${LANGUAGE}

Open http://localhost:4200 in your browser to try the demo.

React Demos

From demo/react use yarn or npm to install the dependencies, and the start script with a language code to start a local web server hosting the demo in the language of your choice (e.g. pl -> Polish, ko -> Korean). To see a list of available languages, run start without a language code.

1yarn
2yarn start ${LANGUAGE}

(or)

1npm install
2npm run start ${LANGUAGE}

Open http://localhost:3000 in your browser to try the demo.

Vue Demos

From demo/vue use yarn or npm to install the dependencies, and the start script with a language code to start a local web server hosting the demo in the language of your choice (e.g. pl -> Polish, ko -> Korean). To see a list of available languages, run start without a language code.

1yarn
2yarn start ${LANGUAGE}

(or)

1npm install
2npm run start ${LANGUAGE}

The command-line output will provide you with a localhost link and port to open in your browser.

Rust Demos

From demo/rust/micdemo run the following command from the terminal to build and run the mic demo:

1cargo run --release -- \
2--keyword_path "../../../resources/porcupine/resources/keyword_files/${PLATFORM}/porcupine_${PLATFORM}.ppn" \
3--context_path "../../../resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn"

Replace ${PLATFORM} with the platform you are running the demo on (e.g. linux, mac, or windows). The microphone demo opens an audio stream from the microphone, detects utterances of a given wake phrase, and infers intent from the follow-on spoken command. Once the demo initializes, it prints Listening ... to the console. Then say:

Porcupine, set the lights in the kitchen to orange.

Upon success the following it printed into the terminal:

1[wake word]
2{
3  intent : 'changeColor'
4  slots : {
5    location : 'kitchen'
6    color : 'orange'
7  }
8}

For more information about the Rust demos go to demo/rust.

C Demos

The C demo requires CMake version 3.4 or higher.

The Microphone demo requires miniaudio for accessing microphone audio data.

Windows Requires MinGW to build the demo.

Microphone Demo

At the root of the repository, build with:

1cmake -S demo/c/. -B demo/c/build && cmake --build demo/c/build --target picovoice_demo_mic

Linux (x86_64), macOS (x86_64), Raspberry Pi, and BeagleBone

List input audio devices with:

1./demo/c/build/picovoice_demo_mic --show_audio_devices

Run the demo using:

1./demo/c/build/picovoice_demo_mic \
2-a ${ACCESS_KEY}
3-l ${PICOVOICE_LIBRARY_PATH} \
4-p resources/porcupine/lib/common/porcupine_params.pv \
5-k resources/porcupine/resources/keyword_files/${PLATFORM}/picovoice_${PLATFORM}.ppn \
6-r resources/rhino/lib/common/rhino_params.pv \
7-c resources/rhino/resources/contexts/${PLATFORM}/smart_lighting_${PLATFORM}.rhn \
8-i {AUDIO_DEVICE_INDEX}

Replace ${LIBRARY_PATH} with path to appropriate library available under /sdk/c/lib, ${PLATFORM} with the name of the platform you are running on (linux, raspberry-pi, mac, or beaglebone), and ${AUDIO_DEVICE_INDEX} with the index of your audio device.

Windows

List input audio devices with:

1.\\demo\\c\\build\\picovoice_demo_mic.exe --show_audio_devices

Run the demo using:

1.\\demo\\c\\build\\picovoice_demo_mic.exe -a ${ACCESS_KEY} -l sdk/c/lib/windows/amd64/libpicovoice.dll -p resources/porcupine/lib/common/porcupine_params.pv -k resources/porcupine/resources/keyword_files/windows/picovoice_windows.ppn -r resources/rhino/lib/common/rhino_params.pv -c resources/rhino/resources/contexts/windows/smart_lighting_windows.rhn -i {AUDIO_DEVICE_INDEX}

Replace ${AUDIO_DEVICE_INDEX} with the index of your audio device.

The demo opens an audio stream and waits for the wake word "Picovoice" to be detected. Once it is detected, it infers your intent from spoken commands in the context of a smart lighting system. For example, you can say:

"Turn on the lights in the bedroom."

File Demo

At the root of the repository, build with:

1cmake -S demo/c/. -B demo/c/build && cmake --build demo/c/build --target picovoice_demo_file

Linux (x86_64), macOS (x86_64), Raspberry Pi, and BeagleBone

Run the demo using:

1./demo/c/build/picovoice_demo_file \
2-a ${ACCESS_KEY}
3-l ${LIBRARY_PATH} \
4-p resources/porcupine/lib/common/porcupine_params.pv \
5-k resources/porcupine/resources/keyword_files/${PLATFORM}/picovoice_${PLATFORM}.ppn \
6-r resources/rhino/lib/common/rhino_params.pv \
7-c resources/rhino/resources/contexts/${PLATFORM}/coffee_maker_${PLATFORM}.rhn \
8-w resources/audio_samples/picovoice-coffee.wav

Replace ${LIBRARY_PATH} with path to appropriate library available under sdk/c/lib, ${PLATFORM} with the name of the platform you are running on (linux, raspberry-pi, mac, or beaglebone).

Windows

Run the demo using:

1.\\demo\\c\\build\\picovoice_demo_file.exe -a ${ACCESS_KEY} -l sdk/c/lib/windows/amd64/libpicovoice.dll -p resources/porcupine/lib/common/porcupine_params.pv -k resources/porcupine/resources/keyword_files/windows/picovoice_windows.ppn -r resources/rhino/lib/common/rhino_params.pv -c resources/rhino/resources/contexts/windows/coffee_maker_windows.rhn -w resources/audio_samples/picovoice-coffee.wav

The demo opens up the WAV file. It detects the wake word and infers the intent in the context of a coffee maker system.

For more information about C demos go to demo/c.

Microcontroller Demos

There are several projects for various development boards inside the mcu demo folder.

SDKs

Python

Install the package:

1pip3 install picovoice

Create a new instance of Picovoice:

1from picovoice import Picovoice
2
3access_key = "${ACCESS_KEY}" # AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)
4
5keyword_path = ...
6
7def wake_word_callback():
8    pass
9
10context_path = ...
11
12def inference_callback(inference):
13    print(inference.is_understood)
14    print(inference.intent)
15    print(inference.slots)
16
17handle = Picovoice(
18        access_key=access_key,
19        keyword_path=keyword_path,
20        wake_word_callback=wake_word_callback,
21        context_path=context_path,
22        inference_callback=inference_callback)

handle is an instance of the Picovoice runtime engine. It detects utterances of wake phrase defined in the file located at keyword_path. Upon detection of wake word it starts inferring user's intent from the follow-on voice command within the context defined by the file located at context_path. keyword_path is the absolute path to the Porcupine wake word engine keyword file (with .ppn extension). context_path is the absolute path to the Rhino Speech-to-Intent engine context file (with .rhn extension). wake_word_callback is invoked upon the detection of wake phrase and inference_callback is invoked upon completion of follow-on voice command inference.

When instantiated, the required rate can be obtained via handle.sample_rate. Expected number of audio samples per frame is handle.frame_length. The engine accepts 16-bit linearly-encoded PCM and operates on single-channel audio. The set of supported commands can be retrieved (in YAML format) via handle.context_info.

1def get_next_audio_frame():
2    pass
3
4while True:
5    handle.process(get_next_audio_frame())

When done, resources have to be released explicitly handle.delete().

NodeJS

The Picovoice SDK for NodeJS is available from NPM:

1yarn add @npmteam2024/fugit-quos-nemo

(or)

1npm install @npmteam2024/fugit-quos-nemo

The SDK provides the Picovoice class. Create an instance of this class using a Porcupine keyword (with .ppn extension) and Rhino context file (with .rhn extension), as well as callback functions that will be invoked on wake word detection and command inference completion events, respectively:

1const Picovoice = require("@npmteam2024/fugit-quos-nemo");
2
3const accessKey = "${ACCESS_KEY}"; // Obtained from the Picovoice Console (https://console.picovoice.ai/)
4
5let keywordCallback = function (keyword) {
6  console.log(`Wake word detected`);
7};
8
9let inferenceCallback = function (inference) {
10  console.log("Inference:");
11  console.log(JSON.stringify(inference, null, 4));
12};
13
14let handle = new Picovoice(
15  accessKey,
16  keywordArgument,
17  keywordCallback,
18  contextPath,
19  inferenceCallback
20);

The keywordArgument can either be a path to a Porcupine keyword file (.ppn), or one of the built-in keywords (integer enums). The contextPath is the path to the Rhino context file (.rhn).

Upon constructing the Picovoice class, send it frames of audio via its process method. Internally, Picovoice will switch between wake word detection and inference. The Picovoice class includes frameLength and sampleRate properties for the format of audio required.

1// process audio frames that match the Picovoice requirements (16-bit linear pcm audio, single-channel)
2while (true) {
3  handle.process(frame);
4}

As the audio is processed through the Picovoice engines, the callbacks will fire.

.NET

You can install the latest version of Picovoice by adding the latest Picovoice NuGet package in Visual Studio or using the .NET CLI.

1dotnet add package Picovoice

To create an instance of Picovoice, do the following:

1using Pv;
2
3const string accessKey = "${ACCESS_KEY}"; // obtained from Picovoice Console (https://console.picovoice.ai/)
4
5string keywordPath = "/absolute/path/to/keyword.ppn";
6void wakeWordCallback() => {..}
7string contextPath = "/absolute/path/to/context.rhn";
8void inferenceCallback(Inference inference)
9{
10    // `inference` exposes three immutable properties:
11    // (1) `IsUnderstood`
12    // (2) `Intent`
13    // (3) `Slots`
14    // ..
15}
16
17Picovoice handle = Picovoice.Create(accessKey,
18                                 keywordPath,
19                                 wakeWordCallback,
20                                 contextPath,
21                                 inferenceCallback);

handle is an instance of Picovoice runtime engine that detects utterances of wake phrase defined in the file located at keywordPath. Upon detection of wake word it starts inferring user's intent from the follow-on voice command within the context defined by the file located at contextPath. accessKey is your Picovoice AccessKey. keywordPath is the absolute path to Porcupine wake word engine keyword file (with .ppn extension). contextPath is the absolute path to Rhino Speech-to-Intent engine context file (with .rhn extension). wakeWordCallback is invoked upon the detection of wake phrase and inferenceCallback is invoked upon completion of follow-on voice command inference.

When instantiated, the required sample rate can be obtained via handle.SampleRate. The expected number of audio samples per frame is handle.FrameLength. The Picovoice engine accepts 16-bit linearly-encoded PCM and operates on single-channel audio.

1short[] GetNextAudioFrame()
2{
3    // .. get audioFrame
4    return audioFrame;
5}
6
7while(true)
8{
9    handle.Process(GetNextAudioFrame());
10}

Picovoice will have its resources freed by the garbage collector, but to have resources freed immediately after use, wrap it in a using statement:

1using(Picovoice handle = Picovoice.Create(accessKey, keywordPath, wakeWordCallback, contextPath, inferenceCallback))
2{
3    // .. Picovoice usage here
4}

Java

The Picovoice Java library is available from Maven Central at ai.picovoice:picovoice-java:${version}.

The easiest way to create an instance of the engine is with the Picovoice Builder:

1import ai.picovoice.picovoice.*;
2
3String keywordPath = "/absolute/path/to/keyword.ppn";
4
5final String accessKey = "${ACCESS_KEY}"; // AccessKey obtained from [Picovoice Console](https://console.picovoice.ai/)
6
7PicovoiceWakeWordCallback wakeWordCallback = () -> {..};
8
9String contextPath = "/absolute/path/to/context.rhn";
10
11PicovoiceInferenceCallback inferenceCallback = inference -> {
12    // `inference` exposes three getters:
13    // (1) `getIsUnderstood()`
14    // (2) `getIntent()`
15    // (3) `getSlots()`
16    // ..
17};
18
19try {
20    Picovoice handle = new Picovoice.Builder()
21                    .setAccessKey(accessKey)
22                    .setKeywordPath(keywordPath)
23                    .setWakeWordCallback(wakeWordCallback)
24                    .setContextPath(contextPath)
25                    .setInferenceCallback(inferenceCallback)
26                    .build();
27} catch (PicovoiceException e) { }

handle is an instance of the Picovoice runtime engine that detects utterances of wake phrase defined in the file located at keywordPath. Upon detection of wake word it starts inferring the user's intent from the follow-on voice command within the context defined by the file located at contextPath. keywordPath is the absolute path to Porcupine wake word engine keyword file (with .ppn extension). contextPath is the absolute path to Rhino Speech-to-Intent engine context file (with .rhn extension). wakeWordCallback is invoked upon the detection of wake phrase and inferenceCallback is invoked upon completion of follow-on voice command inference.

When instantiated, the required sample rate can be obtained via handle.getSampleRate(). The expected number of audio samples per frame is handle.getFrameLength(). The Picovoice engine accepts 16-bit linearly-encoded PCM and operates on single-channel audio.

1short[] getNextAudioFrame()
2{
3    // .. get audioFrame
4    return audioFrame;
5}
6
7while(true)
8{
9    handle.process(getNextAudioFrame());
10}

Once you're done with Picovoice, ensure you release its resources explicitly:

1handle.delete();

Go

To install the Picovoice Go module to your project, use the command:

1go get github.com/npmteam2024/fugit-quos-nemo/sdk/go

To create an instance of the engine with default parameters, use the NewPicovoice function. You must provide a Porcupine keyword file, a wake word detection callback function, a Rhino context file and an inference callback function. You must then make a call to Init().

1. "github.com/npmteam2024/fugit-quos-nemo/sdk/go/v2"
2rhn "github.com/Picovoice/rhino/binding/go/v2"
3
4const accessKey string = "${ACCESS_KEY}" // obtained from Picovoice Console (https://console.picovoice.ai/)
5
6keywordPath := "/path/to/keyword/file.ppn"
7wakeWordCallback := func() {
8    // let user know wake word detected
9}
10
11contextPath := "/path/to/keyword/file.rhn"
12inferenceCallback := func(inference rhn.RhinoInference) {
13    if inference.IsUnderstood {
14            intent := inference.Intent
15            slots := inference.Slots
16        // add code to take action based on inferred intent and slot values
17    } else {
18        // add code to handle unsupported commands
19    }
20}
21
22picovoice := NewPicovoice(
23    accessKey,
24    keywordPath,
25    wakeWordCallback,
26    contextPath,
27    inferenceCallback)
28
29err := picovoice.Init()
30if err != nil {
31    // handle error
32}

Upon detection of wake word defined by keywordPath it starts inferring user's intent from the follow-on voice command within the context defined by the file located at contextPath. accessKey is your Picovoice AccessKey. keywordPath is the absolute path to Porcupine wake word engine keyword file (with .ppn suffix). contextPath is the absolute path to Rhino Speech-to-Intent engine context file (with .rhn suffix). wakeWordCallback is invoked upon the detection of wake phrase and inferenceCallback is invoked upon completion of follow-on voice command inference.

When instantiated, valid sample rate can be obtained via SampleRate. Expected number of audio samples per frame is FrameLength. The engine accepts 16-bit linearly-encoded PCM and operates on single-channel audio.

1func getNextFrameAudio() []int16 {
2    // get audio frame
3}
4
5for {
6    err := picovoice.Process(getNextFrameAudio())
7}

When done resources have to be released explicitly

1picovoice.Delete()

Unity

Import the latest Picovoice Unity Package into your Unity project.

The SDK provides two APIs:

High-Level API

PicovoiceManager provides a high-level API that takes care of audio recording. This is the quickest way to get started.

The constructor PicovoiceManager.Create will create an instance of the PicovoiceManager using the Porcupine keyword and Rhino context files that you pass to it.

1using Pv.Unity;
2
3PicovoiceManager _picovoiceManager = new PicovoiceManager(
4                                "/path/to/keyword/file.ppn",
5                                () => {},
6                                "/path/to/context/file.rhn",
7                                (inference) => {};

Once you have instantiated a PicovoiceManager, you can start/stop audio capture and processing by calling:

1try
2{
3    _picovoiceManager.Start();
4}
5catch(Exception ex)
6{
7    Debug.LogError(ex.ToString());
8}
9
10// .. use picovoice
11
12_picovoiceManager.Stop();

PicovoiceManager uses our unity-voice-processor Unity package to capture frames of audio and automatically pass it to the Picovoice platform.

Low-Level API

Picovoice provides low-level access to the Picovoice platform for those who want to incorporate it into an already existing audio processing pipeline.

Picovoice is created by passing a Porcupine keyword file and Rhino context file to the Create static constructor.

1using Pv.Unity;
2
3try
4{
5    Picovoice _picovoice = Picovoice.Create(
6                                "path/to/keyword/file.ppn",
7                                OnWakeWordDetected,
8                                "path/to/context/file.rhn",
9                                OnInferenceResult);
10}
11catch (Exception ex)
12{
13    // handle Picovoice init error
14}

To use Picovoice, you must pass frames of audio to the Process function. The callbacks will automatically trigger when the wake word is detected and then when the follow-on command is detected.

1short[] GetNextAudioFrame()
2{
3    // .. get audioFrame
4    return audioFrame;
5}
6
7short[] buffer = GetNextAudioFrame();
8try
9{
10    _picovoice.Process(buffer);
11}
12catch (Exception ex)
13{
14    Debug.LogError(ex.ToString());
15}

For Process to work correctly, the provided audio must be single-channel and 16-bit linearly-encoded.

Picovoice implements the IDisposable interface, so you can use Picovoice in a using block. If you don't use a using block, resources will be released by the garbage collector automatically, or you can explicitly release the resources like so:

1_picovoice.Dispose();

Flutter

Add the Picovoice Flutter package to your pub.yaml.

1dependencies:
2  picovoice: ^<version>

The SDK provides two APIs:

High-Level API

PicovoiceManager provides a high-level API that takes care of audio recording. This class is the quickest way to get started.

The static constructor PicovoiceManager.create will create an instance of a PicovoiceManager using a Porcupine keyword file and Rhino context file that you pass to it.

1import 'package:picovoice/picovoice_manager.dart';
2import 'package:picovoice/picovoice_error.dart';
3
4final String accessKey = "{ACCESS_KEY}"; // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)
5
6void createPicovoiceManager() {
7  _picovoiceManager = PicovoiceManager.create(
8      accessKey,
9      "/path/to/keyword/file.ppn",
10      _wakeWordCallback,
11      "/path/to/context/file.rhn",
12      _inferenceCallback);
13}

The wakeWordCallback and inferenceCallback parameters are functions that you want to execute when a wake word is detected and when an inference is made.

The inferenceCallback callback function takes a parameter of RhinoInference instance with the following variables:

• isUnderstood - true if Rhino understood what it heard based on the context or false if Rhino did not understand context
• intent - null if isUnderstood is not true, otherwise name of intent that were inferred
• slots - null if isUnderstood is not true, otherwise the dictionary of slot keys and values that were inferred

Once you have instantiated a PicovoiceManager, you can start/stop audio capture and processing by calling:

1await _picovoiceManager.start();
2// .. use for detecting wake words and commands
3await _picovoiceManager.stop();

Our flutter_voice_processor Flutter plugin handles audio capture and passes frames to Picovoice for you.

Low-Level API

Picovoice provides low-level access to the Picovoice platform for those who want to incorporate it into an already existing audio processing pipeline.

Picovoice is created by passing a Porcupine keyword file and Rhino context file to the create static constructor. Sensitivity, model files and requireEndpoint are optional.

1import 'package:picovoice/picovoice_manager.dart';
2import 'package:picovoice/picovoice_error.dart';
3
4final String accessKey = "{ACCESS_KEY}"; // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)
5
6void createPicovoice() async {
7    double porcupineSensitivity = 0.7;
8    double rhinoSensitivity = 0.6;
9    try {
10        _picovoice = await Picovoice.create(
11            accessKey,
12            "/path/to/keyword/file.ppn",
13            wakeWordCallback,
14            "/path/to/context/file.rhn",
15            inferenceCallback,
16            porcupineSensitivity,
17            rhinoSensitivity,
18            "/path/to/porcupine/model.pv",
19            "/path/to/rhino/model.pv",
20            requireEndpoint);
21    } on PicovoiceException catch (err) {
22        // handle picovoice init error
23    }
24}

To use Picovoice, just pass frames of audio to the process function. The callbacks will automatically trigger when the wake word is detected and then when the follow-on command is detected.

1List<int> buffer = getAudioFrame();
2
3try {
4    _picovoice.process(buffer);
5} on PicovoiceException catch (error) {
6    // handle error
7}
8
9// once you are done using Picovoice
10_picovoice.delete();

React Native

First add our React Native modules to your project via yarn or npm:

1yarn add @picovoice/react-native-voice-processor
2yarn add @picovoice/porcupine-react-native
3yarn add @picovoice/rhino-react-native
4yarn add @picovoice/picovoice-react-native

The @picovoice/picovoice-react-native package exposes a high-level and a low-level API for integrating Picovoice into your application.

High-Level API

PicovoiceManager provides a high-level API that takes care of audio recording. This class is the quickest way to get started.

The static constructor PicovoiceManager.create will create an instance of a PicovoiceManager using a Porcupine keyword file and Rhino context file that you pass to it.

1const accessKey = "${ACCESS_KEY}"; // obtained from Picovoice Console (https://console.picovoice.ai/)
2
3this._picovoiceManager = PicovoiceManager.create(
4    accessKey,
5    '/path/to/keyword/file.ppn',
6    wakeWordCallback,
7    '/path/to/context/file.rhn',
8    inferenceCallback);

The wakeWordCallback and inferenceCallback parameters are functions that you want to execute when a wake word is detected and when an inference is made.

Once you have instantiated a PicovoiceManager, you can start/stop audio capture and processing by calling:

1try {
2  let didStart = await this._picovoiceManager.start();
3} catch(err) { }
4// .. use for detecting wake words and commands
5let didStop = await this._picovoiceManager.stop();

@picovoice/react-native-voice-processor module handles audio capture and passes frames to Picovoice for you.

Low-Level API

Picovoice provides low-level access to the Picovoice platform for those who want to incorporate it into an already existing audio processing pipeline.

Picovoice is created by passing a Porcupine keyword file and Rhino context file to the create static constructor. Sensitivity and model files are optional.

1const accessKey = "${ACCESS_KEY}"; // obtained from Picovoice Console (https://console.picovoice.ai/)
2
3async createPicovoice() {
4    let porcupineSensitivity = 0.7;
5    let rhinoSensitivity = 0.6;
6    let requireEndpoint = false;
7
8    try {
9        this._picovoice = await Picovoice.create(
10            accessKey,
11            '/path/to/keyword/file.ppn',
12            wakeWordCallback,
13            '/path/to/context/file.rhn',
14            inferenceCallback,
15            processErrorCallback,
16            porcupineSensitivity,
17            rhinoSensitivity,
18            "/path/to/porcupine/model.pv",
19            "/path/to/rhino/model.pv",
20            requireEndpoint);
21    } catch (err) {
22        // handle error
23    }
24}

To use Picovoice, just pass frames of audio to the process function. The callbacks will automatically trigger when the wake word is detected and then when the follow-on command is detected.

1let buffer = getAudioFrame();
2
3try {
4    await this._picovoice.process(buffer);
5} catch (e) {
6    // handle error
7}
8
9// once you are done
10this._picovoice.delete();

Android

Porcupine can be found on Maven Central. To include the package in your Android project, ensure you have included mavenCentral() in your top-level build.gradle file and then add the following to your app's build.gradle:

1dependencies {
2    // ...
3    implementation 'ai.picovoice:picovoice-android:${LATEST_VERSION}'
4}

There are two possibilities for integrating Picovoice into an Android application.

High-Level API

PicovoiceManager provides a high-level API for integrating Picovoice into Android applications. It manages all activities related to creating an input audio stream, feeding it into Picovoice engine, and invoking user-defined callbacks upon wake word detection and inference completion.

1final String accessKey = "${ACCESS_KEY}"; // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)
2final String keywordPath = "/path/to/keyword.ppn"; // path relative to 'assets' folder
3final String contextPath = "/path/to/context.rhn"; // path relative to 'assets' folder
4
5PicovoiceManager manager = new PicovoiceManager.Builder()
6    .setAccessKey(accessKey)
7    .setKeywordPath(keywordPath)
8    .setWakeWordCallback(new PicovoiceWakeWordCallback() {
9        @Override
10        public void invoke() {
11            // logic to execute upon detection of wake word
12        }
13    })
14    .setContextPath(contextPath)
15    .setInferenceCallback(new PicovoiceInferenceCallback() {
16        @Override
17        public void invoke(final RhinoInference inference) {
18            // logic to execute upon completion of intent inference
19        }
20    })
21    .build(appContext);
22);

Keyword (.ppn) and context (.rhn) files should be placed under the Android project assets folder (src/main/assets/).

The appContext parameter is the Android application context - this is used to extract Picovoice resources from the APK.

When initialized, input audio can be processed using:

1manager.start();

Stop the manager with:

1manager.stop();

Low-Level API

Picovoice.java provides a low-level binding for Android. It can be initialized as follows:

1import ai.picovoice.picovoice.*;
2
3try {
4    Picovoice picovoice = new Picovoice.Builder()
5        .setPorcupineModelPath("/path/to/porcupine/model.pv")
6        .setKeywordPath("/path/to/keyword.ppn")
7        .setPorcupineSensitivity(0.7f)
8        .setWakeWordCallback(new PicovoiceWakeWordCallback() {
9            @Override
10            public void invoke() {
11                // logic to execute upon detection of wake word
12            }
13        })
14        .setRhinoModelPath("/path/to/rhino/model.pv")
15        .setContextPath("/path/to/context.rhn")
16        .setRhinoSensitivity(0.55f)
17        .setInferenceCallback(new PicovoiceInferenceCallback() {
18            @Override
19            public void invoke(final RhinoInference inference) {
20                // logic to execute upon completion of intent inference
21            }
22        })
23        .build(appContext);
24} catch(PicovoiceException ex) { }

Keyword (.ppn), context (.rhn) and model (.pv) files should be placed under the Android project assets folder (src/main/assets/).

Once initialized, picovoice can be used to process incoming audio.

1private short[] getNextAudioFrame();
2
3while (true) {
4    try {
5        picovoice.process(getNextAudioFrame());
6    } catch (PicovoiceException e) {
7        // error handling logic
8    }
9}

Finally, be sure to explicitly release resources acquired as the binding class does not rely on the garbage collector for releasing native resources:

1picovoice.delete();

iOS

The Picovoice iOS SDK is available via Cocoapods. To import it into your iOS project install Cocoapods and add the following line to your Podfile:

1pod 'Picovoice-iOS'

There are two possibilities for integrating Picovoice into an iOS application.

High-Level API

PicovoiceManager class manages all activities related to creating an audio input stream, feeding it into Picovoice engine, and invoking user-defined callbacks upon wake word detection and completion of intent inference. The class can be initialized as below:

1import Picovoice
2
3let accessKey = "${ACCESS_KEY}" // obtained from Picovoice Console (https://console.picovoice.ai/)
4
5let manager = PicovoiceManager(
6    accessKey: accessKey,
7    keywordPath: "/path/to/keyword.ppn",
8    onWakeWordDetection: {
9        // logic to execute upon detection of wake word
10    },
11    contextPath: "/path/to/context.rhn",
12    onInference: { inference in
13        // logic to execute upon completion of intent inference
14    })

when initialized input audio can be processed using manager.start(). The processing can be interrupted using manager.stop().

Low-Level API

Picovoice.swift provides an API for passing audio from your own audio pipeline into the Picovoice Platform for wake word detection and intent inference.

o construct an instance, you'll need to provide a Porcupine keyword file (.ppn), a Rhino context file (.rhn) and callbacks for when the wake word is detected and an inference is made. Sensitivity and model parameters are optional

1import Picovoice
2
3let accessKey = "${ACCESS_KEY}" // obtained from Picovoice Console (https://console.picovoice.ai/)
4
5do {
6    let picovoice = try Picovoice(
7        accessKey: accessKey,
8        keywordPath: "/path/to/keyword.ppn",
9        porcupineSensitivity: 0.4,
10        porcupineModelPath: "/path/to/porcupine/model.pv"
11        onWakeWordDetection: {
12            // logic to execute upon detection of wake word
13        },
14        contextPath: "/path/to/context.rhn",
15        rhinoSensitivity: 0.7,
16        rhinoModelPath: "/path/to/rhino/model.pv"
17        onInference: { inference in
18            // logic to execute upon completion of intent inference
19        })
20} catch { }

Once initialized, picovoice can be used to process incoming audio. The underlying logic of the class will handle switching between wake word detection and intent inference, as well as invoking the associated events.

1func getNextAudioFrame() -> [Int16] {
2    // .. get audioFrame
3    return audioFrame;
4}
5
6while (true) {
7    do {
8        try picovoice.process(getNextAudioFrame());
9    } catch { }
10}

Once you're done with an instance of Picovoice you can force it to release its native resources rather than waiting for the garbage collector:

1picovoice.delete();

Web

Install the Web SDK using yarn:

1yarn add @picovoice/picovoice-web

or using npm:

1npm install --save @picovoice/picovoice-web

Create an instance of the engine using PicovoiceWorker and run on an audio input stream:

1import { PicovoiceWorker } from "@picovoice/picovoice-web";
2
3function wakeWordCallback(detection: PorcupineDetection) {
4  console.log(`Porcupine detected keyword: ${detection.label}`);
5}
6
7function inferenceCallback(inference: RhinoInference) {
8  if (inference.isFinalized) {
9    if (inference.isUnderstood) {
10      console.log(inference.intent)
11      console.log(inference.slots)
12    }
13  }
14}
15
16function getAudioData(): Int16Array {
17  ... // function to get audio data
18  return new Int16Array();
19}
20
21const picovoice = await PicovoiceWorker.create(
22  "${ACCESS_KEY}",
23  keyword,
24  wakeWordCallback,
25  porcupineModel,
26  context,
27  inferenceCallback,
28  rhinoModel
29);
30
31for (; ;) {
32  picovoice.process(getAudioData());
33  // break on some condition
34}

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

When done, release the resources allocated to Picovoice using picovoice.release().

Angular

1yarn add @picovoice/picovoice-angular @picovoice/web-voice-processor

(or)

1npm install @picovoice/picovoice-angular @picovoice/web-voice-processor

1import { Subscription } from "rxjs"
2import { PicovoiceService } from "@picovoice/picovoice-angular"
3
4...
5
6constructor(private picovoiceService: PicovoiceService) {
7  this.wakeWordDetectionSubscription = picovoiceService.wakeWordDetection$.subscribe(
8          (wakeWordDetection: PorcupineDetection) => {
9            this.inference = null;
10            this.wakeWordDetection = wakeWordDetection;
11          }
12  );
13
14  this.inferenceSubscription = picovoiceService.inference$.subscribe(
15          (inference: RhinoInference) => {
16            this.wakeWordDetection = null;
17            this.inference = inference;
18          }
19  );
20
21  this.contextInfoSubscription = picovoiceService.contextInfo$.subscribe(
22          (contextInfo: string | null) => {
23            this.contextInfo = contextInfo;
24          }
25  );
26
27  this.isLoadedSubscription = picovoiceService.isLoaded$.subscribe(
28          (isLoaded: boolean) => {
29            this.isLoaded = isLoaded;
30          }
31  );
32  this.isListeningSubscription = picovoiceService.isListening$.subscribe(
33          (isListening: boolean) => {
34            this.isListening = isListening;
35          }
36  );
37  this.errorSubscription = picovoiceService.error$.subscribe(
38          (error: string | null) => {
39            this.error = error;
40          }
41  );
42}
43
44async ngOnInit() {
45    try {
46      await this.picovoiceService.init(
47              accessKey,
48              porcupineKeyword,
49              porcupineModel,
50              rhinoContext,
51              rhinoModel
52      );
53    }
54    catch (error) {
55      console.error(error)
56    }
57}
58
59ngOnDestroy() {
60  this.wakeWordDetectionSubscription.unsubscribe();
61  this.inferenceSubscription.unsubscribe();
62  this.contextInfoSubscription.unsubscribe();
63  this.isLoadedSubscription.unsubscribe();
64  this.isListeningSubscription.unsubscribe();
65  this.errorSubscription.unsubscribe();
66  this.picovoiceService.release();
67}

React

1yarn add @picovoice/picovoice-react @picovoice/web-voice-processor

(or)

1npm install @picovoice/picovoice-react @picovoice/web-voice-processor

1import { usePicovoice } from '@picovoice/picovoice-react';
2
3function App(props) {
4  const {
5    wakeWordDetection,
6    inference,
7    contextInfo,
8    isLoaded,
9    isListening,
10    error,
11    init,
12    start,
13    stop,
14    release,
15  } = usePicovoice();
16
17  const initEngine = async () => {
18    await init(
19            ${ACCESS_KEY},
20            porcupineKeyword,
21            porcupineModel,
22            rhinoContext,
23            rhinoModel
24    );
25    await start();
26  }
27
28  useEffect(() => {
29    if (wakeWordDetection !== null) {
30      console.log(`Picovoice detected keyword: ${wakeWordDetection.label}`);
31    }
32  }, [wakeWordDetection])
33
34  useEffect(() => {
35    if (inference !== null) {
36      if (inference.isUnderstood) {
37        console.log(inference.intent)
38        console.log(inference.slots)
39      }
40    }
41  }, [inference])
42}

Vue

1yarn add @picovoice/picovoice-vue @picovoice/web-voice-processor

(or)

1npm install @picovoice/picovoice-vue @picovoice/web-voice-processor

1<script lang='ts'>
2import { usePicovoice } from '@picovoice/picovoice-vue';
3
4export default {
5  data() {
6    const {
7      state,
8      init,
9      start,
10      stop,
11      release
12    } = usePicovoice();
13
14    init(
15      ${ACCESS_KEY},
16      {
17        label: "Picovoice",
18        publicPath: "picovoice_wasm.ppn",
19      },
20      { publicPath: "porcupine_params.pv" },
21      { publicPath: "clock_wasm.rhn" },
22      { publicPath: "rhino_params.pv" },
23    );
24
25    return {
26      state,
27      start,
28      stop,
29      release
30    }
31  },
32  watch: {
33    "state.wakeWordDetection": function(wakeWord) {
34      if (wakeWord !== null) {
35        console.log(wakeWord)
36      }
37    },
38    "state.inference": function(inference) {
39      if (inference !== null) {
40        console.log(inference)
41      }
42    },
43    "state.contextInfo": function(contextInfo) {
44      if (contextInfo !== null) {
45        console.log(contextInfo)
46      }
47    },
48    "state.isLoaded": function(isLoaded) {
49      console.log(isLoaded)
50    },
51    "state.isListening": function(isListening) {
52      console.log(isListening)
53    },
54    "state.error": function(error) {
55      console.error(error)
56    },
57  },
58  onBeforeDestroy() {
59    this.release();
60  },
61};
62</script>

Rust

To add the picovoice library into your app, add picovoice to your app's Cargo.toml manifest:

1[dependencies]
2picovoice = "*"

To create an instance of the engine with default parameters, use the PicovoiceBuilder function. You must provide a Porcupine keyword file, a wake word detection callback function, a Rhino context file and an inference callback function. You must then make a call to init():

1use picovoice::{rhino::RhinoInference, PicovoiceBuilder};
2
3let wake_word_callback = || {
4    // let user know wake word detected
5};
6let inference_callback = |inference: RhinoInference| {
7    if inference.is_understood {
8        let intent = inference.intent.unwrap();
9        let slots = inference.slots;
10        // add code to take action based on inferred intent and slot values
11    } else {
12        // add code to handle unsupported commands
13    }
14};
15
16let mut picovoice = PicovoiceBuilder::new(
17    keyword_path,
18    wake_word_callback,
19    context_path,
20    inference_callback,
21).init().expect("Failed to create picovoice");

Upon detection of wake word defined by keyword_path it starts inferring user's intent from the follow-on voice command within the context defined by the file located at context_path. keyword_path is the absolute path to Porcupine wake word engine keyword file (with .ppn suffix). context_path is the absolute path to Rhino Speech-to-Intent engine context file (with .rhn suffix). wake_word_callback is invoked upon the detection of wake phrase and inference_callback is invoked upon completion of follow-on voice command inference.

When instantiated, valid sample rate can be obtained via sample_rate(). Expected number of audio samples per frame is frame_length(). The engine accepts 16-bit linearly-encoded PCM and operates on single-channel audio:

1fn next_audio_frame() -> Vec<i16> {
2    // get audio frame
3}
4
5loop {
6    picovoice.process(&next_audio_frame()).expect("Picovoice failed to process audio");
7}

C

Picovoice is implemented in ANSI C and therefore can be directly linked to C applications. Its public header file (sdk/c/include/pv_picovoice.h) contains relevant information. An instance of the Picovoice object can be constructed as follows.

1const char* ACCESS_KEY = "${ACCESS_KEY}"; // AccessKey string obtained from [Picovoice Console](https://console.picovoice.ai/)
2
3const char *porcupine_model_path = ... // Available at resources/porcupine/lib/common/porcupine_params.pv
4const char *keyword_path = ...
5const float porcupine_sensitivity = 0.5f;
6
7const char *rhino_model_path = ... // Available at resources/rhino/lib/common/rhino_params.pv
8const char *context_path = ...
9const float rhino_sensitivity = 0.5f;
10const bool require_endpoint = true;
11
12static void wake_word_callback(void) {
13    // take action upon detection of wake word
14}
15
16static void inference_callback(pv_inference_t *inference) {
17    // `inference` exposes three immutable properties:
18    // (1) `IsUnderstood`
19    // (2) `Intent`
20    // (3) `Slots`
21
22    // take action based on inferred intent
23    pv_inference_delete(inference);
24}
25
26pv_picovoice_t *handle = NULL;
27
28pv_status_t status = pv_picovoice_init(
29        access_key,
30        porcupine_model_path,
31        keyword_path,
32        porcupine_sensitivity,
33        wake_word_callback,
34        rhino_model_path,
35        context_path,
36        rhino_sensitivity,
37        require_endpoint,
38        inference_callback,
39        &handle);
40
41if (status != PV_STATUS_SUCCESS) {
42    // error handling logic
43}

Sensitivity is the parameter that enables developers to trade miss rate for false alarm. It is a floating-point number within [0, 1]. A higher sensitivity reduces miss rate (false reject rate) at cost of increased false alarm rate.

handle is an instance of Picovoice runtime engine that detects utterances of the wake phrase provided by keyword_path. Upon detection of wake word it starts inferring user's intent from the follow-on voice command within the context defined in context_path. wake_word_callback is invoked upon the detection of wake phrase and inference_callback is invoked upon completion of follow-on voice command inference.

Picovoice accepts single channel, 16-bit PCM audio. The sample rate can be retrieved using pv_sample_rate(). Finally, Picovoice accepts input audio in consecutive chunks (aka frames) the length of each frame can be retrieved using pv_porcupine_frame_length().

1extern const int16_t *get_next_audio_frame(void);
2
3while (true) {
4    const int16_t *pcm = get_next_audio_frame();
5    const pv_status_t status = pv_picovoice_process(handle, pcm);
6    if (status != PV_STATUS_SUCCESS) {
7        // error handling logic
8    }
9}

Finally, when done be sure to release the acquired resources.

1pv_picovoice_delete(handle);

Microcontroller

Picovoice is implemented in ANSI C and therefore can be directly linked to embedded C projects. Its public header file contains relevant information. An instance of the Picovoice object can be constructed as follows:

1#define MEMORY_BUFFER_SIZE ...
2static uint8_t memory_buffer[MEMORY_BUFFER_SIZE] __attribute__((aligned(16)));
3
4static const uint8_t *keyword_array = ...
5const float porcupine_sensitivity = 0.5f
6
7static void wake_word_callback(void) {
8    // logic to execute upon detection of wake word
9}
10
11static const uint8_t *context_array = ...
12const float rhino_sensitivity = 0.75f
13
14static void inference_callback(pv_inference_t *inference) {
15    // `inference` exposes three immutable properties:
16    // (1) `IsUnderstood`
17    // (2) `Intent`
18    // (3) `Slots`
19    // ..
20    pv_inference_delete(inference);
21}
22
23pv_picovoice_t *handle = NULL;
24
25const pv_status_t status = pv_picovoice_init(
26        MEMORY_BUFFER_SIZE,
27        memory_buffer,
28        sizeof(keyword_array),
29        keyword_array,
30        porcupine_sensitivity,
31        wake_word_callback,
32        sizeof(context_array),
33        context_array,
34        rhino_sensitivity,
35        inference_callback,
36        &handle);
37
38if (status != PV_STATUS_SUCCESS) {
39    // error handling logic
40}

Sensitivity is the parameter that enables developers to trade miss rate for false alarm. It is a floating-point number within [0, 1]. A higher sensitivity reduces miss rate (false reject rate) at cost of increased false alarm rate.

handle is an instance of Picovoice runtime engine that detects utterances of wake phrase defined in keyword_array. Upon detection of wake word it starts inferring user's intent from the follow-on voice command within the context defined in context_array. wake_word_callback is invoked upon the detection of wake phrase and `inference