useOnnxWeb

React hook for inference with ONNX-Runtime web

useOnnxWeb manages an ONNX inference session using a model file that you provide, and accepts additional options. It will re-initialize if the model file is changed. Only one session should exist in the app at a time.

It returns a single function, runInference, which accepts data input and options, and returns a promise that resolves to inference results. The input properties are determined when you create the model file.

Executing runInference while a previous run is in progress will throw an error, so you may want to await or catch:

Uncaught (in promise) Error: output [...] already has value: ...

Installation

1$ npm install use-onnx-web

Usage

1import {
2  useOnnxWeb,
3  // Tensor class for formatting input/output data.
4  Tensor
5  // For typing assistance.
6  InferenceSession
7  RunInference
8} from "use-onnx-web";
9
10const App = () => {
11  // This model performs matrix multiplication.
12  const { runInference } = useOnnxWeb("./model.onnx");
13
14  useEffect(() => {
15    const feeds = {
16      a: new Tensor("float32",
17        [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
18        [3, 4]
19      ),
20      b: new Tensor("float32",
21        [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120],
22        [4, 3]
23      ),
24    };
25
26    runInference(feeds).then((result) => {
27      console.log(result.c.data);
28      // >>> Float32Array(9) [ ... ]
29    });
30  }, []);

The model.onnx file used above is included in this repository, but it could be created in PyTorch like this:

1# set the trained model to inference mode
2model.eval()
3
4# single tensor or tuple for multiple inputs
5trace_input = (
6  torch.randn(3, 4),
7  torch.randn(4, 3)
8)
9
10torch.onnx.export(
11  model,
12  "model.onnx", # where to save the model
13  trace_input, # trace input
14  input_names=["a, b"], # the model's input names
15  output_names=["c"], # the model's output names
16  export_params=True, # store the trained parameter weights inside the model file
17)