Unofficial Ollama JS (browser & node) library

Ollama is an awesome piece of llama software that allows running AI models locally and interacting with them via an API. This API is wrapped nicely in this library.

Originally based on ollama api docs – commit

A simple wrapper for prompting your local ollama API or using the chat format for more structure.

• It is a wrap around fetch to make it easier to use the ollama api.
• When prompting, it manages the context vector for continuous conversations.
• Use the stream or non-stream version of the prompt or chat method to get the response.

Works in node and the browser.

The only dependency (for node) is node-fetch

If you use node v17.5.0+ and the node --experimental-fetch filname.js flag, you no longer need to install fetch (finally)

IIEF & ES6 modules are available for node and the browser.

No require

This library is written with ES6 modules. There is no require. Use this library in NodeJS by either using the .mjs file extension or "type": "module" in your package.json file.

Installation

1npm install ollama-js-client

Via CDN

1<script src="https://cdn.jsdelivr.net/npm/ollama-js-client/dist/browser/iife/ollama-js.global.js"></script>
2
3<script
4  type="module"
5  src="https://cdn.jsdelivr.net/npm/ollama-js-client/dist/browser/index.js"
6></script>

Import the module

1// use in the browser as a type="module" or in node with modules enabled (mjs)
2import Ollama from "ollama-js-client";
3
4// use global in the browser with script tag
5const Ollama = window.OllamaJS;

There are two ways to use the module:

1. Prompt with a text string.

2. "Chat" with an array of messages (like the OpenAI GPT API).

Prompt is a simplfied version of chat, that operates on a context vector that can be kept between calls (this library manages this automatically).

Chat has moved away from the context vector and now operates on an array of messages. This is a nicer way to interact with the API, allowing role (system,assistant,user) and the management of conversations in a more structured way.

There are two ways to receive the response

1. Non-streaming - The whole response is returned at once.

2. Streaming - The response is returned in chunks.

Depending on the use case, you can use the stream or non-stream version of the prompt and chat method. If you don't mind waiting for the whole response, use the non-stream version. If you want to get the response in chunks, use the stream version.

Stream is useful when building UI, so the user can see the responses as they come in.

---

Chat with messages

Using the chat or chat_request method and the OpenAI GPT API message format.

1import Ollama from "ollama-js-client";
2
3// default stream version
4const ollama_instance = new Ollama({
5  model: "llama3",
6  url: "http://127.0.0.1:11434/api/",
7});
8ollama_instance.chat([
9  { role: "system", content: "You are a llama AI" },
10  { role: "assistant", content: "Hello, I am your llama AI friend." },
11  { role: "user", content: "That's funny" },
12], (error, response) => {
13  if (error) {
14    console.error(error);
15  } else {
16    console.log(response);
17  }
18});
19
20// non-stream
21const response = await new Ollama({
22  model: "llama3",
23  url: "http://127.0.0.1:11434/api/",
24}).chat_request([
25  { role: "user", content: "Hello my ai friend" },
26  { role: "assistant", content: "Hello, I am your llama AI friend." },
27  { role: "user", content: "That's funny" },
28]);

---

Prompt with a text

Using the prompt or prompt_stream method

Memory or no memory

Each prompt is a fetch request. The response (last response if stream) contains a context vector. The context vector is used to keep the conversation going. The context is sent with the next prompt request. The context is cleared with the clear method.

Keep the conversation going

If you want to keep the conversation going with the context, you simple have to keep the reference to the instance.

1import Ollama from "ollama-js-client";
2const ollama_instance = new Ollama({
3  model: "llama3",
4  url: "http://127.0.0.1:11434/api/",
5});
6ollama_instance.prompt("Hello my llama AI friend");
7// instance will keep the context vector

Who needs memory?

If you don't want to keep the context vector, you can just create a new instance for each prompt or clear the context vector with the .clear() method.

1import Ollama from "ollama-js-client";
2
3const response = await new Ollama({
4  model: "llama3",
5  url: "http://127.0.0.1:11434/api/",
6}).prompt("Hello my ai friend");
7// context vector is lost
8
9// or clear the instance context vector
10const ollama_instance = await new Ollama({
11  model: "llama3",
12  url: "http://127.0.0.1:11434/api/",
13});
14
15ollama_instance.prompt("Hello my llama AI friend");
16
17ollama_instance.clear();
18// context vector is lost/reset

No memory short sugar syntax

This is the shortest way to use the module. It will create a new Ollama instance and call prompt on it. It will not carry the context vector between calls, as the "reference" to the instance is lost.

The From connection string is

<model>@<url>

1const response = await Ollama.from("llama3@http://127.0.0.1:11434/api/").prompt(
2  "Hello my ai friend"
3);
4console.log(response); // "I am not your friend, buddy."

Basic constructor example

Add more options to the constructor to get more control over the instance.

1import Ollama from "ollama-js-client";
2
3const response = await new Ollama({
4  model: "llama3",
5  url: "http://127.0.0.1:11434/api/",
6  options: {
7    temperature: 1,
8    top_p: 0.9,
9    top_k: 5,
10    system: "system prompt to (overrides what is defined in the Modelfile",
11    template:
12      "the full prompt or prompt template (overrides what is defined in the Modelfile)",
13  },
14}).prompt("Hello my ai friend");

Continue conversation example (prompt)

Keep the reference to the instance to carry the conversation vector (context) between calls.

1import Ollama from "ollama-js-client";
2const 🦙 = new Ollama({
3    model:"llama3",
4    url:"http://127.0.0.1:11434/api/"
5})
6const response = await 🦙.prompt("Hello my ai friend")
7// next prompt will include the previous context vector
8const next_response = await 🦙.prompt("That's funny")
9// clear the context vector
10🦙.clear()

Prompt stream example

Use the stream version of the prompt method to get the response in chunks.

1import Ollama from "ollama-js-client";
2const 🦙 = new Ollama({
3    model:"llama3",
4    url:"http://127.0.0.1:11434/api/",
5})
6const responded = []
7// callback will be called with response chunks until error or response.done is true
8const on_response = (error,response) => {
9    if (error) {
10        console.error(error)
11    }
12    else if (response.done) {
13        console.log(responded.join("\n"))
14    }
15    else {
16        responded.push(response)
17    }
18}
19🦙.prompt_stream("Hello",on_response)

Abort request example

Abort the request by calling the abort method on the instance.

1import Ollama from "ollama-js-client";
2const 🦙 = new Ollama({
3    model:"llama3",
4    url:"http://127.0.0.1:11434/api/",
5})
6let responded = ""
7// callback will be called with response chunks until error or response.done is true
8const on_response = (error,response) => {
9    if (error) {
10        console.error(error)
11    }
12    else if (response.done) {
13        console.log(responded)
14    }
15    else {
16        responded += response
17        if (responded.length > 10) {
18            // abort the request on silly grounds
19            🦙.abort()
20        }
21    }
22}
23await 🦙.prompt_stream("Hello",on_response)

Use in the browser

1<head>
2  <script src="https://cdn.jsdelivr.net/npm/ollama-js-client/browser/iife/ollama-js.global.js"></script>
3</head>
4<body>
5  <div id="output"></div>
6  <textarea id="input"></textarea>
7  <script>
8    const Ollama = window.OllamaJS;
9
10    function setup() {
11        const input = document.getElementById("input")
12        const output = document.getElementById("output")
13
14        const 🦙 = new Ollama({
15            model:"llama3",
16            url:"http://127.0.0.1:11434/api/",
17        })
18
19        const on_response = (error,response) => {
20            if (error) {
21                console.error(error)
22            }
23            else if (response.done) {
24                // done!
25            }
26            else {
27                output.innerHTML += response
28            }
29        }
30
31        input.addEventListener("keyup",async (event) => {
32            if (event.key === "Enter") {
33                await 🦙.prompt_stream(input.value,on_response)
34                input.value = ""
35            }
36        })
37    }
38    setup()
39  </script>
40</body>

---

Extra features

Use the expanded JSONparser

Getting JSON in return is awesome, but it is not always a pure JSON response, the JSONparser is a two step process, first it will try to parse the response as JSON, if that fails it will try to find the first valid json line and parse that until the end of a valid json object.

1import JSONparser from 'ollama-js-client/JSONparser';
2const 🦙 = new Ollama({
3    model:"llama3",
4    url:"http://127.0.0.1:11434/api/",
5})
6const response = await 🦙.prompt("Analyse this sentance and output the result in JSON: Hello there! It's nice to meet you. Is there anything I can help you with or would you like to chat? Please let me know if there's anything specific you'd like to talk about, and I'll do my best to assist you.")
7const json = JSONparser(response.response)
8console.log(json)
9/*
10{
11  type: 'response',
12  text: "Hello there! It's nice to meet you. Is there anything I can help you with or would you like to chat? Please let me know if there's anything specific you'd like to talk about, and I'll do my best to assist you.",
13  speaker: 'Assistant',
14  tone: 'neutral',
15  intent: {
16    helpfulness: 100,
17    positivity: 100,
18    respectfulness: 100,
19    honesty: 100
20  }
21}
22*/

Simple JSONStore

The JSONStore is a simple way to store and retrieve data from a file in node or localStorage in the browser. It is the basic storage requirement for exploring the ollama api and keeping some state.

1import JSONstore from 'ollama-js-client/JSONstore';
2const store = new JSONstore("store.json");
3
4store.set("key", "value");
5
6store.get("key"); // "value"
7
8store.has("key"); // true
9
10store.delete("key");
11
12// append to appendable values
13// arrays can be appended to
14// objects can be merged
15// strings can be concatenated
16// numbers can be added
17store.append("my_array", ["initial value"]);
18store.append("my_array", ["next value"]);
19console.log(store.get("my_array")); // ["initial value", "next value"]
20
21store.append("my_object", { key: "initial value" }});
22store.append("my_object", { another_key: "next value" }});
23console.log(store.get("my_object")); // { key: "initial value", another_key: "next value" }
24
25store.append("my_string", "initial value");
26store.append("my_string", "next value");
27console.log(store.get("my_string")); // "initial valuenext value"
28
29store.append("my_number", 1);
30store.append("my_number", 2);
31console.log(store.get("my_number")); // 3
32
33// always save the store to persist changes
34await store.save();
35
36// in the browser; download the store as a file
37store.download();
38
39// clear the store in memory
40store.clear();
41
42// destory the store file (truncates)
43await store.destory();

In node it is just a JSON file on disk, in the browser it is stored in localStorage and can be downloaded as a JSON file via store.download().

---

Happy llamaing!