@surebob/ollama-local-provider

A client-side Ollama provider for the Vercel AI SDK that makes API calls directly from the browser to your local Ollama instance.

Features

• 🌐 Browser-side API calls to local Ollama
• 🔄 Full streaming support
• 🛠 Compatible with Vercel AI SDK
• 🎯 Zero server-side Ollama calls
• 📦 Easy integration with Next.js and other frameworks

Installation

1npm install @surebob/ollama-local-provider

Usage

Basic Usage

1// In your Next.js page or component
2'use client';
3
4import { useChat } from 'ai/react';
5import { ollama } from '@surebob/ollama-local-provider';
6
7export default function Chat() {
8  const { messages, input, handleInputChange, handleSubmit } = useChat({
9    api: '/api/chat',
10    initialMessages: [],
11  });
12
13  return (
14    <div>
15      {messages.map((m) => (
16        <div key={m.id}>{m.content}</div>
17      ))}
18      <form onSubmit={handleSubmit}>
19        <input value={input} onChange={handleInputChange} />
20        <button type="submit">Send</button>
21      </form>
22    </div>
23  );
24}

API Route Setup

1// app/api/chat/route.ts
2import { StreamingTextResponse } from 'ai';
3import { ollama } from '@surebob/ollama-local-provider';
4
5export async function POST(req: Request) {
6  const { messages } = await req.json();
7  
8  const model = ollama('deepscaler:1.5b', {
9    temperature: 0.7,
10    top_p: 0.9,
11    num_ctx: 4096,
12    repeat_penalty: 1.1,
13  });
14
15  const response = await model.doStream({
16    inputFormat: 'messages',
17    mode: { type: 'regular' },
18    prompt: messages,
19  });
20
21  return new StreamingTextResponse(response.stream);
22}

Requirements

• Ollama running locally on port 11434
• Vercel AI SDK
• Next.js (or other framework with streaming support)

How It Works

This provider uses the ollama/browser client to make API calls directly from the browser to your local Ollama instance. This means:

1. When running locally:

   • Browser → Local Ollama (localhost:11434)

2. When deployed:

   • Browser → User's Local Ollama (localhost:11434)
   • Server never handles Ollama calls

License

MIT