@agenite/ollama
Ollama provider for Agenite, enabling seamless integration with local Ollama models.
Features
- 🚀 Support for all Ollama models (Llama 2, CodeLlama, Mistral, etc.)
- 🛠️ Function/tool calling support
- 🖼️ Image input support (with multimodal models)
- 📝 Rich content types (text, images, tool calls)
- 🔄 Streaming responses
- ⚙️ Extensive model parameter controls
Installation
npm install @agenite/ollama
Make sure you have Ollama installed and running locally.
Usage
Basic Chat
import { OllamaProvider } from '@agenite/ollama';
const provider = new OllamaProvider({
model: 'llama2',
host: 'http://localhost:11434',
temperature: 0.7,
});
const messages = [
{
role: 'user',
content: [{ type: 'text', text: 'What are the main features of Llama 2?' }],
},
];
const response = await provider.generate({ messages });
console.log(response.content[0].text);
Tool Usage
const calculatorTool = {
name: 'calculator',
description: 'Performs basic arithmetic operations',
parameters: {
type: 'object',
properties: {
operation: {
type: 'string',
enum: ['add', 'subtract', 'multiply', 'divide'],
},
a: { type: 'number' },
b: { type: 'number' },
},
required: ['operation', 'a', 'b'],
},
};
const response = await provider.generate({
messages: [
{
role: 'user',
content: [{ type: 'text', text: 'What is 123 multiplied by 456?' }],
},
],
tools: [calculatorTool],
});
Image Input
const provider = new OllamaProvider({
model: 'llava',
host: 'http://localhost:11434',
});
const response = await provider.generate({
messages: [
{
role: 'user',
content: [
{
type: 'image',
source: {
type: 'base64',
media_type: 'image/jpeg',
data: imageBase64,
},
},
{ type: 'text', text: 'What do you see in this image?' },
],
},
],
});
Configuration
The provider accepts the following configuration options:
interface OllamaConfig {
model: string; // Model name (e.g., 'llama2', 'codellama')
host?: string; // Ollama server URL (default: http://localhost:11434)
temperature?: number; // Generation temperature (0-1)
maxTokens?: number; // Maximum tokens to generate
systemPrompt?: string; // System prompt for all conversations
parameters?: { // Additional model parameters
mirostat?: number;
num_ctx?: number;
num_gpu?: number;
// ... and more
};
}
Examples
Check out the examples
directory for more detailed examples:
basic-chat.ts
: Simple text conversation
tool-usage.ts
: Calculator tool implementation
image-input.ts
: Image analysis with multimodal models
Development
# Install dependencies
npm install
# Build
npm run build
# Run tests
npm test
# Run examples
npx ts-node examples/basic-chat.ts
License
MIT