PackFS

Semantic filesystem operations for LLM agent frameworks with natural language understanding.

FYI: This project is, in addition to its actual regular scope, one where I'm testing the use of LLM agents to manage a product AND another independent agent to build something using this package. Their back and forth may cause more feature churn than you're up for. You've been warned.

Overview

PackFS is a TypeScript/Node.js library that provides intelligent filesystem operations through semantic understanding and natural language processing. Built specifically for LLM agent frameworks, it replaces traditional POSIX-style operations with intent-based semantic operations that understand what agents want to accomplish.

Inspired by the research in "LLM-based Semantic File System for Large Codebases" and the Python LSFS implementation, PackFS brings semantic filesystem capabilities to the TypeScript/Node.js ecosystem with a focus on agent framework integration.

Features

• 🧠 Semantic Operations: Natural language file operations - "create a config file", "find all documentation", "organize by topic"
• 🤖 LLM Agent Integration: Native Mastra integration with tool factory pattern, plus support for LangChain.js, LlamaIndex.TS, and KaibanJS
• 📝 Intent-Based Interface: Replace traditional read/write with semantic operations like accessFile, updateContent, discoverFiles
• 🔍 Intelligent Indexing: Automatic keyword extraction and semantic file indexing for fast discovery
• 🔄 Backward Compatibility: Adapter pattern allows gradual migration from traditional filesystem operations
• 💾 Multiple Backends: Memory and persistent disk backends with semantic indexing
• 🔒 Security Controls: Path validation, sandboxing, and permission systems
• 🏢 Dynamic Working Directory: Runtime-configurable paths for multi-project support (NEW)

🚀 For LLM Agents: Quick Start Guide

New to PackFS? Multiple projects have reported agents going down wrong usage paths.

👉 LLM Agent Usage Guide - Copy-paste examples, common patterns, and pitfall solutions specifically designed for AI agents.

Installation

1npm install packfs-core

Quick Start

Primary API: Natural Language Operations

1import { createFileSystem } from 'packfs-core';
2
3// Initialize filesystem with one line
4const fs = createFileSystem('/workspace');
5
6// Natural language file operations - the primary interface for LLMs
7await fs.executeNaturalLanguage('Create a config.json file with default database settings');
8
9await fs.executeNaturalLanguage('Find all documentation files and organize them in a docs folder');
10
11// Semantic search
12const results = await fs.findFiles('configuration files', {
13  searchType: 'semantic',
14  maxResults: 5,
15});
16
17// The result includes interpreted intent and confidence
18const result = await fs.executeNaturalLanguage('Backup all JavaScript files modified today');
19console.log(`Operation confidence: ${result.confidence}`);
20console.log(`Interpreted as: ${JSON.stringify(result.interpretedIntent)}`);

Semantic API: Intent-Based Operations

1// For more control, use the semantic backend directly
2const backend = fs.getSemanticBackend();
3
4// Intent-based file access
5const configResult = await backend.accessFile({
6  purpose: 'read',
7  target: { path: 'config.json' },
8  preferences: { includeMetadata: true },
9});
10
11// Semantic file discovery
12const docs = await backend.discoverFiles({
13  purpose: 'search_semantic',
14  target: { semanticQuery: 'API documentation' },
15  options: { maxResults: 10 },
16});
17
18// Intelligent file organization
19await backend.organizeFiles({
20  purpose: 'group_semantic',
21  target: { directory: 'organized' },
22  criteria: 'Group files by their semantic purpose',
23});

Legacy API: POSIX-Style (Backward Compatible)

1// Traditional operations are still available for gradual migration
2const fs = createFileSystem('/workspace');
3
4// These work but consider using natural language instead
5await fs.writeFile('file.txt', 'content');
6const content = await fs.readFile('file.txt');
7await fs.mkdir('new-folder');
8const files = await fs.readdir('.');

Framework Integrations

Mastra (Native Integration)

PackFS provides native Mastra integration with two patterns:

Option 1: Semantic Filesystem Tool (Recommended)

1import { createMastraSemanticFilesystemTool } from 'packfs-core';
2
3// Create a unified semantic filesystem tool
4// IMPORTANT: workingDirectory is REQUIRED
5const packfsTool = createMastraSemanticFilesystemTool({
6  workingDirectory: '/path/to/your/project', // REQUIRED - must be an absolute path
7  // OR provide a pre-configured filesystem:
8  // filesystem: new DiskSemanticBackend('/path/to/project'),
9  security: {
10    maxFileSize: 5 * 1024 * 1024, // 5MB limit
11    allowedExtensions: ['.md', '.txt', '.json', '.js', '.ts'],
12    forbiddenPaths: ['node_modules', '.git', '.env'],
13  },
14});
15
16// Use with Mastra agents
17const agent = new Agent({
18  name: 'file-assistant',
19  tools: { packfsTool },
20});

Option 2: Tool Suite Pattern

1import { createPackfsTools } from 'packfs-core/integrations/mastra';
2
3// Create separate tools for different operations
4const tools = createPackfsTools({
5  rootPath: '/project', // REQUIRED - resolves to workingDirectory
6  permissions: ['read', 'write', 'search'],
7  security: {
8    maxFileSize: 5 * 1024 * 1024,
9    allowedExtensions: ['.md', '.txt', '.json', '.js', '.ts'],
10    blockedPaths: ['node_modules', '.git'],
11  },
12});
13
14// Use individual tools in your Mastra agent
15const fileContent = await tools.fileReader.execute({
16  context: {
17    purpose: 'read',
18    target: { path: '/project/README.md' },
19  },
20  runtimeContext, // Mastra's runtime context
21});
22
23const searchResults = await tools.fileSearcher.execute({
24  context: {
25    purpose: 'search_content',
26    target: { path: '/project' },
27    options: { pattern: 'API.*endpoint' },
28  },
29  runtimeContext,
30});
31
32// Create new files with validation
33await tools.fileWriter.execute({
34  context: {
35    purpose: 'create',
36    target: { path: '/project/config.json' },
37    content: JSON.stringify({ database: { host: 'localhost' } }),
38  },
39  runtimeContext,
40});

Semantic Operations: The tools support intent-based operations:

• AccessIntent: read, metadata, exists
• DiscoverIntent: list, search_content, search_semantic
• UpdateIntent: create, update, append, delete

Advanced Configuration:

1const tools = createPackfsTools({
2  rootPath: '/workspace',
3  permissions: ['read', 'write', 'search', 'list'],
4  security: {
5    maxFileSize: 10 * 1024 * 1024, // 10MB
6    allowedExtensions: ['.md', '.txt', '.json', '.js', '.ts', '.py'],
7    blockedPaths: ['node_modules', '.git', '.env', 'secrets'],
8    rateLimiting: {
9      maxRequests: 100,
10      windowMs: 60000, // 1 minute
11    },
12  },
13  semantic: {
14    enableRelationships: true,
15    chunkSize: 2000,
16    overlapSize: 200,
17    relevanceThreshold: 0.5,
18  },
19});
20
21// Generated tools: fileReader, fileWriter, fileSearcher, fileLister
22// Each tool includes automatic security validation and error handling

Dynamic Working Directory (NEW):

1// Access files from different projects without reinitializing
2const mastraTool = createMastraSemanticFilesystemTool({
3  workingDirectory: '/main/workspace',
4  filesystem: backend
5});
6
7// Read from project A
8const resultA = await mastraTool.execute({
9  operation: 'access',
10  purpose: 'read',
11  target: { path: 'context.md' },
12  workingDirectory: '/projects/project-a'  // Runtime override
13});
14
15// Read from project B
16const resultB = await mastraTool.execute({
17  operation: 'access',
18  purpose: 'read',
19  target: { path: 'context.md' },
20  workingDirectory: '/projects/project-b'  // Different project
21});
22
23// Also works with natural language
24const result = await mastraTool.execute({
25  naturalLanguageQuery: 'find all documentation files',
26  workingDirectory: '/projects/project-c'
27});

LangChain.js

1import { createLangChainSemanticFilesystemTool } from 'packfs-core';
2
3const tool = createLangChainSemanticFilesystemTool({
4  filesystem: new MemorySemanticBackend(),
5  workingDirectory: '/project',
6  langchain: { verbose: true },
7});
8
9// LangChain DynamicTool compatible
10const response = await tool.func('read the configuration file');
11console.log(response); // File content as string

LlamaIndex.TS

1import { createLlamaIndexSemanticFilesystemTool } from 'packfs-core';
2
3const tool = createLlamaIndexSemanticFilesystemTool({
4  filesystem: new MemorySemanticBackend(),
5  workingDirectory: '/project',
6});
7
8// LlamaIndex FunctionTool compatible
9const result = await tool.call({
10  action: 'search',
11  searchTerm: 'API documentation',
12});

KaibanJS (Multi-Agent Systems)

1import { createKaibanSemanticFilesystemTool } from 'packfs-core';
2
3const tool = createKaibanSemanticFilesystemTool({
4  filesystem: new MemorySemanticBackend(),
5  workingDirectory: '/shared',
6  kaiban: {
7    agentId: 'file-manager',
8    enableStatePersistence: true,
9  },
10});
11
12// Multi-agent collaboration
13const result = await tool.handler({
14  action: 'write',
15  path: '/shared/team-notes.md',
16  content: 'Team meeting notes',
17  collaboration: {
18    shareWith: ['agent-001', 'agent-002'],
19    notifyAgents: true,
20  },
21});

Semantic File Discovery

1import { DiskSemanticBackend } from 'packfs-core';
2
3// Persistent semantic indexing
4const fs = new DiskSemanticBackend({
5  rootPath: '/project',
6  indexPath: '.packfs/semantic-index.json',
7});
8
9// Find files by semantic meaning
10const configFiles = await fs.discoverFiles({
11  purpose: 'search_semantic',
12  target: { semanticQuery: 'configuration and settings' },
13});
14
15// Find files by content patterns
16const apiDocs = await fs.discoverFiles({
17  purpose: 'search_content',
18  target: { pattern: 'API|endpoint|route' },
19});
20
21// Organize files by topic
22const organized = await fs.organizeFiles({
23  purpose: 'group_semantic',
24  source: { path: '/project/docs' },
25  destination: { path: '/project/organized' },
26  options: { groupBy: 'topic' },
27});

Intelligent Error Recovery

PackFS reduces LLM token usage by providing smart suggestions when file operations fail:

1// When a file is not found
2const result = await fs.accessFile({
3  purpose: 'read',
4  target: { path: 'context-network/foundation/index.md' },
5});
6
7if (!result.success) {
8  console.log(result.message);
9  // Output:
10  // File not found: context-network/foundation/index.md
11  //
12  // Suggestions:
13  // • Directory listing of 'context-network/foundation'
14  //   Files: core.md, principles.md, objectives.md ... and 3 more
15  // • Alternative paths that exist
16  //   Try: context-network/index.md
17  // • Found 'index.md' in other locations
18  //   Found in: docs/index.md, src/index.md
19}
20
21// When search returns no results
22const searchResult = await fs.discoverFiles({
23  purpose: 'search_content',
24  target: { semanticQuery: 'quantum blockchain AI' },
25});
26
27if (searchResult.files.length === 0 && searchResult.suggestions) {
28  // Suggestions include:
29  // - Alternative search terms
30  // - Individual term searches: 'quantum', 'blockchain', 'AI'
31}

This feature helps LLMs find what they're looking for without multiple retry attempts, significantly reducing API costs and improving response times.

API Reference

Semantic Interface

• SemanticFileSystemInterface - Abstract base for semantic operations
• MemorySemanticBackend - In-memory semantic filesystem
• DiskSemanticBackend - Persistent semantic filesystem with indexing

Semantic Operations

• accessFile(intent) - Read, preview, or verify file existence
• updateContent(intent) - Create, append, overwrite, or patch files
• discoverFiles(intent) - List, find, or search files semantically
• organizeFiles(intent) - Move, copy, or group files by semantic criteria
• removeFiles(intent) - Delete files or directories
• interpretNaturalLanguage(query) - Convert natural language to intents

Traditional Interface (Backward Compatible)

• FileSystemInterface - Abstract base class for traditional operations
• MemoryBackend - In-memory storage
• DiskBackend - Local filesystem storage
• SecurityEngine - Security validation and access control

Framework Integrations

Native Mastra Integration

• createPackfsTools(config) - Native tool factory for Mastra agents
• MastraSecurityValidator - Security validation with path restrictions and rate limiting
• Intent-based schemas: AccessIntent, DiscoverIntent, UpdateIntent

Legacy Framework Integrations

• createLangChainSemanticFilesystemTool - LangChain.js integration
• createLlamaIndexSemanticFilesystemTool - LlamaIndex.TS integration
• createKaibanSemanticFilesystemTool - KaibanJS multi-agent integration

Development

1# Install dependencies
2npm install
3
4# Build the package
5npm run build
6
7# Run tests
8npm test
9
10# Run linting
11npm run lint
12
13# Format code
14npm run format

Semantic Design Philosophy

PackFS is built around semantic understanding rather than traditional filesystem operations:

• Intent-Based Operations: Instead of fs.readFile(), use semantic operations like accessFile({ purpose: 'read' })
• Natural Language Support: Agents can use plain English - "create a config file" instead of complex API calls
• Contextual Understanding: Operations understand the semantic meaning of file content and organization
• Framework Native: Built specifically for LLM agent frameworks with their patterns and needs

Inspired by the LSFS research paper, PackFS implements the concept of semantic filesystem operations that better match how LLM agents think about and work with files.

Security Considerations

While semantic operations are the primary focus, PackFS maintains strong security:

• Path validation and normalization prevent traversal attacks
• Configurable sandbox restrictions
• File size limits and extension controls
• Permission-based access control

License

MIT

Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests to our GitHub repository.