Simple key-value storage with support for multiple backends

Keyv provides a consistent interface for key-value storage across multiple backends via storage adapters. It supports TTL based expiry, making it suitable as a cache or a persistent key-value store.

Features

There are a few existing modules similar to Keyv, however Keyv is different because it:

• Isn't bloated
• Has a simple Promise based API
• Suitable as a TTL based cache or persistent key-value store
• Easily embeddable inside another module
• Works with any storage that implements the Map API
• Handles all JSON types plus Buffer
• Supports namespaces
• Wide range of efficient, well tested storage adapters
• Connection errors are passed through (db failures won't kill your app)
• Supports the current active LTS version of Node.js or higher

Usage

Install Keyv.

npm install --save keyv

By default everything is stored in memory, you can optionally also install a storage adapter.

npm install --save @keyv/redis
npm install --save @keyv/valkey
npm install --save @keyv/mongo
npm install --save @keyv/sqlite
npm install --save @keyv/postgres
npm install --save @keyv/mysql
npm install --save @keyv/etcd
npm install --save @keyv/memcache

First, create a new Keyv instance.

1import Keyv from 'keyv';

Type-safe Usage

You can create a Keyv instance with a generic type to enforce type safety for the values stored. Additionally, both the get and set methods support specifying custom types for specific use cases.

Example with Instance-level Generic Type:

1const keyv = new Keyv<number>(); // Instance handles only numbers
2await keyv.set('key1', 123);
3const value = await keyv.get('key1'); // value is inferred as number

Example with Method-level Generic Type:

You can also specify a type directly in the get or set methods, allowing flexibility for different types of values within the same instance.

1const keyv = new Keyv(); // Generic type not specified at instance level
2
3await keyv.set<string>('key2', 'some string'); // Method-level type for this value
4const strValue = await keyv.get<string>('key2'); // Explicitly typed as string
5
6await keyv.set<number>('key3', 456); // Storing a number in the same instance
7const numValue = await keyv.get<number>('key3'); // Explicitly typed as number

This makes Keyv highly adaptable to different data types while maintaining type safety.

Using Storage Adapters

Once you have created your Keyv instance you can use it as a simple key-value store with in-memory by default. To use a storage adapter, create an instance of the adapter and pass it to the Keyv constructor. Here are some examples:

1// redis
2import KeyvRedis from '@keyv/redis';
3
4const keyv = new Keyv(new KeyvRedis('redis://user:pass@localhost:6379'));

You can also pass in a storage adapter with other options such as ttl and namespace (example using sqlite):

1//sqlite
2import KeyvSqlite from '@keyv/sqlite';
3
4const keyvSqlite = new KeyvSqlite('sqlite://path/to/database.sqlite');
5const keyv = new Keyv({ store: keyvSqlite, ttl: 5000, namespace: 'cache' });

To handle an event you can do the following:

1// Handle DB connection errors
2keyv.on('error', err => console.log('Connection Error', err));

Now lets do an end-to-end example using Keyv and the Redis storage adapter:

1import Keyv from 'keyv';
2import KeyvRedis from '@keyv/redis';
3
4const keyvRedis = new KeyvRedis('redis://user:pass@localhost:6379');
5const keyv = new Keyv({ store: keyvRedis });
6
7await keyv.set('foo', 'expires in 1 second', 1000); // true
8await keyv.set('foo', 'never expires'); // true
9await keyv.get('foo'); // 'never expires'
10await keyv.delete('foo'); // true
11await keyv.clear(); // undefined

It's is just that simple! Keyv is designed to be simple and easy to use.

Namespaces

You can namespace your Keyv instance to avoid key collisions and allow you to clear only a certain namespace while using the same database.

1const users = new Keyv(new KeyvRedis('redis://user:pass@localhost:6379'), { namespace: 'users' });
2const cache = new Keyv(new KeyvRedis('redis://user:pass@localhost:6379'), { namespace: 'cache' });
3
4await users.set('foo', 'users'); // true
5await cache.set('foo', 'cache'); // true
6await users.get('foo'); // 'users'
7await cache.get('foo'); // 'cache'
8await users.clear(); // undefined
9await users.get('foo'); // undefined
10await cache.get('foo'); // 'cache'

Events

Keyv is a custom EventEmitter and will emit an 'error' event if there is an error. In addition it will emit a clear and disconnect event when the corresponding methods are called.

1const keyv = new Keyv();
2const handleConnectionError = err => console.log('Connection Error', err);
3const handleClear = () => console.log('Cache Cleared');
4const handleDisconnect = () => console.log('Disconnected');
5
6keyv.on('error', handleConnectionError);
7keyv.on('clear', handleClear);
8keyv.on('disconnect', handleDisconnect);

Hooks

Keyv supports hooks for get, set, and delete methods. Hooks are useful for logging, debugging, and other custom functionality. Here is a list of all the hooks:

PRE_GET
POST_GET
PRE_GET_MANY
POST_GET_MANY
PRE_SET
POST_SET
PRE_DELETE
POST_DELETE

You can access this by importing KeyvHooks from the main Keyv package.

1import Keyv, { KeyvHooks } from 'keyv';

1//PRE_SET hook
2const keyv = new Keyv();
3keyv.hooks.addHandler(KeyvHooks.PRE_SET, (key, value) => console.log(`Setting key ${key} to ${value}`));
4
5//POST_SET hook
6const keyv = new Keyv();
7keyv.hooks.addHandler(KeyvHooks.POST_SET, (key, value) => console.log(`Set key ${key} to ${value}`));

In these examples you can also manipulate the value before it is set. For example, you could add a prefix to all keys.

1const keyv = new Keyv();
2keyv.hooks.addHandler(KeyvHooks.PRE_SET, (key, value) => {
3  console.log(`Setting key ${key} to ${value}`);
4  key = `prefix-${key}`;
5});

Now this key will have prefix- added to it before it is set.

In PRE_DELETE and POST_DELETE hooks, the value could be a single item or an Array. This is based on the fact that delete can accept a single key or an Array of keys.

Custom Serializers

Keyv uses buffer for data serialization to ensure consistency across different backends.

You can optionally provide your own serialization functions to support extra data types or to serialize to something other than JSON.

1const keyv = new Keyv({ serialize: JSON.stringify, deserialize: JSON.parse });

Warning: Using custom serializers means you lose any guarantee of data consistency. You should do extensive testing with your serialisation functions and chosen storage engine.

If you do not want to use serialization you can set the serialize and deserialize functions to undefined. This will also turn off compression.

1const keyv = new Keyv();
2keyv.serialize = undefined;
3keyv.deserialize = undefined;

Official Storage Adapters

The official storage adapters are covered by over 150 integration tests to guarantee consistent behaviour. They are lightweight, efficient wrappers over the DB clients making use of indexes and native TTLs where available.

Database	Adapter	Native TTL
Redis	@keyv/redis	Yes
Valkey	@keyv/valkey	Yes
MongoDB	@keyv/mongo	Yes
SQLite	@keyv/sqlite	No
PostgreSQL	@keyv/postgres	No
MySQL	@keyv/mysql	No
Etcd	@keyv/etcd	Yes
Memcache	@keyv/memcache	Yes

Third-party Storage Adapters

You can also use third-party storage adapters or build your own. Keyv will wrap these storage adapters in TTL functionality and handle complex types internally.

1import Keyv from 'keyv';
2import myAdapter from 'my-adapter';
3
4const keyv = new Keyv({ store: myAdapter });

Any store that follows the Map api will work.

1new Keyv({ store: new Map() });

For example, quick-lru is a completely unrelated module that implements the Map API.

1import Keyv from 'keyv';
2import QuickLRU from 'quick-lru';
3
4const lru = new QuickLRU({ maxSize: 1000 });
5const keyv = new Keyv({ store: lru });

The following are third-party storage adapters compatible with Keyv:

• quick-lru - Simple "Least Recently Used" (LRU) cache
• keyv-file - File system storage adapter for Keyv
• keyv-dynamodb - DynamoDB storage adapter for Keyv
• keyv-lru - LRU storage adapter for Keyv
• keyv-null - Null storage adapter for Keyv
• keyv-firestore – Firebase Cloud Firestore adapter for Keyv
• keyv-mssql - Microsoft Sql Server adapter for Keyv
• keyv-azuretable - Azure Table Storage/API adapter for Keyv
• keyv-arango - ArangoDB storage adapter for Keyv
• keyv-momento - Momento storage adapter for Keyv

Add Cache Support to your Module

Keyv is designed to be easily embedded into other modules to add cache support. The recommended pattern is to expose a cache option in your modules options which is passed through to Keyv. Caching will work in memory by default and users have the option to also install a Keyv storage adapter and pass in a connection string, or any other storage that implements the Map API.

You should also set a namespace for your module so you can safely call .clear() without clearing unrelated app data.

Inside your module:

1class AwesomeModule {
2	constructor(opts) {
3		this.cache = new Keyv({
4			uri: typeof opts.cache === 'string' && opts.cache,
5			store: typeof opts.cache !== 'string' && opts.cache,
6			namespace: 'awesome-module'
7		});
8	}
9}

Now it can be consumed like this:

1import AwesomeModule from 'awesome-module';
2
3// Caches stuff in memory by default
4const awesomeModule = new AwesomeModule();
5
6// After npm install --save keyv-redis
7const awesomeModule = new AwesomeModule({ cache: 'redis://localhost' });
8
9// Some third-party module that implements the Map API
10const awesomeModule = new AwesomeModule({ cache: some3rdPartyStore });

Compression

Keyv supports gzip and brotli compression. To enable compression, pass the compress option to the constructor.

1import Keyv from 'keyv';
2import KeyvGzip from '@keyv/compress-gzip';
3
4const keyvGzip = new KeyvGzip();
5const keyv = new Keyv({ compression: KeyvGzip });

You can also pass a custom compression function to the compression option. Following the pattern of the official compression adapters.

Want to build your own CompressionAdapter?

Great! Keyv is designed to be easily extended. You can build your own compression adapter by following the pattern of the official compression adapters based on this interface:

1interface CompressionAdapter {
2	async compress(value: any, options?: any);
3	async decompress(value: any, options?: any);
4	async serialize(value: any);
5	async deserialize(value: any);
6}

In addition to the interface, you can test it with our compression test suite using @keyv/test-suite:

1import { keyvCompresstionTests } from '@keyv/test-suite';
2import KeyvGzip from '@keyv/compress-gzip';
3
4keyvCompresstionTests(test, new KeyvGzip());

API

new Keyv([storage-adapter], [options]) or new Keyv([options])

Returns a new Keyv instance.

The Keyv instance is also an EventEmitter that will emit an 'error' event if the storage adapter connection fails.

storage-adapter

Type: KeyvStorageAdapter
Default: undefined

The connection string URI.

Merged into the options object as options.uri.

.namespace

Type: String Default: 'keyv'

This is the namespace for the current instance. When you set it it will set it also on the storage adapter. This is the preferred way to set the namespace over .opts.namespace.

options

Type: Object

The options object is also passed through to the storage adapter. Check your storage adapter docs for any extra options.

options.namespace

Type: String
Default: 'keyv'

Namespace for the current instance.

options.ttl

Type: Number
Default: undefined

Default TTL. Can be overridden by specififying a TTL on .set().

options.compression

Type: @keyv/compress-<compression_package_name>
Default: undefined

Compression package to use. See Compression for more details.

options.serialize

Type: Function
Default: JSON.stringify

A custom serialization function.

options.deserialize

Type: Function
Default: JSON.parse

A custom deserialization function.

options.store

Type: Storage adapter instance
Default: new Map()

The storage adapter instance to be used by Keyv.

Keyv Instance

Keys must always be strings. Values can be of any type.

.set(key, value, [ttl])

Set a value.

By default keys are persistent. You can set an expiry TTL in milliseconds.

Returns a promise which resolves to true.

.get(key, [options])

Returns a promise which resolves to the retrieved value.

options.raw

Type: Boolean
Default: false

If set to true the raw DB object Keyv stores internally will be returned instead of just the value.

This contains the TTL timestamp.

.delete(key)

Deletes an entry.

Returns a promise which resolves to true if the key existed, false if not.

.clear()

Delete all entries in the current namespace.

Returns a promise which is resolved when the entries have been cleared.

.iterator()

Iterate over all entries of the current namespace.

Returns a iterable that can be iterated by for-of loops. For example:

1// please note that the "await" keyword should be used here
2for await (const [key, value] of this.keyv.iterator()) {
3  console.log(key, value);
4};

API - Properties

.namespace

Type: String

The namespace for the current instance. This will define the namespace for the current instance and the storage adapter. If you set the namespace to undefined it will no longer do key prefixing.

1const keyv = new Keyv({ namespace: 'my-namespace' });
2console.log(keyv.namespace); // 'my-namespace'

here is an example of setting the namespace to undefined:

1const keyv = new Keyv();
2console.log(keyv.namespace); // 'keyv' which is default
3keyv.namespace = undefined;
4console.log(keyv.namespace); // undefined

.ttl

Type: Number
Default: undefined

Default TTL. Can be overridden by specififying a TTL on .set(). If set to undefined it will never expire.

1const keyv = new Keyv({ ttl: 5000 });
2console.log(keyv.ttl); // 5000
3keyv.ttl = undefined;
4console.log(keyv.ttl); // undefined (never expires)

.store

Type: Storage adapter instance
Default: new Map()

The storage adapter instance to be used by Keyv. This will wire up the iterator, events, and more when a set happens. If it is not a valid Map or Storage Adapter it will throw an error.

1import KeyvSqlite from '@keyv/sqlite';
2const keyv = new Keyv();
3console.log(keyv.store instanceof Map); // true
4keyv.store = new KeyvSqlite('sqlite://path/to/database.sqlite');
5console.log(keyv.store instanceof KeyvSqlite); // true

.serialize

Type: Function
Default: JSON.stringify

A custom serialization function used for any value.

1const keyv = new Keyv();
2console.log(keyv.serialize); // JSON.stringify
3keyv.serialize = value => value.toString();
4console.log(keyv.serialize); // value => value.toString()

.deserialize

Type: Function
Default: JSON.parse

A custom deserialization function used for any value.

1const keyv = new Keyv();
2console.log(keyv.deserialize); // JSON.parse
3keyv.deserialize = value => parseInt(value);
4console.log(keyv.deserialize); // value => parseInt(value)

.compression

Type: CompressionAdapter
Default: undefined

this is the compression package to use. See Compression for more details. If it is undefined it will not compress (default).

1import KeyvGzip from '@keyv/compress-gzip';
2
3const keyv = new Keyv();
4console.log(keyv.compression); // undefined
5keyv.compression = new KeyvGzip();
6console.log(keyv.compression); // KeyvGzip

.useKeyPrefix

Type: Boolean
Default: true

If set to true Keyv will prefix all keys with the namespace. This is useful if you want to avoid collisions with other data in your storage.

1const keyv = new Keyv({ useKeyPrefix: false });
2console.log(keyv.useKeyPrefix); // false
3keyv.useKeyPrefix = true;
4console.log(keyv.useKeyPrefix); // true

How to Contribute

We welcome contributions to Keyv! 🎉 Here are some guides to get you started with contributing:

• Contributing - Learn about how to contribute to Keyv
• Code of Conduct - Learn about the Keyv Code of Conduct
• How to Contribute - How do develop in the Keyv mono repo!

License

MIT © Jared Wray