High Performance Layer 1 / Layer 2 Caching with Keyv Storage

cacheable is a high performance layer 1 / layer 2 caching engine that is focused on distributed caching with enterprise features such as CacheSync (coming soon). It is built on top of the robust storage engine Keyv and provides a simple API to cache and retrieve data.

• Simple to use with robust API
• Not bloated with additional modules
• Scalable and trusted storage engine by Keyv
• Memory Caching with LRU and Expiration CacheableMemory
• Resilient to failures with try/catch and offline
• Wrap / Memoization for Sync and Async Functions with Stampede Protection
• Hooks and Events to extend functionality
• Shorthand for ttl in milliseconds (1m = 60000) (1h = 3600000) (1d = 86400000)
• Non-blocking operations for layer 2 caching
• Distributed Caching Sync via Pub/Sub (coming soon)
• Comprehensive testing and code coverage
• ESM and CommonJS support with Typescript
• Maintained and supported regularly

Table of Contents

• Getting Started
• Basic Usage
• Hooks and Events
• Storage Tiering and Caching
• TTL Propagation and Storage Tiering
• Shorthand for Time to Live (ttl)
• Non-Blocking Operations
• CacheSync - Distributed Updates
• Cacheable Options
• Cacheable Statistics (Instance Only)
• Cacheable - API
• CacheableMemory - In-Memory Cache
• CacheableMemory Options
• CacheableMemory - API
• Keyv Storage Adapter - KeyvCacheableMemory
• Wrap / Memoization for Sync and Async Functions
• How to Contribute
• License and Copyright

Getting Started

cacheable is primarily used as an extension to you caching engine with a robust storage backend Keyv, Memonization (Wrap), Hooks, Events, and Statistics.

1npm install cacheable

Basic Usage

1import { Cacheable } from 'cacheable';
2
3const cacheable = new Cacheable();
4await cacheable.set('key', 'value', 1000);
5const value = await cacheable.get('key');

This is a basic example where you are only using the in-memory storage engine. To enable layer 1 and layer 2 caching you can use the secondary property in the options:

1import { Cacheable } from 'cacheable';
2import KeyvRedis from '@keyv/redis';
3
4const secondary = new KeyvRedis('redis://user:pass@localhost:6379');
5const cache = new Cacheable({secondary});

In this example, the primary store we will use lru-cache and the secondary store is Redis. You can also set multiple stores in the options:

1import { Cacheable } from 'cacheable';
2import { Keyv } from 'keyv';
3import KeyvRedis from '@keyv/redis';
4import { LRUCache } from 'lru-cache'
5
6const primary = new Keyv({store: new LRUCache()});
7const secondary = new KeyvRedis('redis://user:pass@localhost:6379');
8const cache = new Cacheable({primary, secondary});

This is a more advanced example and not needed for most use cases.

Hooks and Events

The following hooks are available for you to extend the functionality of cacheable via CacheableHooks enum:

• BEFORE_SET: This is called before the set() method is called.
• AFTER_SET: This is called after the set() method is called.
• BEFORE_SET_MANY: This is called before the setMany() method is called.
• AFTER_SET_MANY: This is called after the setMany() method is called.
• BEFORE_GET: This is called before the get() method is called.
• AFTER_GET: This is called after the get() method is called.
• BEFORE_GET_MANY: This is called before the getMany() method is called.
• AFTER_GET_MANY: This is called after the getMany() method is called.
• BEFORE_SECONDARY_SETS_PRIMARY: This is called when the secondary store sets the value in the primary store.

An example of how to use these hooks:

1import { Cacheable, CacheableHooks } from 'cacheable';
2
3const cacheable = new Cacheable();
4cacheable.onHook(CacheableHooks.BEFORE_SET, (data) => {
5  console.log(`before set: ${data.key} ${data.value}`);
6});

Here is an example of how to use BEFORE_SECONDARY_SETS_PRIMARY hook:

1import { Cacheable, CacheableHooks } from 'cacheable';
2import KeyvRedis from '@keyv/redis';
3const secondary = new KeyvRedis('redis://user:pass@localhost:6379');
4const cache = new Cacheable({secondary});
5cache.onHook(CacheableHooks.BEFORE_SECONDARY_SETS_PRIMARY, (data) => {
6  console.log(`before secondary sets primary: ${data.key} ${data.value} ${data.ttl}`);
7});

This is called when the secondary store sets the value in the primary store. This is useful if you want to do something before the value is set in the primary store such as manipulating the ttl or the value.

Storage Tiering and Caching

cacheable is built as a layer 1 and layer 2 caching engine by default. The purpose is to have your layer 1 be fast and your layer 2 be more persistent. The primary store is the layer 1 cache and the secondary store is the layer 2 cache. By adding the secondary store you are enabling layer 2 caching. By default the operations are blocking but fault tolerant:

• Setting Data: Sets the value in the primary store and then the secondary store.
• Getting Data: Gets the value from the primary if the value does not exist it will get it from the secondary store and set it in the primary store.
• Deleting Data: Deletes the value from the primary store and secondary store at the same time waiting for both to respond.
• Clearing Data: Clears the primary store and secondary store at the same time waiting for both to respond.

When Getting Data if the value does not exist in the primary store it will try to get it from the secondary store. If the secondary store returns the value it will set it in the primary store. Because we use TTL Propagation the value will be set in the primary store with the TTL of the secondary store unless the time to live (TTL) is greater than the primary store which will then use the TTL of the primary store. An example of this is:

1import { Cacheable } from 'cacheable';
2import KeyvRedis from '@keyv/redis';
3const secondary = new KeyvRedis('redis://user:pass@localhost:6379', { ttl: 1000 });
4const cache = new Cacheable({secondary, ttl: 100});
5
6await cache.set('key', 'value'); // sets the value in the primary store with a ttl of 100 ms and secondary store with a ttl of 1000 ms
7
8await sleep(500); // wait for .5 seconds
9
10const value = await cache.get('key'); // gets the value from the secondary store and now sets the value in the primary store with a ttl of 500 ms which is what is left from the secondary store

In this example the primary store has a ttl of 100 ms and the secondary store has a ttl of 1000 ms. Because the ttl is greater in the secondary store it will default to setting ttl value in the primary store.

1import { Cacheable } from 'cacheable';
2import {Keyv} from 'keyv';
3import KeyvRedis from '@keyv/redis';
4const primary = new Keyv({ ttl: 200 });
5const secondary = new KeyvRedis('redis://user:pass@localhost:6379', { ttl: 1000 });
6const cache = new Cacheable({primary, secondary});
7
8await cache.set('key', 'value'); // sets the value in the primary store with a ttl of 100 ms and secondary store with a ttl of 1000 ms
9
10await sleep(200); // wait for .2 seconds
11
12const value = await cache.get('key'); // gets the value from the secondary store and now sets the value in the primary store with a ttl of 200 ms which is what the primary store is set with

TTL Propagation and Storage Tiering

Cacheable TTL propagation is a feature that allows you to set a time to live (TTL) for the cache. By default the TTL is set in the following order:

ttl = set at the function ?? storage adapter ttl ?? cacheable ttl

This means that if you set a TTL at the function level it will override the storage adapter TTL and the cacheable TTL. If you do not set a TTL at the function level it will use the storage adapter TTL and then the cacheable TTL. If you do not set a TTL at all it will use the default TTL of undefined which is disabled.

Shorthand for Time to Live (ttl)

By default Cacheable and CacheableMemory the ttl is in milliseconds but you can use shorthand for the time to live. Here are the following shorthand values:

• ms: Milliseconds such as (1ms = 1)
• s: Seconds such as (1s = 1000)
• m: Minutes such as (1m = 60000)
• h or hr: Hours such as (1h = 3600000)
• d: Days such as (1d = 86400000)

Here is an example of how to use the shorthand for the ttl:

1import { Cacheable } from 'cacheable';
2const cache = new Cacheable({ ttl: '15m' }); //sets the default ttl to 15 minutes (900000 ms)
3cache.set('key', 'value', '1h'); //sets the ttl to 1 hour (3600000 ms) and overrides the default

if you want to disable the ttl you can set it to 0 or undefined:

1import { Cacheable } from 'cacheable';
2const cache = new Cacheable({ ttl: 0 }); //sets the default ttl to 0 which is disabled
3cache.set('key', 'value', 0); //sets the ttl to 0 which is disabled

If you set the ttl to anything below 0 or undefined it will disable the ttl for the cache and the value that returns will be undefined. With no ttl set the value will be stored indefinitely.

1import { Cacheable } from 'cacheable';
2const cache = new Cacheable({ ttl: 0 }); //sets the default ttl to 0 which is disabled
3console.log(cache.ttl); // undefined
4cache.ttl = '1h'; // sets the default ttl to 1 hour (3600000 ms)
5console.log(cache.ttl); // '1h'
6cache.ttl = -1; // sets the default ttl to 0 which is disabled
7console.log(cache.ttl); // undefined

Retrieving raw cache entries

The get and getMany methods support a raw option, which returns the full stored metadata (StoredDataRaw<T>) instead of just the value:

1import { Cacheable } from 'cacheable';
2
3const cache = new Cacheable();
4
5// store a value
6await cache.set('user:1', { name: 'Alice' });
7
8// default: only the value
9const user = await cache.get<{ name: string }>('user:1');
10console.log(user); // { name: 'Alice' }
11
12// with raw: full record including expiration
13const raw = await cache.get<{ name: string }>('user:1', { raw: true });
14console.log(raw.value);   // { name: 'Alice' }
15console.log(raw.expires); // e.g. 1677628495000 or null

1// getMany with raw option
2await cache.set('a', 1);
3await cache.set('b', 2);
4
5const raws = await cache.getMany<number>(['a', 'b'], { raw: true });
6raws.forEach((entry, idx) => {
7  console.log(`key=${['a','b'][idx]}, value=${entry?.value}, expires=${entry?.expires}`);
8});

Non-Blocking Operations

If you want your layer 2 (secondary) store to be non-blocking you can set the nonBlocking property to true in the options. This will make the secondary store non-blocking and will not wait for the secondary store to respond on setting data, deleting data, or clearing data. This is useful if you want to have a faster response time and not wait for the secondary store to respond.

1import { Cacheable } from 'cacheable';
2import {KeyvRedis} from '@keyv/redis';
3
4const secondary = new KeyvRedis('redis://user:pass@localhost:6379');
5const cache = new Cacheable({secondary, nonBlocking: true});

CacheSync - Distributed Updates

cacheable has a feature called CacheSync that is coming soon. This feature will allow you to have distributed caching with Pub/Sub. This will allow you to have multiple instances of cacheable running and when a value is set, deleted, or cleared it will update all instances of cacheable with the same value. Current plan is to support the following:

• AWS SQS
• RabbitMQ
• Nats
• Azure Service Bus
• Redis Pub/Sub

This feature should be live by end of year.

Cacheable Options

The following options are available for you to configure cacheable:

• primary: The primary store for the cache (layer 1) defaults to in-memory by Keyv.
• secondary: The secondary store for the cache (layer 2) usually a persistent cache by Keyv.
• nonBlocking: If the secondary store is non-blocking. Default is false.
• stats: To enable statistics for this instance. Default is false.
• ttl: The default time to live for the cache in milliseconds. Default is undefined which is disabled.
• namespace: The namespace for the cache. Default is undefined.

Cacheable Statistics (Instance Only)

If you want to enable statistics for your instance you can set the .stats.enabled property to true in the options. This will enable statistics for your instance and you can get the statistics by calling the stats property. Here are the following property statistics:

• hits: The number of hits in the cache.
• misses: The number of misses in the cache.
• sets: The number of sets in the cache.
• deletes: The number of deletes in the cache.
• clears: The number of clears in the cache.
• errors: The number of errors in the cache.
• count: The number of keys in the cache.
• vsize: The estimated byte size of the values in the cache.
• ksize: The estimated byte size of the keys in the cache.

You can clear / reset the stats by calling the .stats.reset() method.

This does not enable statistics for your layer 2 cache as that is a distributed cache.

Cacheable - API

• set(key, value, ttl?): Sets a value in the cache.
• setMany([{key, value, ttl?}]): Sets multiple values in the cache.
• get(key): Gets a value from the cache.
• get(key, { raw: true }): Gets a raw value from the cache.
• getMany([keys]): Gets multiple values from the cache.
• getMany([keys], { raw: true }): Gets multiple raw values from the cache.
• has(key): Checks if a value exists in the cache.
• hasMany([keys]): Checks if multiple values exist in the cache.
• take(key): Takes a value from the cache and deletes it.
• takeMany([keys]): Takes multiple values from the cache and deletes them.
• delete(key): Deletes a value from the cache.
• deleteMany([keys]): Deletes multiple values from the cache.
• clear(): Clears the cache stores. Be careful with this as it will clear both layer 1 and layer 2.
• wrap(function, WrapOptions): Wraps an async function in a cache.
• disconnect(): Disconnects from the cache stores.
• onHook(hook, callback): Sets a hook.
• removeHook(hook): Removes a hook.
• on(event, callback): Listens for an event.
• removeListener(event, callback): Removes a listener.
• hash(object: any, algorithm = 'sha256'): string: Hashes an object with the algorithm. Default is sha256.
• primary: The primary store for the cache (layer 1) defaults to in-memory by Keyv.
• secondary: The secondary store for the cache (layer 2) usually a persistent cache by Keyv.
• namespace: The namespace for the cache. Default is undefined. This will set the namespace for the primary and secondary stores.
• nonBlocking: If the secondary store is non-blocking. Default is false.
• stats: The statistics for this instance which includes hits, misses, sets, deletes, clears, errors, count, vsize, ksize.

CacheableMemory - In-Memory Cache

cacheable comes with a built-in in-memory cache called CacheableMemory. This is a simple in-memory cache that is used as the primary store for cacheable. You can use this as a standalone cache or as a primary store for cacheable. Here is an example of how to use CacheableMemory:

1import { CacheableMemory } from 'cacheable';
2const options = {
3  ttl: '1h', // 1 hour
4  useClones: true, // use clones for the values (default is true)
5  lruSize: 1000, // the size of the LRU cache (default is 0 which is unlimited)
6}
7const cache = new CacheableMemory(options);
8cache.set('key', 'value');
9const value = cache.get('key'); // value

You can use CacheableMemory as a standalone cache or as a primary store for cacheable. You can also set the useClones property to false if you want to use the same reference for the values. This is useful if you are using large objects and want to save memory. The lruSize property is the size of the LRU cache and is set to 0 by default which is unlimited. When setting the lruSize property it will limit the number of keys in the cache.

This simple in-memory cache uses multiple Map objects and a with expiration and lru policies if set to manage the in memory cache at scale.

By default we use lazy expiration deletion which means on get and getMany type functions we look if it is expired and then delete it. If you want to have a more aggressive expiration policy you can set the checkInterval property to a value greater than 0 which will check for expired keys at the interval you set.

CacheableMemory Options

• ttl: The time to live for the cache in milliseconds. Default is undefined which is means indefinitely.
• useClones: If the cache should use clones for the values. Default is true.
• lruSize: The size of the LRU cache. Default is 0 which is unlimited.
• checkInterval: The interval to check for expired keys in milliseconds. Default is 0 which is disabled.

CacheableMemory - API

• set(key, value, ttl?): Sets a value in the cache.
• setMany([{key, value, ttl?}]): Sets multiple values in the cache from CacheableItem.
• get(key): Gets a value from the cache.
• getMany([keys]): Gets multiple values from the cache.
• getRaw(key): Gets a value from the cache as CacheableStoreItem.
• getManyRaw([keys]): Gets multiple values from the cache as CacheableStoreItem.
• has(key): Checks if a value exists in the cache.
• hasMany([keys]): Checks if multiple values exist in the cache.
• delete(key): Deletes a value from the cache.
• deleteMany([keys]): Deletes multiple values from the cache.
• take(key): Takes a value from the cache and deletes it.
• takeMany([keys]): Takes multiple values from the cache and deletes them.
• wrap(function, WrapSyncOptions): Wraps a sync function in a cache.
• clear(): Clears the cache.
• size(): The number of keys in the cache.
• keys(): The keys in the cache.
• items(): The items in the cache as CacheableStoreItem example { key, value, expires? }.
• checkExpired(): Checks for expired keys in the cache. This is used by the checkInterval property.
• startIntervalCheck(): Starts the interval check for expired keys if checkInterval is above 0 ms.
• stopIntervalCheck(): Stops the interval check for expired keys.
• hash(object: any, algorithm = 'sha256'): string: Hashes an object with the algorithm. Default is sha256.

Keyv Storage Adapter - KeyvCacheableMemory

cacheable comes with a built-in storage adapter for Keyv called KeyvCacheableMemory. This takes CacheableMemory and creates a storage adapter for Keyv. This is useful if you want to use CacheableMemory as a storage adapter for Keyv. Here is an example of how to use KeyvCacheableMemory:

1import { Keyv } from 'keyv';
2import { KeyvCacheableMemory } from 'cacheable';
3
4const keyv = new Keyv({ store: new KeyvCacheableMemory() });
5await keyv.set('foo', 'bar');
6const value = await keyv.get('foo');
7console.log(value); // bar 

Wrap / Memoization for Sync and Async Functions

Cacheable and CacheableMemory has a feature called wrap that allows you to wrap a function in a cache. This is useful for memoization and caching the results of a function. You can wrap a sync or async function in a cache. Here is an example of how to use the wrap function:

1import { Cacheable } from 'cacheable';
2const asyncFunction = async (value: number) => {
3  return Math.random() * value;
4};
5
6const cache = new Cacheable();
7const options = {
8  ttl: '1h', // 1 hour
9  keyPrefix: 'p1', // key prefix. This is used if you have multiple functions and need to set a unique prefix.
10}
11const wrappedFunction = cache.wrap(asyncFunction, options);
12console.log(await wrappedFunction(2)); // 4
13console.log(await wrappedFunction(2)); // 4 from cache

With Cacheable we have also included stampede protection so that a Promise based call will only be called once if multiple requests of the same are executed at the same time. Here is an example of how to test for stampede protection:

1import { Cacheable } from 'cacheable';
2const asyncFunction = async (value: number) => {
3  return value;
4};
5
6const cache = new Cacheable();
7const options = {
8  ttl: '1h', // 1 hour
9  keyPrefix: 'p1', // key prefix. This is used if you have multiple functions and need to set a unique prefix.
10}
11
12const wrappedFunction = cache.wrap(asyncFunction, options);
13const promises = [];
14for (let i = 0; i < 10; i++) {
15  promises.push(wrappedFunction(i));
16}
17
18const results = await Promise.all(promises); // all results should be the same
19
20console.log(results); // [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

In this example we are wrapping an async function in a cache with a ttl of 1 hour. This will cache the result of the function for 1 hour and then expire the value. You can also wrap a sync function in a cache:

1import { CacheableMemory } from 'cacheable';
2const syncFunction = (value: number) => {
3  return value * 2;
4};
5
6const cache = new CacheableMemory();
7const wrappedFunction = cache.wrap(syncFunction, { ttl: '1h', key: 'syncFunction' });
8console.log(wrappedFunction(2)); // 4
9console.log(wrappedFunction(2)); // 4 from cache

In this example we are wrapping a sync function in a cache with a ttl of 1 hour. This will cache the result of the function for 1 hour and then expire the value. You can also set the key property in the wrap() options to set a custom key for the cache.

When an error occurs in the function it will not cache the value and will return the error. This is useful if you want to cache the results of a function but not cache the error. If you want it to cache the error you can set the cacheError property to true in the wrap() options. This is disabled by default.

1import { CacheableMemory } from 'cacheable';
2const syncFunction = (value: number) => {
3  throw new Error('error');
4};
5
6const cache = new CacheableMemory();
7const wrappedFunction = cache.wrap(syncFunction, { ttl: '1h', key: 'syncFunction', cacheError: true });
8console.log(wrappedFunction()); // error
9console.log(wrappedFunction()); // error from cache

How to Contribute

You can contribute by forking the repo and submitting a pull request. Please make sure to add tests and update the documentation. To learn more about how to contribute go to our main README https://github.com/jaredwray/cacheable. This will talk about how to Open a Pull Request, Ask a Question, or Post an Issue.

License and Copyright

MIT © Jared Wray