Concurrent Callback Queue

A simple, lightweight, zero dependencies pure JavaScript queue implementation that allows for asynchronous operations such as API calls heavy computations or large file processing to be scheduled for execution in a parallel fashion.

The project consists of a single ConcurrentCallbackQueue class that provides a simple API to manage the execution of multiple callbacks concurrently with a configurable limit on the number of concurrent executions. It also provides a retry mechanism for each callback in case of errors, and callback hooks for queue state changes and callback events.

Table of Contents

• Features
• Installation
  • NPM
  • CDN
• Usage
  • Example
  • Basic Usage
  • Configuration Options
  • Multiple Callbacks
  • Retry Mechanism
  • Queue State
  • Event Hooks
  • Queue Control
• Use Cases
• License
• Author
• Contributing
  • Commit Message Format

Features

• Concurrent Execution: Execute multiple callbacks concurrently with a configurable limit on the number of concurrent executions.
• Automatic Start: Option to automatically start the queue execution upon adding a callback.
• Retry Mechanism: Define the number of retry attempts for each callback in case of errors.
• State Management: Callbacks for different queue states (idle, busy, stopped) and events (success, error).
• Custom Callbacks: Customize behavior on callback success, error, and different queue states.

Installation

NPM

If you are using npm as your package manager, you can install the package by running:

1npm install @diomeh/concurrent_callback_queue

Then you can import the ConcurrentCallbackQueue class in your project:

1import { ConcurrentCallbackQueue } from "@diomeh/concurrent_callback_queue";
2const queue = new ConcurrentCallbackQueue();

CDN

You can also include the script directly in your HTML file using the jsDelivr CDN:

As a module:

1<script
2  type="module"
3  src="https://cdn.jsdelivr.net/npm/@diomeh/concurrent_callback_queue/dist/ConcurrentCallbackQueue.esm.js"
4></script>

As a script:

1<script src="https://cdn.jsdelivr.net/npm/@diomeh/concurrent_callback_queue/dist/ConcurrentCallbackQueue.iife.min.js"></script>

This will expose the ConcurrentCallbackQueue class globally, and you can use it in your JavaScript code:

1const queue = new ConcurrentCallbackQueue();

Usage

Example

Detailed examples can be found in the tutorial page.

Here's an example of how to use the ConcurrentCallbackQueue:

1const queue = new ConcurrentCallbackQueue({
2  autoStart: false,
3  maxConcurrent: 2,
4  onCallbackError: (error) => console.error(error),
5});
6
7for (let i = 0; i < 10; i++) {
8  queue.enqueue(() => {
9    const uri = "https://httpstat.us/200,400?sleep=2000";
10    return fetch(uri).then((response) => response.text());
11  }, 3);
12}
13
14queue.start();

Basic Usage

To create and use a basic concurrent callback queue:

1const queue = new ConcurrentCallbackQueue();
2queue.enqueue(() => fetch("https://httpstat.us/200,400?sleep=2000"));

Configuration Options

Execution can be configured with the following options:

• autoStart (boolean): Whether to start the queue automatically when a callback is added.
• maxConcurrent (number): Maximum number of callbacks to execute concurrently.
• onCallbackError (function): Callback executed when an error occurs during callback execution.
• onCallbackSuccess (function): Callback executed after a callback is executed successfully.
• onQueueIdle (function): Callback executed when the queue becomes idle.
• onQueueBusy (function): Callback executed when the queue becomes busy.
• onQueueStop (function): Callback executed when the queue stops.

1const queue = new ConcurrentCallbackQueue({
2  autoStart: false,
3  onCallbackError: (error) => console.error(error),
4});
5queue.enqueue(() => {
6  throw new Error("Error");
7});
8queue.start();

Multiple Callbacks

To enqueue multiple callbacks at once:

1const queue = new ConcurrentCallbackQueue();
2queue.enqueueAll([
3  () => fetch("https://httpstat.us/200,400?sleep=2000"),
4  () => fetch("https://httpstat.us/200,400?sleep=2000"),
5]);

Retry Mechanism

You can specify the number of retry attempts for each callback:

1const queue = new ConcurrentCallbackQueue();
2const retries = 3;
3queue.enqueue(() => fetch("https://httpstat.us/200,400?sleep=2000"), retries);

Queue State

You can check the state of the queue at any time:

1const queue = new ConcurrentCallbackQueue();
2console.log(queue.getState()); // QueueState.IDLE

Possible states are:

• QueueState.IDLE: There are no pending callbacks and the queue will start processing as soon as a callback is added.
• QueueState.BUSY: Currently processing callbacks up to the maximum concurrent limit.
• QueueState.STOPPED: Processing has been stopped and no further callbacks will be executed unless the queue is started again.

Event Hooks

You can define custom event hooks to handle various queue state changes:

1const queue = new ConcurrentCallbackQueue({
2  onQueueIdle: () => console.log("Queue is idle"),
3  onQueueBusy: () => console.log("Queue is busy"),
4  onQueueStop: () => console.log("Queue has stopped"),
5});

There are also hooks for individual callback events:

1const queue = new ConcurrentCallbackQueue({
2  onCallbackSuccess: (result) => console.log("Callback succeeded:", result),
3  onCallbackError: (error) => console.error("Callback failed:", error),
4});
5queue.enqueueAll([
6  () => fetch("https://httpstat.us/200,400?sleep=2000"),
7  () => {
8    throw new Error("Error");
9  },
10]);

Queue Control

You can start, stop, and clear the queue as needed:

1const queue = new ConcurrentCallbackQueue({
2  autoStart: false,
3  maxConcurrent: 1,
4});
5
6queue.enqueueAll([
7  () => fetch("https://httpstat.us/200,400?sleep=2000"),
8  () => fetch("https://httpstat.us/200,400?sleep=2000"),
9]);
10
11queue.start(); // Will execute the first callback
12queue.stop(); // Will stop the queue after the first callback
13queue.clear(); // Will clear all callbacks that have not been executed yet and return them in an array

Use Cases

The ConcurrentCallbackQueue can be used in various scenarios where you need to manage multiple asynchronous operations concurrently:

• API Calls: Execute multiple API requests in parallel.
• Heavy Computations: Run CPU-intensive tasks concurrently.
• File Processing: Process multiple files simultaneously.
• Task Scheduling: Schedule and manage multiple tasks.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Author

David Urbina

Contributing

Contributions are welcome! Feel free to submit a pull request or open an issue if you have any suggestions or feedback.

This project follows the Contributor Covenant Code of Conduct and the Conventional Commits Specification.

Commit Message Format

From the Conventional Commits Specification Summary:

The commit message should be structured as follows:

1{type}[optional scope]: {description}
2
3[optional body]
4
5[optional footer(s)]

Where type is one of the following:

Type	Description	Example Commit Message
fix	Patches a bug in your codebase (correlates with PATCH in Semantic Versioning)	fix: correct typo in README
feat	Introduces a new feature to the codebase (correlates with MINOR in Semantic Versioning)	feat: add new user login functionality
BREAKING CHANGE	Introduces a breaking API change (correlates with MAJOR in Semantic Versioning)	feat!: drop support for Node 8
build	Changes that affect the build system or external dependencies	build: update dependency version
chore	Other changes that don't modify src or test files	chore: update package.json scripts
ci	Changes to CI configuration files and scripts	ci: add CircleCI config
docs	Documentation only changes	docs: update API documentation
style	Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)	style: fix linting errors
refactor	Code change that neither fixes a bug nor adds a feature	refactor: rename variable for clarity
perf	Code change that improves performance	perf: reduce size of image files
test	Adding missing tests or correcting existing tests	test: add unit tests for new feature
Custom Types	Any other type defined by the project for its specific needs	security: address vulnerability in dependencies

For more information, refer to the Conventional Commits Specification.