HTTP Ledger - Express Logging Middleware

A comprehensive Express.js middleware for detailed API request and response logging with beautiful console output, error handling, performance metrics, and advanced features like field masking, custom formatters, and log sampling.

✨ Features

• Comprehensive Logging: Captures method, URL, status code, timing, request/response sizes, headers, query parameters, and bodies
• Advanced Security: Field masking for sensitive data (passwords, tokens, etc.)
• Custom Logging: Custom formatters, log levels, and selective logging
• Performance Optimized: Log sampling and conditional logging
• Request Tracking: Auto-generated request IDs for request tracing
• Error Handling: Robust error handling with graceful fallbacks
• IP Information: Optional IP geolocation support
• Performance Metrics: Request/response size calculation and timing
• Flexible Configuration: Highly customizable logging options
• TypeScript Support: Full TypeScript support with type definitions
• Multiple Module Systems: Supports both CommonJS and ES Modules
• High Performance: Optimized for production use
• Custom Log Handling: Expose a callback to receive log data for custom processing

📦 Installation

1npm install http-ledger

🚀 Quick Start

1const express = require('express');
2const logger = require('http-ledger');
3
4const app = express();
5
6// Basic usage with default settings
7app.use(logger());
8
9// Advanced usage with all features
10app.use(
11  logger({
12    // Basic logging options
13    logBody: true,
14    logResponse: true,
15    logQueryParams: true,
16    excludedHeaders: ['authorization', 'cookie'],
17
18    // Security features
19    maskFields: ['password', 'token', 'secret'],
20
21    // Custom logging
22    customLogLevel: (logData) => (logData.statusCode >= 500 ? 'error' : 'info'),
23    customFormatter: (logData) => ({
24      ...logData,
25      customField: 'custom value',
26    }),
27
28    // Request tracking
29    autoGenerateRequestId: true,
30
31    // Selective logging
32    shouldLog: (req, res) => req.method !== 'OPTIONS',
33    logSampling: 0.1, // Log 10% of requests
34
35    // IP geolocation
36    getIpInfo: async (ip) => {
37      const response = await fetch(`https://ipapi.co/${ip}/json/`);
38      return response.json();
39    },
40
41    // Custom log handling
42    onLog: async (logData) => {
43      await fetch('https://my-log-endpoint', {
44        method: 'POST',
45        headers: { 'Content-Type': 'application/json' },
46        body: JSON.stringify(logData),
47      });
48    },
49  }),
50);
51
52app.listen(3000);

⚙️ Configuration Options

Basic Logging Options

logBody (boolean, default: true)

Whether to include the request body in the log output.

1app.use(logger({ logBody: false })); // Disable request body logging

logResponse (boolean, default: true)

Whether to include the response body in the log output.

1app.use(logger({ logResponse: false })); // Disable response body logging

logQueryParams (boolean, default: true)

Whether to include query parameters in the log output.

1app.use(logger({ logQueryParams: false })); // Disable query parameter logging

excludedHeaders (string[], default: [])

A list of header names (case-insensitive) to exclude from logs. Useful for removing sensitive headers.

1app.use(
2  logger({
3    excludedHeaders: ['authorization', 'cookie', 'x-api-key'],
4  }),
5);

Security Features

maskFields (string[], default: [])

A list of field names to mask in request/response bodies, headers, and query parameters. Sensitive values will be replaced with ***.

1app.use(
2  logger({
3    maskFields: ['password', 'token', 'secret', 'apiKey', 'creditCard'],
4  }),
5);
6
7// Input: { "user": "john", "password": "secret123" }
8// Output: { "user": "john", "password": "***" }

Nested field masking is supported:

1app.use(
2  logger({
3    maskFields: ['user.password', 'data.secret', 'nested.deep.secret'],
4  }),
5);

Custom Logging

customLogLevel (function, optional)

A custom function to determine the log level based on log data. Must return 'info', 'warn', or 'error'.

1app.use(
2  logger({
3    customLogLevel: (logData) => {
4      if (logData.statusCode >= 500) return 'error';
5      if (logData.statusCode >= 400) return 'warn';
6      if (logData.timeTaken > 1000) return 'warn'; // Slow requests
7      return 'info';
8    },
9  }),
10);

customFormatter (function, optional)

A custom function to format log data before output. Can transform or add additional fields.

1app.use(
2  logger({
3    customFormatter: (logData) => ({
4      ...logData,
5      environment: process.env.NODE_ENV,
6      service: 'user-api',
7      customTimestamp: new Date().toISOString(),
8      // Add custom metrics
9      isSlowRequest: logData.timeTaken > 1000,
10      requestCategory: logData.method === 'GET' ? 'read' : 'write',
11    }),
12  }),
13);

Request Tracking

autoGenerateRequestId (boolean, default: false)

Whether to auto-generate a request ID if one is not present in the X-Request-ID header. The generated ID will be added to response headers.

1app.use(logger({ autoGenerateRequestId: true }));
2// Adds X-Request-ID header to responses if not present

Selective Logging

shouldLog (function, optional)

A function that determines whether to log a specific request. Return false to skip logging.

1app.use(
2  logger({
3    shouldLog: (req, res) => {
4      // Skip health check endpoints
5      if (req.path === '/health') return false;
6
7      // Skip successful GET requests to static files
8      if (
9        req.method === 'GET' &&
10        req.path.startsWith('/static/') &&
11        res.statusCode === 200
12      ) {
13        return false;
14      }
15
16      return true;
17    },
18  }),
19);

logSampling (number, optional)

A sampling rate between 0 and 1 to log only a percentage of requests. Useful for high-traffic applications.

1app.use(logger({ logSampling: 0.1 })); // Log 10% of requests
2app.use(logger({ logSampling: 0.5 })); // Log 50% of requests
3app.use(logger({ logSampling: 1.0 })); // Log all requests (default)

External Integrations

getIpInfo (function, optional)

An async function that takes an IP address and returns IP-related information (e.g., geolocation).

1app.use(
2  logger({
3    getIpInfo: async (ip) => {
4      try {
5        const response = await fetch(`https://ipapi.co/${ip}/json/`);
6        const data = await response.json();
7        return {
8          country: data.country_name,
9          city: data.city,
10          region: data.region,
11          timezone: data.timezone,
12        };
13      } catch (error) {
14        return {}; // Return empty object on error
15      }
16    },
17  }),
18);

onLog (function, optional)

A callback that receives the full log data object after each request. Can be used to send logs to external services, databases, or custom endpoints. The callback is async-safe and will not block logging if it throws.

1app.use(
2  logger({
3    onLog: async (logData) => {
4      // Send to external logging service
5      await fetch('https://logs.example.com/api/logs', {
6        method: 'POST',
7        body: JSON.stringify(logData),
8        headers: { Authorization: `Bearer ${process.env.LOG_API_KEY}` },
9      });
10
11      // Or save to database
12      await db.logs.insert(logData);
13
14      // Or send to monitoring service
15      await monitoringService.track(logData);
16    },
17  }),
18);

Error handling in onLog:

1app.use(
2  logger({
3    onLog: () => {
4      throw new Error('External service down');
5    },
6  }),
7);
8// Console: onLog callback threw: Error: External service down
9// Logging continues normally despite the error

📊 Log Output Format

The middleware outputs structured JSON logs with comprehensive information:

1{
2  "method": "POST",
3  "url": "/api/users?page=1&limit=10",
4  "statusCode": 201,
5  "timeTaken": 45.23,
6  "requestSize": 256,
7  "responseSize": 1024,
8  "timestamp": {
9    "request": "2024-01-01T12:00:00.000Z",
10    "response": "2024-01-01T12:00:00.045Z"
11  },
12  "headers": {
13    "user-agent": "Mozilla/5.0...",
14    "accept": "application/json",
15    "content-type": "application/json"
16  },
17  "queryParams": {
18    "page": "1",
19    "limit": "10"
20  },
21  "body": {
22    "name": "John Doe",
23    "email": "john@example.com",
24    "password": "***"
25  },
26  "responseBody": {
27    "id": 123,
28    "name": "John Doe",
29    "email": "john@example.com",
30    "createdAt": "2024-01-01T12:00:00.000Z"
31  },
32  "ipInfo": {
33    "country": "United States",
34    "city": "New York",
35    "region": "NY",
36    "timezone": "America/New_York"
37  },
38  "userAgent": "Mozilla/5.0...",
39  "referer": "https://example.com/signup",
40  "requestContentType": "application/json",
41  "responseContentType": "application/json",
42  "httpVersion": "1.1",
43  "requestId": "req-abc123def456",
44  "hostname": "server-01",
45  "logLevel": "info"
46}

🔧 Advanced Usage Examples

Production Configuration

1app.use(
2  logger({
3    // Disable verbose logging in production
4    logBody: process.env.NODE_ENV === 'development',
5    logResponse: process.env.NODE_ENV === 'development',
6
7    // Mask sensitive fields
8    maskFields: ['password', 'token', 'secret', 'apiKey'],
9
10    // Sample logs in high-traffic environments
11    logSampling: process.env.NODE_ENV === 'production' ? 0.1 : 1.0,
12
13    // Custom log levels
14    customLogLevel: (logData) => {
15      if (logData.statusCode >= 500) return 'error';
16      if (logData.statusCode >= 400) return 'warn';
17      return 'info';
18    },
19
20    // Send to external logging service
21    onLog: async (logData) => {
22      await fetch(process.env.LOG_ENDPOINT, {
23        method: 'POST',
24        body: JSON.stringify(logData),
25        headers: { Authorization: `Bearer ${process.env.LOG_API_KEY}` },
26      });
27    },
28  }),
29);

Development Configuration

1app.use(
2  logger({
3    // Enable all logging for debugging
4    logBody: true,
5    logResponse: true,
6    logQueryParams: true,
7
8    // Exclude noisy headers
9    excludedHeaders: ['user-agent', 'accept-encoding'],
10
11    // Custom formatting for readability
12    customFormatter: (logData) => ({
13      ...logData,
14      // Add color coding for different status codes
15      color: logData.statusCode >= 400 ? 'red' : 'green',
16      // Add request timing category
17      timing: logData.timeTaken > 1000 ? 'slow' : 'fast',
18    }),
19  }),
20);

API Gateway Configuration

1app.use(
2  logger({
3    // Only log API requests
4    shouldLog: (req, res) => req.path.startsWith('/api/'),
5
6    // Auto-generate request IDs for tracing
7    autoGenerateRequestId: true,
8
9    // Mask API keys and tokens
10    maskFields: ['apiKey', 'token', 'authorization'],
11
12    // Custom log level based on business logic
13    customLogLevel: (logData) => {
14      if (logData.url.includes('/admin/')) return 'warn';
15      if (logData.statusCode >= 500) return 'error';
16      return 'info';
17    },
18  }),
19);

🛡️ Error Handling

The middleware includes comprehensive error handling:

• Graceful Degradation: If any component fails, the middleware continues to function
• Error Logging: Errors are captured and included in the log output
• Fallback Mechanisms: Multiple fallback strategies for timing and size calculations
• Safe JSON Serialization: Handles circular references and complex objects
• Async-Safe Callbacks: onLog and getIpInfo callbacks won't break logging if they throw
• Function Validation: Invalid functions are safely ignored

⚡ Performance Considerations

• Minimal Overhead: Optimized for high-performance applications
• Async Operations: IP info fetching and onLog callback are non-blocking
• Memory Efficient: Streams large responses without buffering
• Configurable Logging: Disable expensive operations when not needed
• Log Sampling: Reduce logging overhead in high-traffic environments
• Selective Logging: Skip logging for specific requests or conditions

📝 TypeScript Support

Full TypeScript support with comprehensive type definitions:

1import { Request, Response, NextFunction } from 'express';
2import logger, { ApiLoggerOptions, LogData, LogLevel } from 'http-ledger';
3
4const options: ApiLoggerOptions = {
5  logBody: true,
6  logResponse: true,
7  excludedHeaders: ['authorization'],
8  maskFields: ['password', 'token'],
9  customLogLevel: (logData: LogData): LogLevel => {
10    return logData.statusCode >= 500 ? 'error' : 'info';
11  },
12  getIpInfo: async (ip: string) => {
13    return { country: 'US', city: 'New York' };
14  },
15  onLog: (logData: LogData) => {
16    // Custom log handling
17  },
18};
19
20app.use(logger(options));

📦 Module System Support

The package supports multiple module systems:

CommonJS

1const logger = require('http-ledger');

ES Modules

1import logger from 'http-ledger';

TypeScript

1import logger, { ApiLoggerOptions, LogData } from 'http-ledger';

🛠️ Development

Prerequisites

• Node.js >= 18.0.0
• npm or yarn

Setup

1git clone <repository>
2cd http-ledger
3npm install

Available Scripts

• npm run build - Build both CommonJS and ES modules
• npm run bundlesize - Check bundle size of compiled output
• npm test - Run tests with coverage
• npm run test:watch - Run tests in watch mode
• npm run lint - Run ESLint
• npm run format - Format code with Prettier

Testing

1npm test
2npm run test:watch

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass
6. Submit a pull request

📄 License

MIT License - see LICENSE file for details.

📋 Changelog

v0.0.3

• New: Advanced security features with field masking
• New: Custom log formatters and log levels
• New: Request ID auto-generation
• New: Selective logging with shouldLog function
• New: Log sampling for high-traffic applications
• Enhanced: Improved error handling and type safety
• Enhanced: Better performance optimizations
• Enhanced: Comprehensive test coverage
• Enhanced: Updated dependencies

v0.0.2

• Initial release with basic logging functionality

🆘 Support

For issues and questions, please open an issue on GitHub.