The @pm2/io module comes along with PM2. It is the PM2 library responsible for gathering the metrics, reporting exceptions, exposing remote actions and every interaction with your application.

You can also use it as a standalone agent, if you want to connect your nodejs process to PM2 Enterprise but without having to launch your application with PM2.

Table of Contents

• Installation
• Expose Custom Metrics
• Expose Remote Actions
• Report Custom Errors
• Distributed Tracing
• Configuration
• Migration Guide
• Development
• Notes

Installation

With npm:

1npm install @pm2/io --save

With yarn:

1yarn add @pm2/io

V8 Runtime Metrics

To retrieve by default V8 Runtime metrics like:

• V8 Garbage Collector metrics
• CPU Context Switch
• Page Fault

Install:

1npm install @pm2/node-runtime-stats

And restart the application.

Custom Metrics

@pm2/io allows you to gather metrics from your code to be reported in the PM2 Plus/Enterprise dashboard.

Create a custom metrics

You can create a new custom metrics with the method metric() of @pm2/io.

1const io = require('@pm2/io');
2
3const users = io.metric({
4  name: 'Realtime user',
5});
6users.set(10)

This arguments are available:

• name: The metric name (required; string)
• id: The type of metric (default 'metric', string)
• unit: unit of the measure (default ''; string)
• historic: keep the history in PM2 Plus (default: true; boolean)

There are 4 different types of metrics:

• gauge: To expose a variable's value
• counter: A discrete counter to be triggered manually to count a number of occurrence
• meter: To measure a frequency, a number of occurrences of a repeating event per unit of time
• histogram: To measure a statistic, a statistic on a metric over the last 5 minutes

Metric: Variable Exposition

The first type of metric, called metric, allows to expose a variable's value. The variable can be exposed passively, with a function that gets called every second, or actively, with a method that you use to update the value.

Active Mode

In active mode, you need to create a probe and call the method set() to update the value.

1const myMetric = io.metric({
2  name: 'Realtime Value'
3});
4
5myMetric.set(23);

Passive Mode

In passive mode you hust need to return the variable to be monitored:

1const myMetric = io.metric({
2  name: 'Realtime Value',
3  value: () => {
4    return variable_to_monitor
5  }
6});

Counter: Discrete Counter

The second type of metric, called counter, is a discrete counter that helps you count the number of occurrence of a particular event. The counter starts at 0 and can be incremented or decremented.

1const io = require('@pm2/io');
2
3const currentReq = io.counter({
4  name: 'Current req processed',
5  type: 'counter',
6});
7
8http.createServer((req, res) => {
9  // Increment the counter, counter will eq 1
10  currentReq.inc();
11  req.on('end', () => {
12    // Decrement the counter, counter will eq 0
13    currentReq.dec();
14  });
15});

Meter: Frequency

The third type of metric, called meter, compute the frequency of an event. Each time the event happens, you need to call the mark() method. By default, the frequency is the number of events per second over the last minute.

1const io = require('@pm2/io');
2
3const reqsec = io.meter({
4  name: 'req/sec',
5  type: 'meter',
6});
7
8http.createServer((req, res) => {
9  reqsec.mark();
10  res.end({ success: true });
11});

Additional options:

• samples: (optional)(default: 1) Rate unit. Defaults to 1 sec.
• timeframe: (optional)(default: 60) Timeframe over which the events will be analyzed. Defaults to 60 sec.

Histogram: Statistics

Collect values and provide statistic tools to explore their distribution over the last 5 minutes.

1const io = require('@pm2/io');
2
3const latency = io.histogram({
4  name: 'latency',
5  measurement: 'mean'
6});
7
8var latencyValue = 0;
9
10setInterval(() => {
11  latencyValue = Math.round(Math.random() * 100);
12  latency.update(latencyValue);
13}, 100);

Options are:

• measurement : default: mean; min, max, sum, count, variance, mean, stddev, median, p75, p95, p99, p99.

Expose Remote Actions: Trigger Functions remotely

Remotely trigger functions from PM2 Plus or Enterprise.

Simple actions

The function takes a function as a parameter (cb here) and need to be called once the job is finished.

Example:

1const io = require('@pm2/io');
2
3io.action('db:clean', (cb) => {
4  clean.db(() => {
5    // cb must be called at the end of the action
6    return cb({ success: true });
7  });
8});

Report user error

By default, in the Issue tab, you are only alerted for uncaught exceptions. Any exception that you catch is not reported. You can manually report them with the notifyError() method.

1const io = require('@pm2/io');
2
3io.notifyError(new Error('This is an error'), {
4  // you can some http context that will be reported in the UI
5  http: {
6    url: req.url
7  },
8  // or anything that you can like an user id
9  custom: {
10    user: req.user.id
11  }
12});

Express error reporting

If you want you can configure your express middleware to automatically send you an error with the error middleware of express :

1const io = require('@pm2/io')
2const express = require('express')
3const app = express()
4
5// add the routes that you want
6app.use('/toto', () => {
7  throw new Error('ajdoijerr')
8})
9
10// always add the middleware as the last one
11app.use(io.expressErrorHandler())

Koa error reporting

We also expose a custom koa middleware to report error with a specific koa middleware :

1const io = require('@pm2/io')
2const Koa = require('koa')
3const app = new Koa()
4
5// the order isn't important with koa
6app.use(pmx.koaErrorHandler())
7
8// add the routes that you want
9app.use(async ctx => {
10  ctx.throw(new Error('toto'))
11})

Distributed Tracing

The Distributed Tracing allows to captures and propagates distributed traces through your system, allowing you to visualize how customer requests flow across services, rapidly perform deep root cause analysis, and better analyze latency across a highly distributed set of services. If you want to enable it, here the simple options to enable:

1const io = require('@pm2/io').init({
2  tracing: {
3    enabled: true,
4    // will add the actual queries made to database, false by default
5    detailedDatabasesCalls: true,
6    // if you want you can ignore some endpoint based on their path
7    ignoreIncomingPaths: [
8      // can be a regex
9      /misc/,
10      // or a exact string
11      '/api/bucket'
12      // or a function with the request
13      (url, request) => {
14        return true
15      }
16    ],
17    // same as above but used to match entire URLs
18    ignoreOutgoingUrls: [],
19    /**
20     * Determines the probability of a request to be traced. Ranges from 0.0 to 1.0
21     * default is 0.5
22     */
23    samplingRate: 0.5
24  }
25})

By default we ignore specific incoming requests (you can override this by setting ignoreIncomingPaths: []):

• Request with the OPTIONS or HEAD method
• Request fetching a static ressources (*.js, *.css, *.ico, *.svg, .png or *webpack*)

What's get traced

When your application will receive a request from either http, https or http2 it will start a trace. After that, we will trace the following modules:

• http outgoing requests
• https outgoing requests
• http2 outgoing requests
• mongodb-core version 1 - 3
• redis versions > 2.6
• ioredis versions > 2.6
• mysql version 1 - 3
• mysql2 version 1 - 3
• pg version > 6
• vue-server-renderer version 2

Custom Tracing API

The custom tracing API can be used to create custom trace spans. A span is a particular unit of work within a trace, such as an RPC request. Spans may be nested; the outermost span is called a root span, even if there are no nested child spans. Root spans typically correspond to incoming requests, while child spans typically correspond to outgoing requests, or other work that is triggered in response to incoming requests. This means that root spans shouldn't be created in a context where a root span already exists; a child span is more suitable here. Instead, root spans should be created to track work that happens outside of the request lifecycle entirely, such as periodically scheduled work. To illustrate:

1const io = require('@pm2/io').init({ tracing: true })
2const tracer = io.getTracer()
3// ...
4
5app.get('/:token', function (req, res) {
6  const token = req.params.token
7  // the '2' correspond to the type of operation you want to trace
8  // can be 0 (UNKNOWN), 1 (SERVER) or 2 (CLIENT)
9  // 'verifyToken' here will be the name of the operation
10  const customSpan = tracer.startChildSpan('verifyToken', 2)
11  // note that customSpan can be null if you are not inside a request
12  req.Token.verifyToken(token, (err, result) => {
13    if (err) {
14      // you can add tags to the span to attach more details to the span
15      customSpan.addAttribute('error', err.message)
16      customSpan.end()
17      return res.status(500).send('error')
18    }
19    customSpan.addAttribute('result', result)
20    // be sure to always .end() the spans
21    customSpan.end()
22    // redirect the user if the token is valid
23    res.send('/user/me')
24  })
25})
26
27// For any significant work done _outside_ of the request lifecycle, use
28// startRootSpan.
29const traceOptions = {
30    name: 'my custom trace',
31    // the '1' correspond to the type of operation you want to trace
32    // can be 0 (UNKNOWN), 1 (SERVER) or 2 (CLIENT)
33    kind: '1'
34  }
35plugin.tracer.startRootSpan(traceOptions, rootSpan => {
36  // ...
37  // Be sure to call rootSpan.end().
38  rootSpan.end()
39});

Configuration

Global configuration object

1export class IOConfig {
2  /**
3   * Automatically catch unhandled errors
4   */
5  catchExceptions?: boolean = true
6  /**
7   * Configure the metrics to add automatically to your process
8   */
9  metrics?: {
10    eventLoop: boolean = true,
11    network: boolean = false,
12    http: boolean = true,
13    gc: boolean = true,
14    v8: boolean = true
15  }
16  /**
17   * Configure the default actions that you can run
18   */
19  actions?: {
20    eventLoopDump?: boolean = true
21  }
22  /**
23   * Configure availables profilers that will be exposed
24   */
25  profiling?: {
26    /**
27     * Toggle the CPU profiling actions
28     */
29    cpuJS: boolean = true
30    /**
31     * Toggle the heap snapshot actions
32     */
33    heapSnapshot: boolean = true
34    /**
35     * Toggle the heap sampling actions
36     */
37    heapSampling: boolean = true
38    /**
39     * Force a specific implementation of profiler
40     *
41     * available:
42     *  - 'addon' (using the v8-profiler-node8 addon)
43     *  - 'inspector' (using the "inspector" api from node core)
44     *  - 'none' (disable the profilers)
45     *  - 'both' (will try to use inspector and fallback on addon if available)
46     */
47    implementation: string = 'both'
48  }
49  /**
50   * Configure the transaction tracing options
51   */
52  tracing?: {
53    /**
54     * Enabled the distributed tracing feature.
55     */
56    enabled: boolean
57    /**
58     * If you want to report a specific service name
59     * the default is the same as in apmOptions
60     */
61    serviceName?: string
62    /**
63     * Generate trace for outgoing request that aren't connected to a incoming one
64     * default is false
65     */
66    outbound?: boolean
67    /**
68     * Determines the probability of a request to be traced. Ranges from 0.0 to 1.0
69     * default is 0.5
70     */
71    samplingRate?: number,
72    /**
73     * Add details about databases calls (redis, mongodb etc)
74     */
75    detailedDatabasesCalls?: boolean,
76    /**
77     * Ignore specific incoming request depending on their path
78     */
79    ignoreIncomingPaths?: Array<IgnoreMatcher<httpModule.IncomingMessage>>
80    /**
81     * Ignore specific outgoing request depending on their url
82     */
83    ignoreOutgoingUrls?: Array<IgnoreMatcher<httpModule.ClientRequest>>
84    /**
85     * Set to true when wanting to create span for raw TCP connection
86     * instead of new http request
87     */
88    createSpanWithNet: boolean
89  }
90  /**
91   * If you want to connect to PM2 Enterprise without using PM2, you should enable
92   * the standalone mode
93   *
94   * default is false
95   */
96  standalone?: boolean = false
97  /**
98   * Define custom options for the standalone mode
99   */
100  apmOptions?: {
101    /**
102     * public key of the bucket to which the agent need to connect
103     */
104    publicKey: string
105    /**
106     * Secret key of the bucket to which the agent need to connect
107     */
108    secretKey: string
109    /**
110     * The name of the application/service that will be reported to PM2 Enterprise
111     */
112    appName: string
113    /**
114     * The name of the server as reported in PM2 Enterprise
115     *
116     * default is os.hostname()
117     */
118    serverName?: string
119    /**
120     * Broadcast all the logs from your application to our backend
121     */
122    sendLogs?: Boolean
123    /**
124     * Avoid to broadcast any logs from your application to our backend
125     * Even if the sendLogs option set to false, you can still see some logs
126     * when going to the log interface (it automatically trigger broacasting log)
127     */
128    disableLogs?: Boolean
129    /**
130     * Since logs can be forwared to our backend you may want to ignore specific
131     * logs (containing sensitive data for example)
132     */
133    logFilter?: string | RegExp
134    /**
135     * Proxy URI to use when reaching internet
136     * Supporting socks5,http,https,pac,socks4
137     * see https://github.com/TooTallNate/node-proxy-agent
138     *
139     * example: socks5://username:password@some-socks-proxy.com:9050
140     */
141    proxy?: string
142  }
143}

You can pass whatever options you want to io.init, it will automatically update its configuration.

Migration Guides

2.x to 3.x

Here the list of breaking changes :

• Removed io.scopedAction because of low user adoption
• Removed io.notify in favor of io.notifyError (droppin replacement)
• Removed support for gc-stats module
• Removed Heap profiling support when using the profiler addon (which wasn't possible at all)
• Removed deep-metrics support (the module that allowed to get metrics about websocket/mongo out of the box), we are working on a better solution.
• Removed io.transpose
• Removed io.probe() to init metrics
• Changed the configuration structure

High chance that if you used a custom configuration for io.init, you need to change it to reflect the new configuration. Apart from that and the io.notify removal, it shouldn't break the way you instanciated metrics. If you find something else that breaks please report it to us (tech@keymetrics.io).

3.x to 4.x

The only difference with the 4.x version is the new tracing system put in place, so the only changs are related to it:

• Dropped the support for node 4 (you can still use the 3.x if you use node 4 but you will not have access to the distributed tracing)
• Changed the tracing configuration (see options above)

Development

To auto rebuild on file change:

1$ npm install
2$ npm run watch

To test only one file:

1$ npm run unit <typescript-file-to-test.ts>

Run transpilation + test + coverage:

1$ npm run test

Run transpilation + test only:

1$ npm run unit <test>

Notes

Curently this package isn't compatible with amqp if you use the network metrics. We recommend to disable the metrics with the following configuration in this case :

1io.init({
2  metrics: {
3    network: false
4  }
5})