multer-sharp-s3

---

Multer Sharp S3 is streaming multer storage engine permit to transform / resize the image and upload to AWS S3.

This project is mostly an integration piece for existing code samples from Multer's storage engine documentation. With add-ons include AWS S3 and sharp

Minimum Requirement:

Node v12.13.0, npm v6+

Installation

npm:

npm install --save aws-sdk multer-sharp-s3

yarn:

yarn add aws-sdk multer-sharp-s3

Tests

Change aws configuration in your local.

yarn test

Importing

NodeJS

1const s3Storage = require('multer-sharp-s3');
2
3const storage = s3Storage(options);

TypeScript

1import * as s3Storage from 'multer-sharp-s3';
2
3const storage = s3Storage(options);

Usage

1const express = require('express');
2const multer = require('multer');
3const s3Storage = require('multer-sharp-s3');
4const aws = require('aws-sdk');
5
6aws.config.update({
7  secretAccessKey: config.uploads.aws.secretAccessKey, // Not working key, Your SECRET ACCESS KEY from AWS should go here, never share it!!!
8  accessKeyId: config.uploads.aws.accessKeyId, // Not working key, Your ACCESS KEY ID from AWS should go here, never share it!!!
9  region: config.uploads.aws.region, // region of your bucket
10})
11
12const s3 = new aws.S3()
13const app = express();
14
15// without resize image
16const storage = s3Storage({
17  s3,
18  Bucket: config.uploads.aws.Bucket,
19  Key: `${config.uploads.aws.Bucket}/test/${Date.now()}-myImage`,
20  ACL: config.uploads.aws.ACL,
21})
22const upload = multer({ storage: storage })
23
24app.post('/upload', upload.single('myPic'), (req, res) => {
25    console.log(req.file); // Print upload details
26    res.send('Successfully uploaded!');
27});
28
29// or
30
31// single resize without Key
32const storage2 = gcsSharp({
33  s3,
34  Bucket: config.uploads.aws.Bucket,
35  ACL: config.uploads.aws.ACL,
36  resize: {
37    width: 400,
38    height: 400
39  },
40  max: true
41});
42const upload2 = multer({ storage: storage2 });
43
44app.post('/uploadandresize', upload2.single('myPic'), (req, res, next) => {
45    console.log(req.file); // Print upload details
46    res.send('Successfully uploaded!');
47});
48
49/* If you need generate image with specific size
50 * simply to adding `multiple: true` property and
51 * resize must be an `array` and must be include `suffix` property
52 * and suffix has a special value that is 'original'
53 * it will no transform image, just upload the image as is
54 * example below with `Key` as callback function
55 */
56const storage = s3Storage({
57  Key: (req, file, cb) => {
58    crypto.pseudoRandomBytes(16, (err, raw) => {
59      cb(err, err ? undefined : raw.toString('hex'))
60    })
61  },
62  s3,
63  Bucket: config.uploads.aws.Bucket,
64  multiple: true,
65  resize: [
66    { suffix: 'xlg', width: 1200, height: 1200 },
67    { suffix: 'lg', width: 800, height: 800 },
68    { suffix: 'md', width: 500, height: 500 },
69    { suffix: 'sm', width: 300, height: 300 },
70    { suffix: 'xs', width: 100 },
71    { suffix: 'original' }
72  ],
73});
74const upload = multer({ storage });
75
76app.post('/uploadmultiplesize', upload.single('myPic'), (req, res, next) => {
77    console.log(req.file); // print output
78    res.send('Successfully uploaded!');
79});
80
81/* 
82 *  If the directory property exists, 
83 *  the suffix property is ignored and 
84 *  inserted separated by Bucket's directory.
85 */
86const storage3 = s3Storage({
87  Key: (req, file, cb) => {
88    crypto.pseudoRandomBytes(16, (err, raw) => {
89      cb(err, err ? undefined : raw.toString('hex'))
90    })
91  },
92  s3,
93  Bucket: config.uploads.aws.Bucket,
94  multiple: true,
95  resize: [
96    { suffix: 'lg', directory: 'large', width: 800, height: 800 },  // insert BUCKET/large/filename
97    { suffix: 'md', directory: 'medium', width: 500, height: 500 }, // insert BUCKET/medium/filename
98    { suffix: 'sm', directory: 'small', width: 300, height: 300 },  // insert BUCKET/small/filename
99  ],
100});
101const upload3 = multer({ storage3 });
102
103app.post('/uploadmultiplesize', upload3.single('myPic'), (req, res, next) => {
104    console.log(req.file); // print output
105    res.send('Successfully uploaded!');
106});
107
108// also can upload any file (non image type)
109const storage = s3Storage({
110  s3,
111  Bucket: config.uploads.aws.Bucket,
112  Key: `${config.uploads.aws.Bucket}/test/${Date.now()}-myFile`,
113  ACL: config.uploads.aws.ACL,
114  // resize or any sharp options will ignore when uploading non image file type
115  resize: {
116    width: 400,
117    height: 400,
118  },
119})
120const upload = multer({ storage })
121
122app.post('/uploadfile', upload.single('myFile'), (req, res, next) => {
123    console.log(req.file); // print output
124    res.send('Successfully uploaded!');
125});
126

for more example you can see here

Multer-Sharp-S3 options

multer sharp s3 is inherit from s3 upload property putObjectRequest. Below are special / custom options from this package

option	default	value	role
S3	no	object	instance from AWS.S3 class. it mus be specify
Key	randomString	string or function	your s3 Key
Bucket	no	string	Required your bucket name on AWS S3 to upload. Environment variable - AWS_BUCKET
ACL	'public-read'	string	Required acl credentials file for AWS S3
multiple	false	boolean	for multiple resize to work
resize	no	object or Array<object> when multiple is true. Note: suffix must be specify when using resize as Array	size specification

Sharp options

Please visit this sharp for detailed overview of specific option.

multer sharp s3 embraces sharp options, as table below:

option	default	value	role
resize	undefined	object for output image, as follow: { width?: 300, height?: 200, options?: {...resizeOptions} }. doc: sharpResizeOptions	size specification
crop	undefined		crop image
background	undefined		set the background for the embed, flatten and extend operations.
embed	undefined		embed on canvas
max	undefined		set maximum output dimension
min	undefined		set minimum output dimension
withoutEnlargement	undefined		do not enlarge small images
ignoreAspectRatio	undefined		ignore aspect ration while resizing images
extract	undefined		extract specific part of image
trim	undefined		Trim boring pixels from all edges
flatten	undefined		Merge alpha transparency channel, if any, with background.
extend	undefined		Extends/pads the edges of the image with background.
negate	undefined		Produces the negative of the image.
rotate	undefined		Rotate the output image by either an explicit angle
flip	undefined		Flip the image about the vertical Y axis.
flop	undefined		Flop the image about the horizontal X axis.
blur	undefined		Mild blur of the output image
sharpen	undefined		Mild sharpen of the output image
gamma	undefined		Apply a gamma correction.
grayscale or greyscale	undefined		Convert to 8-bit greyscale; 256 shades of grey.
normalize or normalise	undefined		Enhance output image contrast by stretching its luminance to cover the full dynamic range.
withMetadata	undefined		Include all metadata (EXIF, XMP, IPTC) from the input image in the output image.
convolve	undefined		Convolve the image with the specified kernel.
threshold	undefined		Any pixel value greather than or equal to the threshold value will be set to 255, otherwise it will be set to 0
toColourspace or toColorspace	undefined		Set the output colourspace. By default output image will be web-friendly sRGB, with additional channels interpreted as alpha channels.
toFormat	undefined	'jpeg', 'png', 'magick', 'webp', 'tiff', 'openslide', 'dz', 'ppm', 'fits', 'gif', 'svg', 'pdf', 'v', 'raw' or object. if object specify as follow: { type: 'png', options: { ...toFormatOptions } } doc: sharpToFormat	type of output file to produce.

NOTE Some of the contents in the above table maybe is not be updated, you can check more here

---

Why

Because We need to transform an image using sharp and upload it to AWS S3 using multer middleware at once. Build on top with TypeScript and reactive using RxJS as helper library in this package.

refer to: intro rx

License

MIT Copyright (c) 2017 - forever Abdul Fattah Ikhsan