@Gapi Advanced Starter

@Nginx, @Rabbitmq, @Postgres, @Sequelize, @Docker, @Graphql

This is advanced example project related with GAPI

To check basic example project go to basic-example

Included @gapi/sequelize module

To start developing clone repository

1git clone https://github.com/Stradivario/gapi-starter-postgres-sequelize

Better use command line utility(gapi) to install it type following command:

1npm i -g @gapi/cli

Type the following command to create new project from scratch via CLI

1gapi new my-project --advanced

To start project for "development" type:

1npm start

To stop project for "production" type:

Following command will stop pm2 processes started

1npm run stop:prod

Testing

To start developing with testing GAPI uses JEST and gapi-cli is preconfigurated for your needs! :)

To run single test type:

1gapi test

Testing watch mode

Note: You need to start server before running tests

Note: Everytime you make change to server it will restart server and execute tests

Note: To add more tests just create e2e.spec.ts or unit.spec.ts somewhere inside the application

Start the application

1gapi start

Execute test with --watch argument

1gapi test --watch

You will end up with something like this

Custom logic before testing ( for example creating MOCK users to database before testing)

Create file test.ts inside root/src/test.ts with this content

Everytime you run test with --before argument it will set environment variable BEFORE_HOOK

1  if (process.env.BEFORE_HOOK) {
2    // do something here
3  }

Then execute tests with --before

1gapi test --before

This command will start root/src/test.ts file and will wait for process.exit(0) so you can customize your before logic check this link for reference

Following commands will start RabbitMQ, PostgreSQL, API, NGINX as a services you need DOCKER for them

API will be served on https://localhost:80 and https://localhost:80/subscriptions

Docker

To build project with Docker type:

1gapi app build

To start project with Docker type:

1gapi app start

To stop project type:

1gapi app stop

Workers

All workers will be mapped as Proxy and will be reverted to https://localhost:80 and https://localhost:80/subscriptions

So you don't have to worry about if some of your workers stopped responding

TODO: Create monitoring APP for all workers and main API

To start workers type:

1gapi workers start

To stop workers type:

1gapi workers stop

To add more workers

By default there are 4 workers with 4 processes with "exec_mode: cluster" of the original process inside single docker container

You can control Processes inside single docker container from "root/process.yml" file.

1apps:
2  - script   : './src/main.ts'
3    name     : 'APP'
4    exec_mode: 'cluster'
5    instances: 4
6

To map new worker as a stream open root/nginx/config/private/default

1upstream app_servers {
2    server 182.10.0.3:9000; # Main process
3    server 182.10.0.21:9000; # Worker 1
4    server 182.10.0.22:9000; # Worker 2
5    server 182.10.0.23:9000; # Worker 3
6    server 182.10.0.24:9000; # Worker 4
7
8    # Add more workers here
9    # server 182.10.0.25:9000; # Worker 5
10}
11
12server {
13    listen 80;
14    server_name api.yourdomain.com;
15    access_log api-yourdomain.access.log;
16
17    location / {
18       proxy_set_header Upgrade $http_upgrade;
19       proxy_set_header Connection "upgrade";
20       client_max_body_size 50M;
21       proxy_set_header Host $http_host;
22       proxy_set_header X-Real-IP $remote_addr;
23       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
24       proxy_set_header X-Forwarded-Proto $scheme;
25       proxy_set_header X-Frame-Options SAMEORIGIN;
26       proxy_buffers 256 16k;
27	     proxy_buffering off;
28       proxy_buffer_size 16k;
29       proxy_read_timeout 600s;
30       proxy_pass http://app_servers;
31    }
32
33    location /subscriptions {
34         # prevents 502 bad gateway error
35        proxy_buffers 8 32k;
36        proxy_buffer_size 64k;
37
38        # redirect all HTTP traffic to localhost:9000;
39        proxy_pass http://app_servers/subscriptions;
40        proxy_set_header X-Real-IP $remote_addr;
41        proxy_set_header Host $http_host;
42        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
43        #proxy_set_header X-NginX-Proxy true;
44
45        # enables WS support
46        proxy_http_version 1.1;
47        proxy_set_header Upgrade $http_upgrade;
48        proxy_set_header Connection "upgrade";
49	      proxy_buffering off;
50        proxy_read_timeout 999999999;
51
52    }
53    if ($scheme = http) {
54       return 301 https://$server_name$request_uri;
55    }
56    listen 443;
57    ssl on;
58    ssl_certificate         /usr/share/certs/cert.pem;
59    ssl_certificate_key     /usr/share/certs/cert.key;
60}
61
62

When you add another worker it should be on different IP with same port 9000

Open root/gapi.conf.yml file you will find this file:

1config:
2# Application configuration
3  app: 
4    local:
5      API_PORT: 9000
6      API_CERT: ./cert.key
7      NODE_ENV: development
8      AMQP_HOST: 182.10.0.5
9      AMQP_PORT: 5672
10      DB_PORT: 5432
11      DB_NAME: postgres
12      DB_HOST: 182.10.0.4
13      DB_USERNAME: dbuser
14      DB_PASSWORD: dbuserpass
15      GRAPHIQL: true
16      ENDPOINT_TESTING: http://localhost:9000/graphql
17      TOKEN_TESTING: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImtyaXN0aXFuLnRhY2hldkBnbWFpbC5jb20iLCJzY29wZSI6WyJBRE1JTiJdLCJpZCI6MSwiaWF0IjoxNTE2OTk2MzYxfQ.7ANr5VHrViD3NkCaDr0nSWYwk46UAEbOwB52pqye4AM
18      GRAPHIQL_TOKEN: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImtyaXN0aXFuLnRhY2hldkBnbWFpbC5jb20iLCJpZCI6MSwic2NvcGUiOlsiQURNSU4iXSwiaWF0IjoxNTIwMjkxMzkyfQ.9hpIDPkSiGvjTmUEyg_R_izW-ra2RzzLbe3Uh3IFsZg
19    prod:
20      API_PORT: 9000
21      API_CERT: ./cert.key
22      NODE_ENV: production
23      AMQP_HOST: 182.10.0.5
24      AMQP_PORT: 5672
25      DB_PORT: 5432
26      DB_HOST: 182.10.0.4
27      DB_USERNAME: dbuser
28      DB_PASSWORD: dbuserpass
29      DB_NAME: postgres
30# Testing configuration for local(dev) or worker(running tests as a separate worker with separate database)
31  test: 
32    local: extends app/local
33    worker:
34      API_PORT: 9000
35      API_CERT: ./cert.key
36      NODE_ENV: production
37      DB_PORT: 5432
38      DB_HOST: 182.10.0.99
39      AMQP_HOST: 182.10.0.5
40      AMQP_PORT: 5672
41      DB_USERNAME: dbuser
42      DB_PASSWORD: dbuserpass
43      DB_NAME: postgres
44      ENDPOINT_TESTING: http://182.10.0.101:9000/graphql
45      TOKEN_TESTING: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImtyaXN0aXFuLnRhY2hldkBnbWFpbC5jb20iLCJzY29wZSI6WyJBRE1JTiJdLCJpZCI6MSwiaWF0IjoxNTE2OTk2MzYxfQ.7ANr5VHrViD3NkCaDr0nSWYwk46UAEbOwB52pqye4AM
46  schema:
47    introspectionEndpoint: http://localhost:9000/graphql
48    introspectionOutputFolder: ./src/app/core/api-introspection
49
50commands:
51  testing:
52    stop:
53      - docker rm -f gapi-api-prod-worker-tests-executor
54      - docker rm -f gapi-api-prod-worker-tests-provider
55    start:
56      - gapi testing start-provider
57      - sleep 10
58      - gapi testing start-executor
59      - echo Cleaning...
60      - gapi testing stop
61    start-executor:
62      - docker run -d --network=gapiapiprod_gapi --ip=182.10.0.100 --name gapi-api-prod-worker-tests-executor gapi/api/prod
63      - docker exec gapi-api-prod-worker-tests-provider npm -v
64      - gapi test --worker --before
65    start-provider: docker run -d --network=gapiapiprod_gapi --ip=182.10.0.101 --name gapi-api-prod-worker-tests-provider gapi/api/prod
66  workers:
67    start:
68      - gapi workers start-1
69      - gapi workers start-2
70      - gapi workers start-3
71      - gapi workers start-4
72    stop:
73      - docker rm -f gapi-api-prod-worker-1
74      - docker rm -f gapi-api-prod-worker-2
75      - docker rm -f gapi-api-prod-worker-3
76      - docker rm -f gapi-api-prod-worker-4
77    start-1: docker run -d --network=gapiapiprod_gapi --ip=182.10.0.21 --name gapi-api-prod-worker-1 gapi/api/prod
78    start-2: docker run -d --network=gapiapiprod_gapi --ip=182.10.0.22 --name gapi-api-prod-worker-2 gapi/api/prod
79    start-3: docker run -d --network=gapiapiprod_gapi --ip=182.10.0.23 --name gapi-api-prod-worker-3 gapi/api/prod
80    start-4: docker run -d --network=gapiapiprod_gapi --ip=182.10.0.24 --name gapi-api-prod-worker-4 gapi/api/prod
81    example-worker-with-port: docker run -d --network=gapiapiprod_gapi --ip=182.10.0.25 --name gapi-api-prod-worker-5 -p 9001:9000 gapi/api/prod
82  app:
83    start:
84      - docker-compose -p gapi-api-prod up --force-recreate -d
85      - gapi rabbitmq enable-dashboard
86    stop:
87      - gapi nginx stop
88      - gapi api stop
89      - gapi rabbitmq stop
90      - gapi postgres stop
91    build: docker build -t gapi/api/prod .
92  api:
93    stop: docker rm -f gapi-api-prod
94  nginx:
95    stop: docker rm -f gapi-api-nginx
96  postgres:
97    stop: docker rm -f gapi-api-postgres
98  rabbitmq:
99    stop: docker rm -f gapi-api-rabbitmq
100    restart: docker restart gapi-api-rabbitmq
101    enable-dashboard: docker exec gapi-api-rabbitmq rabbitmq-plugins enable rabbitmq_management
102
103# You can define your custom commands for example 
104# commands:
105#   your-cli:
106#     my-command: 'npm -v'
107# This command can be executed as "gapi your-cli my-command"

Adding one more worker:

1start-5: 'docker run -d --network=gapiapiprod_gapi --ip=182.10.0.25 --name gapi-api-prod-worker-5 -p 9005:9000 gapi/api/prod'

Then edit start task inside workers to start new worker 5

1start: 'gapi workers start-1 && gapi workers start-2 && gapi workers start-3 && gapi workers start-4 & gapi workers start-5'

Thats' it!! Now you have 4 processes like CLUSTERS inside 1 docker container with ip 182.10.0.25 and external port(optional) 9005;

You can specify worker also without port because all workers are inside internal network called "gapiapiprod_gapi"

182.10.0.21/22/23/24/25:9000

1start-5: 'docker run -d --network=gapiapiprod_gapi --ip=182.10.0.25 --name gapi-api-prod-worker-5 gapi/api/prod'

If you want to change port forwarding to another port you need to set just nginx configuration:

1  nginx:
2    image: sameersbn/nginx:1.10.1-5
3    ports:
4      - "81:80"
5      - "443:443"

Now you can find your API served onto https://localhost:81/ and https://localhost:81/subscriptions

All workers don't care about that because they will be served and mapped from nginx to port 80.

You can check docker-compose file to configurate environment variables

1version: '2'
2services:
3
4  nginx:
5    image: sameersbn/nginx:1.10.1-5
6    ports:
7      - "81:80"
8      - "444:443"
9    volumes:
10      - ./nginx/config:/etc/nginx
11      - ./nginx/html:/usr/share/nginx/html/
12      - ./nginx/certs:/usr/share/certs
13    restart: always
14    container_name: gapi-api-nginx
15    networks:
16      gapi:
17        ipv4_address: 182.10.0.2
18
19  api:
20    image: gapi/api/prod:latest
21    ports:
22      - "9000"
23    environment:
24      - NODE_ENV=production
25      - API_PORT=9000
26      - GRAPHIQL_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InRhNGVuc3RvcmVAZ21haWwuY29tIiwic2NvcGUiOlsiQURNSU4iXSwiaWQiOjIsImlhdCI6MTUxMTk3NDkzNX0.M9PnW2IrVp4XGRvbzFrl0tx6vqs6oXItFK-wF5roneI
27      - AMQP_HOST=182.10.0.5
28      - AMQP_PORT=5672
29      - CERT_PATH=
30
31      # Production db config
32      - DB_HOST=182.10.0.4
33      - DB_PORT=5432
34      - DB_NAME=postgres
35      - DB_USERNAME=dbuser
36      - DB_PASSWORD=dbuserpass
37
38    restart: always
39    mem_limit: 1000000000
40    cpu_shares: 73
41    container_name: gapi-api-prod
42    depends_on:
43      - nginx
44      - rabbitMq
45      - PostgreSQLDev
46    networks:
47      gapi:
48        ipv4_address: 182.10.0.3
49
50  PostgreSQLDev:
51    image: sameersbn/postgresql:9.5-3
52    ports:
53      - "5432"
54    environment:
55      - DEBUG=false
56      - TIMEZONE=Europe/Sofia
57      - LOCALE=bg_BG.UTF-8
58
59      - DB_USER=dbuser
60      - DB_PASS=dbuserpass
61      - DB_NAME=postgres
62      - DB_TEMPLATE=
63
64      - DB_EXTENSION=
65
66      - REPLICATION_MODE=
67      - REPLICATION_USER=
68      - REPLICATION_PASS=
69      - REPLICATION_SSLMODE=
70    restart: always
71    container_name: gapi-api-postgres
72    networks:
73      gapi:
74        ipv4_address: 182.10.0.4
75
76  PostgreSQLDevTesting:
77    image: sameersbn/postgresql:9.5-3
78    ports:
79      - "5432"
80    environment:
81      - DEBUG=false
82      - TIMEZONE=Europe/Sofia
83      - LOCALE=bg_BG.UTF-8
84
85      - DB_USER=dbuser
86      - DB_PASS=dbuserpass
87      - DB_NAME=postgres
88      - DB_TEMPLATE=
89
90      - DB_EXTENSION=
91
92      - REPLICATION_MODE=
93      - REPLICATION_USER=
94      - REPLICATION_PASS=
95      - REPLICATION_SSLMODE=
96    restart: always
97    container_name: gapi-api-postgres-testing
98    networks:
99      gapi:
100        ipv4_address: 182.10.0.99
101
102  rabbitMq:
103    image: rabbitmq:3.7.2
104    ports:
105      - "15672:15672"
106      - "5672:5672"
107      - "5671:5671"
108      - "4369:4369"
109    restart: always
110    container_name: gapi-api-rabbitmq
111    networks:
112      gapi:
113        ipv4_address: 182.10.0.5
114
115  pgadmin:
116    image: thajeztah/pgadmin4
117    ports:
118      - "5050"
119    volumes:
120      - /usr/bin/:/usr/pg
121      - /usr/database/:/usr/database
122      - /tmp/:/tmp
123    restart: always
124    container_name: gapi-api-pg-admin
125    networks:
126      gapi:
127        ipv4_address: 182.10.0.6
128
129networks:
130  gapi:
131    driver: bridge
132    ipam:
133     config:
134       - subnet: 182.10.0.0/16
135         gateway: 182.10.0.1
136
137

After successfully started project you can open your browser to localhost:80 or 182.10.0.3:9000 the api will be served there

TODO: Better documentation...

Enjoy ! :)