Microservice development on local part 1: Docker Compose

(Looking for how to get Docker Compose logs? Check out Part 2)

Picture this: You have a service backend that has 20 features split among 4 teams. The existing monolith is becoming a pain because one bad line of code may take down the entire service. It’s also getting expensive with its growing hunger for computing resources. Hence, the decision came to breaking it into a set of self-contained microservices.

The advantages are clear. From microservices.io, it creates a seperation of concerns between each feature. Now each team only needs to worry about their own services. Furthermore, these services can scale on customer demand.

But that creates a new development situation: Where everything used to be in a single repo, it is now in many repos. Running npm run start is now done 5 times in 5 terminals before getting anything done. Plus, changing what’s in each service’s .env means going to 5 different folders.

Enter Docker Compose

Docker Compose brings back starting your entire development environment with just 1 command. Why not replicate a prod environment on local for testing and debugging? This skips having to deploy to a non-prod environment on a paid cluster. Being on local, exposing the Docker Compose environment allows accessing local services.

The setup in this article will resemble this repo. It contains 3 services:

1. nestjs-orders-app (PORT 3200): Records and retrieves orders in DB
2. operations-portal (PORT 3000): Allows viewing and editing incoming orders. Count of send attempts maintained using state machine
3. payment-app (PORT 3400): Process payments. Randomly decides payment response to mock payment failure

GitHub - YFLooi/local-docker-microservice-setup: Run multiple repos as microservices on local using…

Setting up the Docker Compose yml

Let’s go over docker-compose.local-microservices.yml. Using a custom name instead of docker-compose.yml is helpful for organisation. Running this yml involves opening a terminal in the same folder and running:

docker compose -f docker-compose.local-microservices.yml up

If it was named docker-compose.yml instead, it can run using:

docker compose up

Now for what’s in the yml:

version: "3.8"
services:
  nestjs-orders-app:
    image: nestjs-orders-app
    build:
      context: nestjs-orders-app
      dockerfile: ./deployment/Dockerfile
      args:
        SAMPLE_AUTH_TOKEN: token-token
    environment:
      - SAMPLE_ENV=env-variable-string-value
    restart: on-failure
    network_mode: host
    volumes:
      - ./nestjs-orders-app:/home/node/app
  operations-portal:
    image: operations-portal
    build:
      context: operations-portal
      dockerfile: ./deployment/Dockerfile
    network_mode: host
    volumes:
      - ./operations-portal:/home/node/app
  payment-app:
    image: payment-app
    build:
      context: payment-app
      dockerfile: ./deployment/Dockerfile
    restart: on-failure
    network_mode: host
    volumes:
      - ./payment-app:/home/node/app
  # Ex of getting pre-built image from Dockerhub
  # redis:
  #   image: redis:3.0-alpine
  #   ports:
  #       - "6379:6379"

That’s a lot of stuff. Let’s go over the keywords:

version: “3.8”

This specifies the Compose file format is for version 3.8. Different versions are compatible with different Docker engine releases.

services:
  nestjs-orders-app:
    image: nestjs-orders-app
    build:
      context: nestjs-orders-app
      dockerfile: ./deployment/Dockerfile
      args:
        SAMPLE_AUTH_TOKEN: token-token
    environment:
      - SAMPLE_ENV=env-variable-string-value
    restart: on-failure
    network_mode: host
    volumes:
      - ./nestjs-orders-app:/home/node/app

services are the basic running units in Compose. This setting creates a service named nestjs-orders-app and its built image would have the same name. Hence, we want each repo to run on its own service.

build specifies the settings for assembling the repo’s image. Going over the settings:

• context
  Root folder of repo. By default, context is relative to the location of the Compose file. Here, it’s changed to the nestjs-orders-app repo folder.
• dockerfile
  A Dockerfile contains instructions for building and starting images. With the context set to the nestjs-orders-app repo folder, specify the path to the Dockerfile within /deployment folder
  Tip: Your production repo would usually have its own Dockerfile. Place the Dockerfile for your local Compose service in a folder with name starting with . to exclude it from Git (hidden folder). After that, set context and dockerfile accordingly.
• args
  Think of these as environment variables used at build time. The Dockerfile uses them where ARG is defined.

environment specifies the environment variables used at container runtime. Defining them in the Compose file instead of a .env file in the repo’s root makes them possible to change from 1 location.

restart determines handling for terminated containers, typically those that fail to start. on-failure means the container will attempt to start again if the exit code indicates an error and it will keep going until successful.

network_mode sets the access permission of the service container. By default, Compose services cannot access services outside the Compose environment. Here, setting to host allows accessing host services. I typically use this setting with MongoDB on my local because it allows me to persist test data and query them with MongoDB Compass =D

Note that network_mode: host is incompatible with ports which sets the port exposed by a container. Hence, the service’s port follows that defined in the repo. For example, nestjs-orders-app is set to use port 3200.

volumes specifies the location to persist data generated and used by Docker containers.

Starting and stopping Docker Compose

With the settings above defined, open a terminal in the same folder as the Compose yml and build the service images with:

docker compose -f docker-compose.local-microservices.yml build

During development, run this command again each time new changes need to be included.

Start by entering:

docker compose -f docker-compose.local-microservices.yml up

To run containers in background without terminal logs, run as:

docker compose -f docker-compose.local-microservices.yml up --detach

To include environment variables for all containers, define a .env file and add the file path to the start command:

docker compose -f docker-compose.local-microservices.yml --env-file ./.env up

Logs upon starting Docker Compose

For this example repo, wait a couple of seconds for all containers to start. Opening http://localhost:3000 in your browser would reveal the following website. Try it out and see the logs that appear in the terminal:

Website running off example repo’s web service

Now to stop. If you’re not running containers in background, press Ctrl+c in the running terminal. Else:

docker compose -f docker-compose.local-microservices.yml down

To stop and also remove all created volumes (handy for saving disk space):

docker compose -f docker-compose.local-microservices.yml down -v

Logs upon stopping Docker Compose

That’s all for now! Part 2 shows how to find and examine the logs coming from Compose containers.

Tips

• Using an older version of Docker Compose and getting unexpected errors? Were the commands same as above and starting with docker compose?
  Use docker-compose instead. Ex:
  docker-compose -f docker-compose.local-microservices.yml up
• Prefer to use a database or cache service that is within the Docker Compose environment? Grab its image from Dockerhub by specifying its image name in the Docker Compose yml:

database: #service name
  image: postgres:9.4 # image name
  ports:
     - "5432:5432"

References

Article repo: https://github.com/YFLooi/local-docker-microservice-setup

https://www.ais.com/using-docker-compose-to-locally-develop-and-test-microservices/

GitHub - YFLooi/local-docker-microservice-setup: Run multiple repos as microservices on local using…