XVI. A docker-compose file for the Spring Cloud based microservices

elevysi · 06 August 2019 |

Time has come to round things up by reminding ourselves of what we have done so far:

  • We have created a basic Spring Boot Application for managing posts and called it the basicservice; the service relies on a mysql database refered to as basicdb.
  • We introduced a second Spring Boot Application for handling authorization and authentication, we called it the authenticationservice; the application also has its own mysql database called authdb
  • We created a third Spring Boot Application for service discovery and called it the discoveryservice
  • We have introduced Oauth2 and made the authenticationservice a single sign on server for the basicservice
    • We also implemented authentication through social identity providers, besides the local authentication scheme offered thanks to the application database
  • We introduced a Spring Cloud Based Configuration server and called it the configurationservice; it is also a Spring Boot Application of its own
  • We have seen how we can automate a Spring Boot application docker image build using Maven

To sum things up, here are what we have got going:

  • 4 Spring Boot Applications; i.e. basicservice, authenticationservice, discoveryservice and configurationservice
  • 2 mysql databases; i.e. basidb and authdb

We now need a way to orchestrate the deployment of the ecosystem efficiently into production; the tool to help us achieve such an aim will be docker-compose, a docker tool that allows us to define and run multi-container docker applications. We will create a YAML file and name it docker-compose.yml; this file will ultimately help us deploy the docker images for both the Spring Boot Applications and their mysql databases. A particular care needs to be given to the persistence of things such as the database information in cases the docker images are restarted for instance. We will hence introduce and use the notion of volumes for achieving such an aim.
Let's go ahead and start filling in the services of the ecosystem into the docker-compose file; let's start by declaring the basicdb docker image below:

version: '2.2'
services:
  basicdb:
      image: mysql:5.7
      environment:
          MYSQL_DATABASE: basicdb
          MYSQL_USER: root
          MYSQL_PASSWORD: root
          MYSQL_ROOT_PASSWORD: root
      volumes:
          - "./my.cnf:/etc/mysql/conf.d/config-file.cnf"
          - "./basicdb:/var/lib/mysql:rw"
      ports:
          - "3317:3306"
      networks:
          - network-course
  authenticationdb:
      image: mysql:5.7
      environment:
          MYSQL_DATABASE: authdb
          MYSQL_USER: root
          MYSQL_PASSWORD: root
          MYSQL_ROOT_PASSWORD: root
      volumes:
          - "./my.cnf:/etc/mysql/conf.d/config-file.cnf"
          - "./authdb:/var/lib/mysql:rw"
      ports:
          - "3318:3306"
      networks:
          - network-course
networks:
  network-course:
    driver: bridge

Since this is our first look at the docker-compose file, let's address the global settings:

  • note that there might be an indent between the start of a line and the start of the declaration; note that some declarations begins at a higher indent than the declarations on the lines above them. These are meant to be sub-properties of the preceding line(s)
  • note that the version, services and the networks declaration lie at the lowest line indent since they are meant to be global; properties falling at a bigger indent lie as children of the properties with higher precedence and smaller indent index
  • We will be declaring our docker images as subset of the services declaration

Besides the above global considerations, we have declared our first service as the mysql database images. Unlike their Spring Boot Applications counterparts whose images we have already produced and made accessible to the docker engine, these will need to be built from scratch as described in the steps below:

  • We have defined a name for the docker image we want to create (i.e. basicdb and authenticationdb)
  • We have indicated the base image we want to build from; i.e. mysql:5.7
  • Specify some environments variables such as:
    • The name of the database
    • The user and password for MYSQL connection
    • The MYSQL Server Root Password
    • The ports at which the application will be running in the format EXPOSED_PORT:IMAGE_PORT
      Syntax 3307:3306 means that the mysql image exposes port 3306 mapped to port 3307 of the local environment. Connection to the database image will need to use the exposed port rather than the image local port
  • Defined volumes for persisting the above MYSQL configurations as well as the data in case the docker images are restarted for instance. We have done so with the following declarations:
    • - "./my.cnf:/etc/mysql/conf.d/config-file.cnf" : used to persit the mysql server configurations. The part before the semi-collon (:) indicates the path to the local folder used to persits information that lies to the path indicated at the path after the semi-collon. The local path is relative to the folder in which lies the current file we are editing, i.e. the docker-compose file
    • - "./basicdb:/var/lib/mysql:rw" : used to indicate that the folder at path ./basicdb (subfolder named basicdb in the current folder) is mapped to the docker image’s data persistence folder, i.e. /var/lib/mysql. Besides the mapping, we provide the current docker process read and write rights to the folder through the rw argument.
  • The current docker image will be aware and use the declared paths for data persistence and server configuration

We need to be aware of the fact that we have defined the Spring Boot application database connection parameters through their application properties file. We need to make sure that the declared connection properties values map to the values we are configuring within the docker-compose YAML file. For instance, we need to configure the basicservice database connection as shown in the excerpt below (the database port has been changed so as to accomodate several mysql server images, each running on its own port):

spring:
  datasource:
    url: jdbc:mysql://localhost:3317/basicdb?useSSL=false&serverTimezone=UTC
    username: root
    password: root

The next thing we will want to do is handle the Spring Cloud Configuration Server. As a Spring Boot application, it will need a docker image and some configurations. Luckily, we have already built a docker image for each Spring Boot Application. The configuration server handles the management of the other Spring Boot Applications propertes. We have used a local lookup startegy and hence need to indicate the local paths to the properties files. Once again, we will take advantage of the volumes property to make the docker image aware of where to find the configuration file within the local deployment environment. We will, afterwards, modify the lookup paths, within the Configuration server to match the volume that we will have declared within the docker-compose file as shown below:

configurationserver:
    image: course/configurationserver:0.0.1
    volumes:
          - "./configurationFiles:/configfiles"
    ports:
       - "3000:3000"
    environment:
       PROFILE: "native"
       SERVER_PORT: "3000"
    networks:
          - network-course

Please note the following:

  • The image values needs to map to the name:tag we have given the Spring Cloud Configuration Server application docker image, i.e. course/configurationserver:0.0.1
  • We have defined the needed volume for the convenience of tidying things up; similar to the way we persisted the data for the MYSQL docker images, we define volumes that will hold the other application properties. This has the advantage that we can then refer to the configure path hereby within the configuration server application properties file to lookup the other applications configuration files. As such, we have defined the volume "./configfiles:/configfiles" and will adapt the application properties file accordingly to match the current path. After copying the applications properties folders to the indicated path (i.e. in the current folder, create a subfolder named configfiles and paste the properties folder into it). We round things up by adapting the Configuration Server Properties file to look as follows:
    server:
       port: 3000
    spring:
      profiles:
        active: native
      cloud:
         config:
           server:
               native:
                   searchLocations:
                                   file:///configfiles/authenticationservice,
                                   file:///configfiles/basicservice,
                                   file:///configfiles/discoveryservice
    Note the use of paths with the prefix file:///configfiles/ to refer to the volume the docker image can interpret.

The next part is going to handle our other 3 Spring Boot applications containers. Since these will be a similar process for all the other applications, let’s have a look at the implementation for the basic service:

  basicservice:
    image: course/basicservice:0.0.1
    depends_on:
      - basicdb
    ports:
      - "1000:1000"
    environment:
      PROFILE: "container"
      SERVER_PORT: "1000"
      CONFIGSERVER_URI: "http://configurationserver:3000"
      DISCOVERY_URL: "http://eurekadiscoveryservice:8761/eureka/"
      DISCOVERYSERVICE_PORT:        "8761"
      AUTHENTICATIONSERVICE_URI: http://localhost:2000
      AUTHENTICATIONSERVICE_CONTAINER_URI: http://authenticationservice:2000
      DATABASESERVER_PORT:      "3306"
      DATABASE_HOST: "basicdb"
      DATABASE_USER: "root"
      DATABASE_PASSWORD: "root"
      DATABASE_NAME: "basicdb"
      CONFIGSERVER_PORT: "3000"
      OAUTH2CLIENTID: basicService
      OAUTH2CLIENTSECRET: root
      OAUTH2CLIENTSCOPE: read
    networks:
          - network-course

The discovery service has similar properties except that it does not need mysql connection parameters. There are quite a few things that we have addressed besides the initial declarations we have already acknolwedged before such as the docker image and the running ports. Let’s have a look at the defined environment variables:

  • We have defined a new profile name and called it container; if you recall our properties management through Spring Cloud Configuration Server, we have created a folder to accomodate the application properties for the basic service. The folder itself contains two files, i.e. basicservice.yml and basicservice-container.yml. We are instructing the docker image to use the container profile which uses the file basicservice-container.yml
  • We have declared properties that we dynamically inject into the application properties files as envirnment variables. For instance, we have declared the MYSQL_USER here as . We can refer to the declared environment properties, within the the application properties file as $ENV_VARIABLE_NAME. The name will hence be used to map the variable name to the value it was given to within the docker-compose file.
  • We specify some environment properties such as the configuration URI and the defaultZone Uri; if you recall the build of the Spring Boot Applications docker images, we use the -D flag to overwrite some applications properties. We have defined these properties as environment variables for which values need to be set up here. Hence the command from the run.sh file becomes as follows:
    java -Dserver.port=$SERVER_PORT   \
    -Deureka.client.serviceUrl.defaultZone=$DISCOVERYSERVICE_URI             \
    -Dspring.cloud.config.uri=$CONFIGSERVER_URI                          \
    -Dspring.profiles.active=$PROFILE -jar /usr/local/basicservice/app.jar
    The environment variables referenced in the above excerpt have been declared alongside their corresponding service.

With that in mind, let’s look at the basicservice application properties for both the default and the container profile.

#basicservice.yml
server:
  servlet:
    session:
      cookie:
        name: TUTORIALBASESESSION
spring:
  datasource:
    url: jdbc:mysql://localhost:3317/basicdb?useSSL=false&serverTimezone=UTC
    username: root
    password: root
    
security:
  oauth2:
    client:
      clientId: basicService
      clientSecret: root
      scope:
        read
      accessTokenUri: http://localhost:2000/oauth/token
      userAuthorizationUri: http://localhost:2000/oauth/authorize
    resource:
      userInfoUri: http://localhost:2000/api/user
      
eureka:
  instance:
    prefer-ip-address: true
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://localhost:8761/eureka/



#basicservice-container.yml
server:
  servlet:
    session:
      cookie:
        name: TUTORIALBASESESSION
spring:
  profiles: container
  datasource:
    url: jdbc:mysql://${DATABASE_HOST}:${DATABASESERVER_PORT}/${DATABASE_NAME}?useSSL=false&serverTimezone=UTC
    username: ${DATABASE_USER}
    password: ${DATABASE_PASSWORD}
    
security:
  oauth2:
    client:
      clientId: ${OAUTH2CLIENTID}
      clientSecret: ${OAUTH2CLIENTSECRET}
      scope:
        ${OAUTH2CLIENTSCOPE}
      accessTokenUri: ${AUTHENTICATIONSERVICE_CONTAINER_URI}/oauth/token
      userAuthorizationUri: ${AUTHENTICATIONSERVICE_URI}/oauth/authorize
    resource:
      userInfoUri: ${AUTHENTICATIONSERVICE_CONTAINER_URI}/api/user
      
eureka:
  instance:
    prefer-ip-address: true
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: ${DISCOVERY_URL}

The authenticaiton has been explained quite extensively with the use of the Oauth2 protocol; we are going to declare some properties within the docker-compose file as environment variables and refer to the variables within the application properties file. This has the advantage that we can change the values in the docker-compose file when they get changed without the need of touching the application properties. Such needed properties are going to be things like the social login conneciton paramters. A particular attention needs to be directed to the Oauth2 properties of the container profile:

  • Front channel properties such as the userAuthorizationUri are to the actual browser URI of the authentication service, i.e. http://localhost:2000 
  • Back channel properties such as the accessTokenUri and the userInfoUri are set to the container address of the authentication service, i.e. http://authenticationservice:2000
  • Failing to do the above may result an invalid user redirection for authorization and a connection refused exception for the access token requests
  • Please note the use of the network-course within all the services declarations, allowing them to refer to each other (e.g. http://authenticationservice:2000 to map to an http access to the service whose name is authenticationservice on port 2000).
  • We can also use environment variables by creating a file named .env within the same folder as the docker-compose file. A sample .env file would look as shown below
    AUTHENTICATIONSERVICE_URI=http://localhost:2000
    AUTHENTICATIONSERVICE_CONTAINER_URI=http://authenticationservice:2000
    DISCOVERY_URL=http://eurekadiscoveryservice:8761/eureka/

    We could then reference the declared properties, within the docker-compose file as shown below:

    basicservice:
        image: course/basicservice:0.0.1
        ports:
          - "1000:1000"
        environment:
          DISCOVERY_URL: ${DISCOVERY_URL}
          AUTHENTICATIONSERVICE_URI: ${AUTHENTICATIONSERVICE_URI}
          AUTHENTICATIONSERVICE_CONTAINER_URI: ${AUTHENTICATIONSERVICE_URI}

We can run the set of container images through a single command:

$ docker-compose up

We may also choose to run each docker image individually with the command below:

$ docker-compose up nameOfImage

Please substitute the placeholder with the actual value for the name used to name the docker image you wish to run. Some commands come in handy; we may use the -d flag when running a docker image in detached mode. We may see the logs generated from the docker image with the command docker-compose logs -f nameOfImage