How to use Docker for updating Old Production Applications?

When working on continuously expanding projects over long periods of time, software developers often come across code that must be updated in order for the project to progress. If the code is outdated, preparing the environment and launching certain applications become problematic. Docker is an open-source, virtual environment tool primarily used on Linux and OS6 applications that allows the developer to more quickly update and deploy apps inside various virtual software containers. These containers do not require separate operating systems. They access the Linux kernel’s features, meaning several containers can be run simultaneously by a single server.

Docker further speeds up deployment by providing consistent development and production environments. All members of the team can use the same system libraries, language run-times, and OS. It supports multiple language versions, including Python, Ruby, Java, and Node, meaning developers can run a script inside a Docker image, rather than installing various languages and running scripts separately. Docker’s “Copy-on-Write” mechanism duplicates any requested code in a standard UNIX pattern. Developers can also begin running a container in less than 0.1 seconds, and the entire container requires less than 1 MB of disc-space. These are significant improvements over standard VMs. To help developers take advantage of these many benefits, this article shows how to update an old production application with Docker. The app uses a non-standard Sphinx build, wkhtmltopdf, NPM, and Resque. They will initially be kept within the application and then moved to their own containers. The goals are to:

• Keep 100% forward compatibility with devs not using Docker.
• Make the app use environment variables for configuration.
• Build Sphinx, install Node.js, and wkhtmltopdf in a basic container.
• Keep the app and Resque in the same container.
• Simplify the Procfile.
• Create a base Docker-composed configuration file and extend it for development and CI envs.

Step 1: Setup

Here is what you’ll need to install Docker and its NFS plug-in for OSX Native, OSX Brew and VirtualBox, or Ubuntu.

For OSX Native:

https://docs.docker.com/docker-for-mac/

For OSX via Brew and VirtualBox:

1brew install caskroom/cask/brew-cask
2brew cask install virtualbox
3brew install docker docker-machine docker-compose
4
5# create the vm
6docker-machine create -d virtualbox default
7# import environment variables for the docker-cli
8eval "$(docker-machine env default)"

For Ubuntu:

(

install docker

) (

install docker-compose

) Now that you have Docker installed, you can run the application on it. Docker apps are configured via a Dockerfile, which defines how the container is built.

Step 2: Unify File Configuration

Assume the app you are updating has hard-coded config files under the version control. When running the app in Docker, the secrets will no longer be stored inside the image. When modifying this file to unify the configuration,

Hash#fetch

helps return default value unless it already exists in the ENV variable:

1# database.yml 
2development: &default
3  username: <%= ENV.fetch('MYSQL_USER', 'foo') %>
4  password: <%= ENV.fetch('MYSQL_PASSWORD', 'bar') %>
5  host:     <%= ENV.fetch('MYSQL_HOST', '127.0.0.1') %>
6  database: app_development
7.....
8# mongoid.yml
9development:
10  sessions:
11    default:
12      database: app_development
13      hosts:
14        - <%= ENV.fetch('MONGO_HOST', 'localhost') %>:ENV.fetch('MONGO_PORT', 27017) %>
15...
16# sphinx.yml 
17development: &default
18  address: <%= ENV.fetch('SPHINX_HOST', 'localhost') %>
19 ...

Step 3: Run Dockerfile

This app uses light-weighted Ruby: 2.1 as the base image and changes

BUNDLE_PATH

to get cache via volume (https://docs.docker.com/engine/reference/builder):

1FROM ruby:2.1-slim
2RUN apt-get update -qq && \
3  apt-get install -y \
4    build-essential git \
5    libmysqlclient-dev mysql-client libxslt-dev libxml2-dev \
6    curl net-tools --no-install-recommends

Step 4: Install NPM and Node.js via Node Version Manager

At this point, you are explicitly specifying the Node libraries. As mentioned, the application is using a non-standard Sphinx build. Therefore, you can’t use already existing Docker images for this purpose:

1ENV NVM_DIR      "/root/.nvm"
2ENV NVM_VERSION  0.33.0
3ENV NODE_VERSION 7.5.0
4
5RUN curl https://raw.githubusercontent.com/creationix/nvm/v${NVM_VERSION}/install.sh | bash \
6    && . $NVM_DIR/nvm.sh \
7    && nvm install $NODE_VERSION \
8    && nvm alias default $NODE_VERSION \
9    && nvm use default
10
11ENV NODE_PATH $NVM_DIR/v$NODE_VERSION/lib/node_modules
12ENV PATH      $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH

Step 5: Containerize the App

This step is a temporary solution to keep Sphinx within the application. When you are preparing for production release, Sphinx must be moved to a separate container:

1WORKDIR /tmp
2
3RUN curl -o sphinx.tar.gz https://sphinxsearch.com/files/sphinx-2.2.11-release.tar.gz \
4    && tar -zxvf sphinx.tar.gz > /dev/null \
5
6    && curl -o libstemmer_c.tgz https://snowball.tartarus.org/dist/libstemmer_c.tgz \
7    && tar -xvzf libstemmer_c.tgz > /dev/null \
8
9    && cd sphinx-*/ \
10    && cp -R ../libstemmer_c/* ./libstemmer_c \
11    && ./configure --with-mysql --with-libstemmer > /dev/null \
12    && make > /dev/null && make install > /dev/null

During this step, you also need to clean up extraneous data to reduce the image size:

1# Cleanup
2RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Step 6: Mount Bundled Gems Folder Inside Container

Docker supports mount points called “volumes” which let you access data from either the native host or another container. In this case, you can mount your bundled gems folder inside the container:

1ENV APP /app
2ENV BUNDLE_PATH /bundle
3VOLUME /bundle
4
5# Gems
6RUN gem install bundler
7
8ADD Gemfile .
9ADD Gemfile.lock .
10ADD vendor .
11RUN bundle install --jobs 4 --retry 5

(To learn more about

BUNDLE_PATH

and why gems must be installed as the last step, see:

https://medium.com/@fbzga/how-to-cache-bundle-install-with-docker-7bed453a5800

)

Step 7: Install Node Packages

When using Docker to perform this step, the NPM is invoked with root privileges, so it will change the UID to the user account or to the UID specified by the user config, which, in this case, defaults to nobody. Start by setting the unsafe-perm flag to run scripts with root privileges:

1ADD package.json .
2RUN npm install --unsafe-perm

(For more details on running scripts with root privileges, see:

https://docs.npmjs.com/misc/scripts#user

) During the last part of this step, you add the app’s code, expose the internal 3000 port, and process any incoming commands thought

bin/docker-entrypoint.sh

:

1# App
2RUN mkdir $APP
3WORKDIR $APP
4ADD . $APP
5
6EXPOSE 3000
7
8ENTRYPOINT bin/docker-entrypoint.sh $0 $@

Step 8: Create docker-compose.yml File

Before executing any code in the app’s container, you need to make sure that the old PID files have been deleted and the database is available:

1#!/bin/bash
2bundle check || bundle install --jobs 4 --retry 5
3
4pids=( server.pid realtime_updater.pid )
5for file in "${pids[@]}"
6do
7  path="tmp/pids/$file"
8  if [ -f $path ]; then
9    rm $path
10  fi
11done
12
13./bin/wait-for-mysql.sh
14
15echo "[INFO] Running in app: $@"
16exec "$@"

Now you’ll use

wait-for-mysql.sh

to hold the execution until the DB is ready:

1#!/bin/bash
2echo "[INFO] Waiting for mysql"
3
4until mysql -h"$MYSQL_HOST" -P3306 -u"$MYSQL_ROOT_USER" -p"$MYSQL_ROOT_PASSWORD" -e 'show databases;'; do
5  >&2 printf "."
6  sleep 1
7done
8
9echo "[INFO] Mysql ready"

Next, you need to integrate the app with Mysql, Mongo, Redis, volumes and expose the env variables. As an additional step, you can create

docker-compose-base.yml

which will be extended in the compose file for dev and ci/prod. Many of the images on Dockerhub accept env variables for configuration. In this case, you want to config mysql’s root user and expose the data in the application container, as well as in the Redis and Mongo URLs:

1# .docker/.env
2REDIS_HOST=redis
3MONGO_HOST=mongo
4
5# .docker/db.env
6MYSQL_ROOT_USER=root
7MYSQL_ROOT_PASSWORD=password
8
9MYSQL_USER=root
10MYSQL_PASSWORD=password
11
12MYSQL_HOST=db
13MYSQL_DATABASE=app_development

The

docker-compose.yml

includes the application service, along with the images for Mysql, Redis and Mongo:

1# docker-compose.yml
2version: '2'
3services:
4  app:
5    build: .
6    command: ./bin/startup.sh
7    volumes:
8      - .:/app
9      - bundle:/bundle
10    ports:
11      - '3000:3000'
12      - '4000:4000'
13    env_file:
14      - ./.docker/db.env
15      - ./.docker/.env
16    links:
17      - db
18      - redis
19      - mongo
20
21  db:
22    image: mysql:5.7
23    ports:
24      - 3306
25    env_file:
26      - ./.docker/db.env
27    volumes:
28      - mysql:/var/lib/mysql
29
30  redis:
31    image: redis:latest
32    volumes:
33      - redis:/data
34
35  mongo:
36    image: mongo
37    volumes:
38      - mongo:/data/db
39
40volumes:
41  bundle:
42  mysql:
43  redis:
44  mongo:
45
46
47# Procfile.docker
48web: bundle exec rails s -b 0.0.0.0 -p 3000 -e ${RACK_ENV:-development}
49react: npm start
50resque: QUEUE=... bundle exec rake resque:work
51sphinx: bundle exec rake ts:start NODETACH=true
52....

1#startup.sh
2#!/bin/bash
3
4# tells the bash script to exit whenever anything returns a non-zero return value.
5set -e
6
7./bin/wait-for-mysql.sh
8
9# echo "[INFO] Precreate databases..."
10# rake db:version || bundle exec rake db:setup
11
12# echo "[INFO] Running db:migrate..."
13# rake db:migrate
14
15bundle exec foreman start -f Procfile.docker

At this point, you have completed all the steps needed to create your new Docker container and can move on to starting and running the application.

Step 9: Start and Run the Application

Your entry point:

1docker-compose up

Point your browser to: https://localhost:3000 or https://192.168.99.100:3000. The Docker default machine’s IP is 192.168.99.100. To make sure your IP is correct, run:

1docker-machine ip docker

If you want to bash to the app container or run custom command, don’t forget to enable and map services to the host by passing the “service-ports” option to

RUN command

:

1docker-compose run --service-ports app bash

Here’s what you need to run the specs:

1docker-compose run app bundle exec rspec

Step 10: Prepare to Release the App to Production

1. Move applications logs to STDOUT/STDERR. During the updating process, logs will be saved to the container’s default file system.
2. Integrate this STDOUT/STDERR log file into a centralized logging system. This is usually a 3rd party service.
3. Move Sphinx, Node.js, Resque, wkhtmltopdf and all schedule-based tasks and processes to their own containers.
4. Create a compose file for CI and prod.

Containerizing outdated apps with Docker is becoming increasingly popular due to its many benefits. It speeds up the entire development process while improving portability, transparency, and security.