Using Docker Compose with Nx Monorepo For Multi Apps Development
Introduction
The concept and toolings of Monorepo are gaining a fare share of momentum in recent years. Today, I am going to take some time to explain how we can streamline multi-apps development works by leveraging the power of Monorepo and Container Orchestration tool.
You can find the working sample in this GitHub repo. The tools that I am using are Nx (Monorepo) and Docker Compose (Container Orchestration).
If you run into any issues or questions about this topic, please feel free to reach out to me. I will be glad to connect with you on this. Trust me, I know the pain of setting this whole thing up. It took me well over a weekend due to my initial lack of Docker knowledge. Haha..
Topics Coverage
Please feel free to skip to the sections of your interest.
- Why use Docker-Compose with Nx during development time?
- Docker-compose Configurations for Multi Apps in Nx
- How to make Hot Reload works for Nx in Docker?
- Outcomes
Why use Docker-Compose with Nx during development time?
The reason might differ from one engineer to another.
For myself and the works that I am involved in day to day, there are a lot of context switching required. I am mostly switching and working on 4 different applications' development at the same time. Now, imagine the following situation...
You are currently developing app-1, then suddenly there is an urgent request that would need you to hop on to fix something in app-2. What do you normally do? Exit the multiple bashs/shells running the apis, frontend for app-1 and then spin up another bunch for app-2... Right? Then, suddenly you realize some ports are clashing, and damn you forgot to turn off one of the api... oh.. and what are the cli commands to run app-2 again? Haha..
The list of issue goes on. Hence, the gigantic context switching and a direct jeopardize to productivity. This problem is mainly due to not having the capability to gain a single cohesive view of all "components" required to run an application.
Well, with some help from docker-compose, the game truly changes for me 😎 Because... Now, I can view everything through Docker Desktop's dashboard. It's a one stop station. Spinning applications up and down is just a matter of a few clicks. Clean!
Of course, docker-compose is only of the many solutions out there for this problem. There are some great devs whom I know of, favor solutions like ConEmu, etc. as well. And that's fine too. Also we can't deny the fact that, anything runs in a virtual environment will be slower due to additional layering.
Docker-compose Configurations for Multi Apps in Nx
The following example is based upon this GitHub repo here. It might be a good idea for you to clone/fork the repo to follow and study along. At this point in time, I have created a total of 4 apps in the Nx workspace.
- 2 React apps (app-1 and app-2)
- 1 Angular app (app-3)
- 1 Express api (api-1)
In Nx, all "servable" applications are called apps regardless of frontend or api. Hence, the 4.
Alright, now the objective here is to spin up all apps in one go with just a simple "docker-compose up" command. The following are the 4 files that are crucial for this Docker setup.
1. .dockerignore
This is much like the .gitignore file, except it is used by Docker to ignore files/folders in the "context" of a Docker build. And "context" in docker-compose is the root path for the build process. Bear in mind, it can be a different directory than the target Dockerfile. This was one confusing aspect when I first started working with Docker.
Anyway, here is my .dockerignore file
1node_modules
2Not a mistake, here I am just excluding node_modules from my build context. Reason is to avoid Docker to COPY my entire node_modules from code base into the image build, which can be terribly slow.
2. Dockerfile
1FROM node:17-alpine3.12
2
3WORKDIR /app
4
5COPY . .
6
7RUN npm install
8
9CMD ["npm", "run", "start", "app-1"]
10Dockerfile is a build step definition file to tell Docker how to build an image, step by step.
I am simply copying everything from my code base into the build context except the node_modules which is exempted via the .dockerignore file already. After that, we have to run "npm install" to reinstall the dependencies for the image.
Since we are using a monorepo, the source files to build every app will be the same, i.e the monorepo itself. Only difference being the entrypoints to individual applications. Hence, we only need one Dockerfile to build all the "services" defined in our docker-compose file later.
3. docker-compose.base.yml
1version: '3'
2
3services:
4 nx-app-base:
5 restart: always
6 build:
7 context: .
8 dockerfile: Dockerfile
9 environment:
10 - DEV_PLATFORM=DOCKER
11 volumes:
12 - ./:/app
13 - mono-node-modules:/app/node_modules
14I created a base docker-compose.base.yml file to consolidate the common configurations into one file, so we can have a cleaner docker-compose.yml later.
Here, notice that the build section is telling Docker that the build context is the current directory and use the Dockerfile at this directory. If your Dockerfile is in a different location, you can specify the relative path to it here, e.g. ./.docker/Dockerfile.
The environment section is for us to specify environment variables. For our case, we are using Node image. Hence, the values defined here will go into process.env.[the_environment_variable_name], i.e. the Node environment variables. Here, we defined an environment variable called "DEV_PLATFORM". This value will be used to override a Webpack config file in following next section for the hot reload feature to work.
Lastly, the volumes section defined 2 very important things.
-
./:/app
- Maps our source codes to the container's /app directory (which is defined as the working directory in our Dockerfile)
- This will enable our host machine and the containers to share the same source files, enabling file monitoring and hot reloads later on
-
mono-node-modules:/app/node_modules
- mono-node-modules will be defined in our docker-compose.yml later as the shared volume. This is for all containers to share the same node_modules volume
- In a way, this will speed up "npm install" after the first service, because we now only need to install the node dependencies once
- For our case, it makes total sense because in monorepo, dependencies are shared
4. docker-compose.yml
1version: '3'
2
3services:
4 # React App
5 app-1:
6 extends:
7 file: docker-compose.base.yml
8 service: nx-app-base
9 command: npm run app-1:dev:docker
10 ports:
11 - 4201:4200
12
13 # React App
14 app-2:
15 extends:
16 file: docker-compose.base.yml
17 service: nx-app-base
18 command: npm run app-2:dev:docker
19 ports:
20 - 4202:4200
21
22 # Angular App
23 app-3:
24 extends:
25 file: docker-compose.base.yml
26 service: nx-app-base
27 command: npm run app-3:dev:docker
28 ports:
29 - 4203:4200
30
31 # Express App
32 api-1:
33 extends:
34 file: docker-compose.base.yml
35 service: nx-app-base
36 command: npm run api-1:dev:docker
37 ports:
38 - 4310:3333 # API Entry port
39 - 4311:4311 # Server Debugging port
40
41volumes:
42 mono-node-modules: null
43In this docker-compose.yml file, there are 4 "services" defined corresponding to the 4 Nx apps in our workspace. As you can see, each of these services has an "extends" property that extends the base service defined in our docker-compose.base.yml file in earlier section.
Also note that mono-node-modules is defined as the shared volume in the last section, which is then consumed by individual services as they extend the base service definition.
How to make Hot Reload works for Nx in Docker?
This actually depends on the OS you're running on. In my case, my containers are running as Linux containers. On the other hand, I am developing on a Windows machine.
That is a problem for node-based apps that rely on Webpack's HMR. By default, if no extra configuration is included, the HMR detects changes via the OS's filesystem. For my case, Linux and Windows run on different filesystems. Hence, the hot reload issue.
In order to make the hot reload works for Nx in Docker, be it React, Angular, Express, any other node-based apps, we then need to override its webpack configs to explicitly tells Webpack that we want to use polling mechanism instead of listening to the filesystem.
For React and Express apps
To override the webpack config for React and Express apps in Nx, we need to first update the app's project.json or in older version of Nx, the root workspace.json file or the architect/schematic file to instruct the builder that we want to use a custom Webpack config file.
This is illustrated here in the repo
1{
2 "root": "apps/app-1",
3 "sourceRoot": "apps/app-1/src",
4 "projectType": "application",
5 "targets": {
6 "build": {
7 ...
8 "webpackConfig": ".webpack/react-dev.config.js"
9 },
10....
11Next step would then be the custom Webpack config file itself. The full sample can be found here.
1const nrwlConfig = require("@nrwl/react/plugins/webpack.js");
2
3module.exports = (config, context) => {
4 nrwlConfig(config); // first call it so that it @nrwl/react plugin adds its configs
5
6 if (process.env.DEV_PLATFORM === 'DOCKER') {
7 // Make Hot Module Reload (HMR) works
8 // Use polling mechanism to handle Filesystem disparities among diff OS
9 config.watchOptions = {
10 aggregateTimeout: 500,
11 poll: 1000,
12 }
13
14 // Handle WebSocket port binding when Docker Host:Container port is different
15 config.devServer = {
16 ...config.devServer,
17 client: {
18 webSocketURL: "auto://0.0.0.0:0/ws",
19 },
20 };
21 }
22
23 return config;
24};
25Note that we're using the environment variable "DEV_PLATFORM" here defined in our docker-compose.base.yml earlier on. It will then include the polling options only when the value is equal to "DOCKER".
For Angular app
For Angular app, there is no straight forward mechanism to override its Webpack config. Though it is possible with the help of another tool like @angular-builders/custom-webpack, I certainly wouldn't prefer to do so. That's mostly because that is yet another blackbox itself. A blackbox on top of the already obscure blackbox? Hmm... Not a fan seriously.
However, we can still turn on the Webpack's polling option in Angular with its cli parameter fortunately. So, we can do it with just
1nx serve app-3 --host=0.0.0.0 --poll 5000
2I tried this on both Nx v12 and v13. While Nx v13 (with Angular 13) is still working fine from the looks of it, it isn't entirely the case with v12. Somehow, changes were detected constantly and rebuilds kept going on and on, causing infinite reloads... Didn't bother to figure that out anyway. Escaping hard from Angular! 😆
Outcomes
Okay! So, finally here is how my Docker Desktop dashboard looks like after all the hard works. All 4 apps are up and running fine. I can turn them on and off via the dashboard itself. Cool!
Also, we can take a look at the Inspect tab of these 4 containers. Their node_modules are all pointing towards the mono-node-modules volume that we have defined earlier on! Niceeeee!
The apps are then accessible at the exposed ports, i.e. 4201, 4202, 4203. Here is how my app-1 looks! And yea, when I change my source files, it gets updated and reloads with no prob! Truly hottttt! 😆
Conclusion
Been playing extensively with Docker and docker-compose close to a month now. And man, I've never looked back since! It's such a helpful tool. Getting to do a "context switch" much faster, means that I can be a lot more productive in my day to day job!
Though there's a serious uphill learning curve there about how Docker works in general and all those unfamiliar configs and commands involved, once you get it, you get it! Just don't give up! It's beneficial to both your productivity in a long run and of course, your value as a tech folk in the market today! 😆
Also worth mentioning, it was especially tiring after getting it works with sample projects and then... it hits a whole other bunch of probs when it comes to corporate proxies, private Npm, Nuget source, etc 🙁 Definitely one of the worst things to solve in corporate IT environments haha... If you too run into a similar situation as such and felt helpless, drop me a message, I might just have the answer for you.
Anyway, some of those topics/toolings like Monorepo, Microservice, Microfrontend, Containerization, Container Orchestration, Docker, Docker-Compose are somewhat cohesive, and often complement each other in ways that help improve software development flows. Hack, they even unlock tonnes of possibilities in software architectures. It's good to study and research a bit along those lines. Such a great time to be in the tech field now more than ever seriously.
Here is the January 2022 subscription from Apartment Coffee. Really love this batch of Ethiopian beans. Floral and juicy, totally my kind of beans! Love it!
