CI/CD has its proven importance for many years. Once set up, it saves the team’s time and money from mistakenly pushing bad, half baked or insecure code into higher envs (QA/UAT/Prod). More importantly, it helps avoiding reputational damage due to bad builds.
Traditionally the resulting artifacts from a CI/CD pipeline was an archive file (typically a tar.gz) which was then extracted in the unix machine and then the support team had to ensure that all the dependent run-time libs are in place. In the new world, post the evolution of Docker Containers and Dev Ops, the preference is to generate a docker image as the deployable artifact which then helps avoid installation of dependencies separately.
Also, traditionally we used repos like Nexus or Artifactory for storage and retrieval of the build packages. But with docker images, we use private/public Docker Hub repository, ECR by AWS, ACR by Azure or GCR by Google Cloud etc.
Typical build pipeline to create a docker image
In the above typical build pipelines for provisioning containers its very important to understand how the triggers and the different stages of pipelines has been setup.
Step1: First we author layers of dockerfiles with at least 3 layers recommended as Base, Platform and App Images. Each files has its own purpose as described below.
- Base Image: Pull the public base OS images from verified official public repositories or from Docker Hub, apply OS hardening, patching, CIS benchmark or STID and push the newly created base image your choice of Private repo for example AWS ECR.
- Platform Image: The platform images are created from private base images for a specific application runtime (Like Java, Mode, PHP, Python) by developers. It should follow the best practises mentioned below
- Tag images by using the upstream version of the platform so that downstream images can easily get latest without having to update Dockerfile.
- Use environment variable for configuration parameters.
- Application Image: The application images are created from private platform images for a specific application by developers. This image will usually copy their source code or compiled binaries on top of platform image. Always tag these images by using the git gash, build number and/or branch. Also try to keep the modifications to minimal on the top of underlying platform image to reduce bugs.
Step2: Setup an automated scheduled deployment after every 30 days for the base container and there after any change in base trigger the platform container and subsequently app container. This will make sure that your container are always up to date with latest security patches and updates.
Step3: Vulnerabilities can be found at the operating system (OS), application dependencies, and/or application code layer. Use tools like Clair Scanner or Snyk. Run vulnerability scanning based on AWS Lambda scheduled trigger once a day for all images and trigger image rebuild process when a vulnerability is found.