Software systems in a typical enterprise environment are quite complex (& grows more complex over time). It becomes a humongous task to coordinate different activities such as code & configuration changes especially when a large team is involved in development and operations. DevOps in software is seen as a cultural movement to ease out operational challenges related to integration and deployment.
Continuous Integration (CI) is an underlying technique for DevOps. According to the CI principle, if a task takes a lot of time and energy, it should be done more often, making it significantly less painful at each step. Continuous Deployment (CD) is the last leg to make software releases available to staging/production environments. Enterprises are finding it crucial to employ a CI/CD pipeline in a software project(s) to ensure quality and timely deliveries.
Google Cloud Platform (GCP) provides Cloud Build service as a toolchain to kick start the CI for a project. Some of its features are listed below:
However, Cloud Build does not support deployment/ setup creation natively and build artifacts management. It needs to be extended (or coupled with other services) to provide Continuous Deployment and hence have a complete CI/ CD pipeline. Some of the GCP services that are useful in this regard are Container Registry (for build artifact management) and Deployment Manager (Infrastructure as Code/Infrastructure deployment service)
To build a complete CI/CD pipeline for a GCP-based project, some work is required for deployment/ setup creation and artefact management. This is a cumbersome task, and DevOps engineers may find it hard to make quick progress using the available services (out of the box). In such a situation, having an enabling framework that helps build the CI/CD pipeline without significant effort is very useful.
HSC’s team has been working on GCP-based projects and has developed a CI/CD pipeline enabler framework. This framework additionally supports notification for sharing the progress of the pipeline. The developed framework helps in the realization of the CI/CD pipeline shown below:
The framework helps in executing the workflow shown below:
The key components of a typical workflow are briefly discussed below:
The project lead/manager is responsible for enabling the CI/CD pipeline using this framework, and the DevOps team simply triggers the pipeline by applying a git TAG after they check-in their changes.
HSC has been using this framework in multiple projects and has helped enable CI/CD. These projects encompass a complete AI/ML application lifecycle, i.e. ETL, model training, testing/ validation, evaluation, deployment, monitoring, and a web app to see the application results.
Implementing a CI/CD pipeline helps reduce the risks in each build and makes a software application available to prospective customers faster than ever before. Interested in deploying a CI/CD pipeline. Early bug detection to avoid any last-minute surprises, greater visibility of new builds, test cases, and issues arising due to the same, and achieving faster feedback to ensure flawless code delivery is a few added advantages of deploying the CI/CD pipeline framework.