Deployment Pipeline Facilitates Complex Workloads Migration

From reducing infrastructure costs to minimizing security risks, organizations are increasingly aware of the benefits they can reap from modernizing their continuous integration and continuous delivery (CI/CD) pipelines and migrating them to the cloud — the AWS cloud in particular. However, conducting a migration isn’t easy — especially when it involves a diverse mix of technologies and platforms.

Containerizing workloads, using CI/CD, and automating infrastructure provisioning are among the tactics that help facilitate the migration process. But many organizations lack the in-house expertise to put them into action or don’t have the resources to devote a complex migration program. That was the case for one of ClearScale’s customers.

The Value of Expertise

The organization had moved many of its applications to AWS. However, migrating the remainder of the applications was complicated by the fact that there were so many different technologies and platforms involved. Among them were the main on-premise database and ancillary components, such as the Tableau® platform for data warehouse analysis and Pivotal® RabbitMQ for application intercommunication. Java legacy applications hosted in Windows OS and containerized applications running in environments orchestrated by Rancher 1.0 and Kubernetes were also included.

There was no automation in place to provision and manage the necessary infrastructure resources, and the company had to complete the full migration within a tight timeframe. Outside expertise was needed for workload migration, as well as automation and implementing AWS best practices such as a well-architected framework.

ClearScale had the broad knowledge base and track record of migration success required for this project. Our expertise was also validated by AWS, earning us distinction as an AWS Migration Delivery Partner.

Solution Centerpiece: CI/CD Deployment Pipeline

One of the biggest challenges presented by the migration project was the numerous technologies and platforms. At the heart of the solution was the implementation of an application lifecycle using four environments: development, QA (quality assurance), UAT (user acceptance testing), and production.

ClearScale created an integrated CI/CD deployment pipeline for use across all environments. Working with the customer, ClearScale developed a multi-component migration solution leveraging AWS best practices and services. It also made use of services from other vendors that were compatible with those of AWS to fully address the complexity of the customer’s needs.

The graphic above shows a typical implementation of an application lifecycle used for different environments where a Jenkins task is triggered either automatically or manually to build a new Docker image. Once done, the image is placed in Amazon ECR (Elastic Container Registry), then this image is deployed to a specific environment using a tool such as kubectl or using helm charts. The right roles and secrets for that environment are also needed.

This deployment cloud pipeline leverages the capabilities of tools such as AWS CodeDeploy, a platform- and language-agnostic, fully managed deployment service, and 3rd party tools such as SaltStack automation software. Among all the options available, this tool was specifically chosen since the customer was already using it and had experience with it.

In the graphic above we see the deployment pipeline solution uses blue/green deployment, a DevOps practice that reduces downtime on updates. It allows for testing a new app version using a small number of users before sending all production traffic to it. When the new version is up and running, traffic can seamlessly be switched to it. If there’s an issue with the newly deployed app version, it can be rolled back to the previous version faster than with in-place deployments.

Another tool, HashiCorp Packer, an open-source project, is used to build an Amazon Machine Image (AMI) that provides the information required to launch instances. The creation of the AMI is triggered as a downstream Jenkins job, after successful pre-release tests, and can then act as an upstream job for the infrastructure build-out using Terraform, another HashiCorp open source project.

Rounding Out the Solution

The full ClearScale solution is too complex to detail here. However, in addition to the pipeline component, it included:

• Using AWS Landing Zone to automate the setup of a secure, multi-account AWS environment and employing AWS Organizations to define the AWS account structure and segregate the production and non-production environments while supporting central policy management and consolidated billing.
• Designing a shared services layer for centralized management of multiple environments and accounts.
• Architecting a reliable, elastic-compute solution to host apps using Amazon Amazon Elastic Container Service (EC2) and AWS Elastic Kubernetes Service (EKS).
• Creating a durable, highly available database solution to store app data that leverages Microsoft SQL Server Always On Availability Groups.
• Developing a durable, highly available shared storage solution for Windows apps using Amazon FSx.
• Implementing a robust directory service solution using Microsoft Active Directory and AWS Active Directory.
• Deploying a monitoring and logging solution using AWS CloudWatch and AWS ElasticSearch.
• Migrating the legacy Java applications by updating from Windows 2012 to 2016 and using HashiCorp Packer to create golden Amazon Machine Images (AMIs) for automating the cloud pipeline.
• Migrating a product stack containing an Internet Information Server (IIS) application, an SQL database, and an application server with the infrastructure deployment following the same automation pipeline created for the Java applications.
• Deploying microservices using AWS Elastic Kubernetes Service (EKS) for container orchestration, with the containers deployed using Helm charts, a packaging manager that packages Kubernetes releases into a convenient zip file, through Jenkins®, an open-source automation server. The container images are stored in AWS ECR (Elastic Container Registry) where the container images lifecycle is controlled by ECR policies.
• Implementing a DR strategy for the infrastructure and apps using a pilot light approach and tools such as AWS DataSync to replicate EFS service between the two regions.
• Developing automation tools to enable the customer to provision and manage infrastructure as code (IaC), deploy applications to AWS EC2 Auto-Scaling Groups and the AWS EKS platform, and manage configurations on application servers.
• Configuring automated backups for MS SQL servers and Windows application servers.
• Developing and executing the complex migration plan and cutover.

Migration Generates Exceptional Results

The desired outcome for any migration project is its completion with minimal disruption to daily operations. That held true for this particular customer as well. Equally important, all apps were migrated to secure AWS environments within the required timeframe. Because of the specific components used in the solution, such as shared services, the company has also been able to take advantage of reduced operations overhead.

The automated provisioning for infrastructure and apps, as well as the dockerized apps running on a robust managed container platform by AWS, have freed up internal resources that can now be dedicated to product development instead of infrastructure management.

In addition, potential data loss and app failures have been mitigated and a robust DR solution delivers peace of mind.

Ready. Set. Migrate.

All companies are unique and so are their workloads. As a result, there’s no one-size-fits-all solution for workload migrations. However, ClearScale can help you develop a cloud migration solution that fits your organization’s specific needs.

To learn more contact us for a free consultation.