Deployment Pipeline Facilitates Complex Workloads Migration

May 27 2020
image

From reducing infrastructure costs to minimizing security risks, organizations are increasingly aware of the benefits they can reap from modernizing their continuous integration and continuous delivery (CI/CD) pipelines and migrating them to the cloud — the AWS cloud in particular. However, conducting a migration isn’t easy — especially when it involves a diverse mix of technologies and platforms.

Containerizing workloads, using CI/CD, and automating infrastructure provisioning are among the tactics that help facilitate the migration process. But many organizations lack the in-house expertise to put them into action or don’t have the resources to devote a complex migration program. That was the case for one of ClearScale’s customers.

The Value of Expertise

The organization had moved many of its applications to AWS. However, migrating the remainder of the applications was complicated by the fact that there were so many different technologies and platforms involved. Among them were the main on-premise database and ancillary components, such as the Tableau® platform for data warehouse analysis and Pivotal® RabbitMQ for application intercommunication. Java legacy applications hosted in Windows OS and containerized applications running in environments orchestrated by Rancher 1.0 and Kubernetes were also included.

There was no automation in place to provision and manage the necessary infrastructure resources, and the company had to complete the full migration within a tight timeframe. Outside expertise was needed for workload migration, as well as automation and implementing AWS best practices such as a well-architected framework.

ClearScale had the broad knowledge base and track record of migration success required for this project. Our expertise was also validated by AWS, earning us distinction as an AWS Migration Delivery Partner.

Solution Centerpiece: CI/CD Pipeline

One of the biggest challenges presented by the migration project was the numerous technologies and platforms. At the heart of the solution was the implementation of an application lifecycle using four environments: development, QA (quality assurance), UAT (user acceptance testing) and production.

ClearScale created an integrated CI/CD deployment pipeline for use across all environments. Working with the customer, ClearScale developed a multi-component migration solution leveraging AWS best practices and services. It also made use of services from other vendors that were compatible with those of AWS to fully address the complexity of the customer’s needs.

image

The graphic above shows a typical implementation of an application lifecycle used for different environments where a Jenkins task is triggered either automatically or manually to build a new Docker image. Once done, the image is placed in Amazon ECR (Elastic Container Registry), then this image is deployed to a specific environment using a tool such as kubectl or using helm charts. The right roles and secrets for that environment are also needed.

This pipeline leverages the capabilities of tools such as AWS CodeDeploy, a platform- and language-agnostic, fully managed deployment service, and 3rd party tools such as SaltStack automation software. Among all the options available, this tool was specifically chosen since the customer was already using it and had experience with it.

image

In the graphic above we see the pipeline solution uses blue/green deployment, a DevOps practice that reduces downtime on updates. It allows for testing a new app version using a small amount of users before sending all production traffic to it. When the new version is up and running, traffic can seamlessly be switched to it. If there’s an issue with the newly deployed app version, it can be rolled back to the previous version faster than with in-place deployments.

Another tool, HashiCorp Packer, an open source project, is used to build an Amazon Machine Image (AMI) that provides the information required to launch instances. Creation of the AMI is triggered as a downstream Jenkins job, after successful pre-release tests, and can then act as an upstream job for the infrastructure build-out using Terraform, another HashiCorp open source project.

Rounding Out the Solution

The full ClearScale solution is too complex to detail here. However, in addition to the pipeline component, it included:

  • Using AWS Landing Zone to automate the setup of a secure, multi-account AWS environment and employing AWS Organizations to define the AWS account structure and segregate the production and non-production environments while supporting central policy management and consolidated billing.

  • Designing a shared services layer for centralized management of the multiple environments and accounts.

  • Architecting a reliable, elastic-compute solution to host apps using Amazon Amazon Elastic Container Service (EC2) and AWS Elastic Kubernetes Service (EKS).

  • Creating a durable, highly available database solution to store app data that leverages Microsoft SQL Server Always On Availability Groups.

  • Developing a durable, highly available shared storage solution for Windows apps using Amazon FSx.

  • Implementing a robust directory service solution using Microsoft Active Directory and AWS Active Directory.

  • Deploying a monitoring and logging solution using AWS CloudWatch and AWS ElasticSearch.

  • Migrating the legacy Java applications by updating from Windows 2012 to 2016 and using HashiCorp Packer to create golden Amazon Machine Images (AMIs) for automating the pipeline.

  • Migrating a product stack containing an Internet Information Server (IIS) application, an SQL database and an application server with the infrastructure deployment following the same automation pipeline created for the Java applications.

  • Deploying microservices using AWS Elastic Kubernetes Service (EKS) for container orchestration, with the containers deployed using Helm charts, a packaging manager that packages Kubernetes releases into a convenient zip file, through Jenkins®, an open source automation server. The container images are stored in AWS ECR (Elastic Container Registry) where the container images lifecycle is controlled by ECR policies.

  • Implementing a DR strategy for the infrastructure and apps using a pilot light approach and tools such as AWS DataSync to replicate EFS service between the two regions.

  • Developing automation tools to enable the customer to provision and manage infrastructure as code (IaC), deploy applications to AWS EC2 Auto-Scaling Groups and the AWS EKS platform, and to manage configurations on application servers.

  • Configuring automated backups for MS SQL servers and Windows application servers.

  • Developing and executing the complex migration plan and cutover.

Migration Generates Exceptional Results

The desired outcome for any migration project is its completion with minimal disruption to daily operations. That held true for this particular customer as well. Equally important, all apps were migrated to secure AWS environments within the required timeframe. Because of the specific components used in the solution, such as shared services, the company has also been able to take advantage of reduced operations overhead.

The automated provisioning for infrastructure and apps, as well as the dockerized apps running on a robust managed container platform by AWS, have freed up internal resources that can now be dedicated to product development instead of infrastructure management.

In addition, potential data loss and app failures have been mitigated and a robust DR solution delivers peace of mind.

Ready. Set. Migrate.

All companies are unique and so are their workloads. As a result, there’s no one size-fits-all solution for workload migrations. However, ClearScale can help you develop a migration solution that fits your organization’s specific needs.

To learn more contact us for a free consultation.

Get in touch today to speak with a cloud expert and discuss how we can help:

Call us at 1-800-591-0442
Send us an email at sales@clearscale.com
Fill out a Contact Form
Read our Customer Case Studies

San Francisco

Headquarters

71 Stevenson St.

Suite 400

San Francisco, CA 94105

O: 1-800-591-0442

F: 1-415-655-6601

San Jose

5450 Thornwood Dr

Suite #L

San Jose, CA 95123

Denver

1400 16th Street,

Suite 400

Denver, CO 80202

O: 1-720-932-8028

Phoenix

2942 N 24th Street,

Suite 114

Phoenix, AZ 85016

O: 1-602-560-1198

New York

165 Broadway, 23rd Floor

New York City, NY 10006

O: 1-646-759-3656

Houston

11757 Katy Freeway

Suite 1300

Houston, Texas 77079

O: 1-281-854-2088

Toronto

100 King Street West

Suite 5600

Toronto, Ontario, M5X 1C9

O: 1-416-479-5447

About Us  |  Careers  |  Privacy Policy
@ Subscribe
Share