Leading Marketing Automation Firm Migrates Their HBase Solution to AWS with Help of ClearScale

For companies that existed long before the birth of cloud computing, serious contenders in any technology vertical implemented a standard practice of deploying substantial infrastructure footprints to address growth and traffic spikes. This resulted in a large-scale data center presence with overprovisioned capacity. As demand for their software products continued to grow, so did the demands on their data centers; hardware needed to be upgraded, and more servers deployed to scale with their business.

At some point, however, this approach will only work if the company maintaining the data centers finds it feasible to continue the following: upgrading servers to the latest and fastest hardware profiles; adding additional memory; configuring and optimizing communications between servers and the on-site network; and incurring additional overhead in the form of higher costs of energy use for cooling larger server stacks.

In addition, to protect customers and shareholders, many companies will choose to build disaster recovery (DR) data centers in different geographic locales. These DR data centers mirror their primary operational site, utilizing identical hardware footprints. The DR data centers provide business continuity should the primary become unavailable.

It takes a very focused team to manage redundant data centers effectively. This increases the cost and complexity of operations, detracting from the company focus on building software products for customers.

In this scenario, a leading marketing automation software company approached ClearScale, an AWS Certified Premier Partner. They asked to have their SaaS platform, which relied heavily on HBase, migrated from their data centers into the AWS Cloud. The company’s expertise was in building marketing automation products for their customers. But, the ongoing maintenance and expenses related to maintaining data centers to support this growing product line were causing issues with both scalability and performance. The company found it challenging to keep scaling and optimizing its data centers. They needed a solution that addressed these challenges and would allow them to focus on building products and expanding their customer base.

The marketing software company knew that AWS offered a variety of services that would allow them to migrate their SaaS platform into the cloud, leveraging the tremendous scale and performance of the AWS public cloud infrastructure. However, they had increasing business pressure to perform this migration quickly with minimal impact on their customers. The company realized that it needed expertise in AWS services and implementation to execute its HBase migration strategy successfully.

The ClearScale HBase Migration Solution

ClearScale immediately recognized the challenges the client faced and performed an analysis of their current data center implementation. Once completed, ClearScale provided architectural guidance and recommendations in the form of a detailed architectural design. The recommendation was to use a multi-master HBase cluster topology with a cluster of three master nodes and replicated two slave clusters. The clusters, regardless of how many were instantiated, were monitored by a three-node Zookeeper cluster to determine which HBase slave nodes were active or in standby mode. The HBase masters and Zookeeper cluster were distributed across three Availability Zones (AWS data centers within a given region) to maintain quorum requirements in the event that an entire AZ becomes unavailable. The slave clusters were deployed to two separate AZs to maintain redundancy and availability of data.

ClearScale took this implementation one step further by introducing AWS CloudFormation to automate infrastructure provisioning. CloudFormation enables automated deployments by managing AWS resources (such as the AWS Virtual Private Cloud (VPC), the internet and NAT gateways, the S3 buckets for the storage of data, encryption and decryption key storage, and the AWS Identity and Access Management (IAM) configuration.

Once the templates for CloudFormation were configured, they could be managed through the AWS Management Console or APIs if the client needed to deploy new environments or update an existing environment. The approach provided for high availability of resources as well as scaling to meet capacity demands. This addressed the client’s major concerns with their existing data center scaling and performance issues.

From a monitoring standpoint, ClearScale implemented a solution using AWS CloudWatch which was designed to provide visibility into the utilization and performance of infrastructure resources and the client’s applications. These metrics, along with any customized metrics the client requested, would be stored in CloudWatch log files and then delivered to the AWS ElasticSearch service via Kinesis Firehose. AWS Lambda functions will provide data transformation and the Kibana WebUI will be used for visual representation of the log analytics. This empowers an administrative user to easily determine issues found within the log files by interacting through the intuitive Kibana WebUI interface.

Once ClearScale designed and deployed the new solution to the client’s AWS account, the environment and migration process was thoroughly tested and validated. Once all migration tasks were validated, ClearScale ran the live HBase data migration from the client’s data centers to AWS. A final round of testing was completed to ensure that the migration was successful, that the HBase implementation was stable, and that the customer data was intact.

At the same time, this activity was taking place, ClearScale was creating extensive documentation on how to monitor and manage the new HBase AWS architecture. Training was provided to the client’s key personnel. This allowed the client to quickly understand how to maintain their solution on their own without expending a large number of their own resources to research and test possible solutions on AWS. The result was a fully available, scalable, and performant turn-key solution that was transitioned to the client with minimal impact on their operations or their customers’ expectations.

The Benefits

At ClearScale, we believe that all clients are partners, not just customers. With each engagement, ClearScale takes the time necessary to fully understand your particular needs and determine the optimal path to success within the AWS family of services. We drive cloud strategies, design solutions, deploy and test the approach, and then provide appropriate documentation and training necessary for your company to continue on the mission you set out to accomplish.

Learn more about ClearScale’s AWS cloud migration services here.