Achieving Full Potential with Automation

Across all industries, automation is radically transforming the way business is conducted. Time-consuming tasks that once required an army of human beings can now be completed via automation technology. Just some of its many benefits include cost savings, greater operational efficiency, and the chance to move human capital to tasks better suited for human intelligence, such as problem-solving. Organizations that can find new ways of executing workflows through automation are poised to become leaner, faster, and more intelligent.

The Challenge: Managing the Workflow

One client in the Heavy Machinery design industry didn’t think automation of its processes was possible. Its projects mostly relied on a single-user piece of desktop software that required individual users to work by hand in order to carry out tasks. Scaling operations in a traditional way would take hordes of employees and many extra work hours — all without the added benefit of any additional performance enhancements such as freeing up some resources or increasing quality.

In the best-case scenario, this client would be able to run multiple instances of its software in a “headless,” fully automated cloud-based setting. Specific tasks normally done by software programs would be broken into smaller, bite-sized chunks that could be parallelized across multiple instances. This client came to ClearScale, an AWS Premier Consulting Partner, to create a custom solution to their scaling challenge.

The ClearScale Solution: Cloud Automation

ClearScale was prepared to tackle the challenge head-on. Working in close cooperation with the client’s engineering team, ClearScale studied the client’s desktop application and wrote a series of scripts that helped the software to run automatically, with absolutely zero user interaction required.

To break the scripts into multiple segments to be parallelized, ClearScale defined set standards for each chunk of data, each of which would be passed to the software as a single iteration of the entire job. A significant amount of attention was also paid to make sure the software would run reliably even while no one was monitoring it.

Infrastructure Diagram

Moreover, the job format was split across three main stages: geometry preparation, analysis, and visualization, each able to run independently in its own fleet of EC2 instances while using separate S3 buckets to house the results. A bucket with the results of one stage can then be used as source data for the next stage. Running each stage independently allowed the client to select (and pass as metadata) a type of EC2 instance for that stage. For example, analysis is done on CPU-bound instances, but visualization performs best on GPU instances.

ClearScale developed an AWS infrastructure that took advantage of Amazon Elastic Compute Cloud (EC2) to handle SQS queues, SNS topics, and Lambda functions in order to build automation and scale fleets of spot instances. Custom software was also written on Java to talk to SQS and custom software scripts on the EC2 instances.

Each automated software task was split into three unique stages where each could run independently with its own fleet of EC2 instances, and use separate S3 buckets to store resulting data. Running each stage independently allowed the client to select the ideal EC2 instance for each step in its workflow process. ClearScale also took steps to ensure EC2 could find and leverage the GPU capabilities of the client’s software.

The Benefits: Get Ready to Do More With Less

With their new, ClearScale-designed infrastructure, the client confirms that the time it takes to carry out tasks that come from their desktop software has been reduced by quite a lot. Instead of waiting for individual machines to finish processing, the client can now turn around multiple projects both faster and at a higher quality. (Remember when we talked earlier about those additional performance enhancements that are lacking in a traditional scaling process? Well, here they are.) The client is also able to control the costs of cloud resources by terminating the EC2 fleet as soon as data-intensive tasks are complete.

Thanks to ClearScale, this machinery design firm is now able to take advantage of what once seemed impossible: maintaining an automated, cloud-based workflow that leverages its existing software to do more with less. Automating routine software tasks frees up your valuable resources so you are able to spend more time doing other important human-intelligence tasks, like talking directly with your customers.

Learn more about ClearScale’s AWS DevOps services here.