ClearScale Blog

Talk to a cloud specialist

Data Ingestion Pipeline for Big Data Aggregation and Analysis

11 Mar
By: ClearScale
image

The Challenge A financial analytics company's data analysis application had proved highly successful, but that success was also a problem. With a growing number of isolated data centers generating constant data streams, it was increasingly difficult to efficiently gather, store, and analyze all that data. The company knew a cloud-based Big Data analytics infrastructure would help, specifically a data ingestion pipeline that could aggregate data streams from individual data centers into a central cloud-based data storage. One of the challenges

Read more

Migrating HDP Cluster to Amazon EMR to Save Costs and Ease the Upgrade Process

04 Mar
By: ClearScale
image

Big data has transformed the way companies conduct business. Through the careful analysis of large amounts of information, organizations can be empowered with modern decision-making capabilities that make it easier to draw conclusions and carry out actions based on accurate, relevant, and up-to-date data. Companies that embrace Big Data gain access to tools that can significantly reduce the cost of doing business. Using Big Data to help an organization lower costs requires a detailed understanding of how, when, and where

Read more

Leveraging Amazon Kinesis Streams for Low Latency Data Ingestion

08 Jan
By: ClearScale
image

For many organizations, having so much data at their fingertips can be both a blessing and a curse. While lots of data provides teams with valuable insights that can guide effective decision making, such an avalanche of information can also be a struggle to sift through and make sense of. Data is also ever-changing — and it’s up to teams to keep up so that they’re always gleaning the most relevant insights. The teams that stand apart are the

Read more

Discovering the Power of Amazon QuickSight for Business Intelligence Needs

01 Nov
By: ClearScale
image

In order for any organization to fully understand their business operations, customer base, or established processes, there is a need to be able to evaluate key data points to determine current state and use the information to make informed decisions on how the business needs to operate. Many solutions exist today that organizations leverage to obtain these types of business intelligence (BI) analytics. Not all BI solutions are necessarily equal, however. Some solutions are complex and monolithic in nature and

Read more

Using AWS Batch to Analyze and Extract Information from Large Document Data Stores

12 Sep
By: ClearScale
image

The growth of data stores has grown significantly over the last decade, especially with the introduction of IoT managed devices, such as medical devices. From a business perspective, attempting to run reporting and analysis against an ever-growing data store presents issues with some technologies due to the processing time it takes to do transformation routines on increasing volumes of information. When dealing with large volumes of data, finding ways to easily apply transformations and enrichment to data objects can be

Read more

Using Amazon ElasticSearch to Improve Performance when Querying Data in MySQL

18 Jun
By: ClearScale
image

Search tools have increasingly grown power and functionality over time, and both users and companies have become more reliant on them to identify information and patterns quickly and efficiently. However, even the most robust search tool can experience issues when compiling information from large sets of data. This can be due to either complex relational database joins or the sheer volume of data a single query must parse through to identify the necessary information. Every industry experiences this at some

Read more

Leveraging Rsync and Rundeck to Replicate AWS Elastic File System

04 Jun
By: ClearScale
image

Since its inception, Amazon Web Services has created numerous Cloud-based services to give their clients the ability to create phenomenal architectures and applications with minimal investment and high scalability and redundancy. However, some of those services provide the backbone of basic functionality with just enough to make it useful but not necessarily enough to meet each and every need of clients that rely on them. In times like this, it falls on customers of AWS to find alternative and creative

Read more

Leveraging the Power of Tableau and Redshift on AWS Cloud for Better Analytics

21 May
By: ClearScale
image

The medical care industry has sometimes lagged behind other industries when it comes to adopting new technologies, like Big Data Analytics to make care more efficient and cost-effective. Due to the high demand for qualified specialists, the industry for years has relied on certain methodologies to help identify candidates for these types of roles which at times has been costly and not without issues. A client of ClearScale’s has attempted to solve this vexing issue by leveraging technology to

Read more

Dynamic Orchestration Workflow Using Apache Airflow

07 May
By: ClearScale
image

Data is the undisputed ruler that drives everything from business strategy and decisions to near real-time algorithmic workflows. In a data-rich world, capturing that information and processing it is a monumental task, even when a clearly thought out architectural approach is implemented. When new data sources are found that require data extraction, transformation, and loading (ETL) into a data store, often that activity is done with command-line requests, scheduled cron jobs, or scripts that call scripts to achieve this activity

Read more

Collecting and Enriching Data by Leveraging Snowplow for Deep Analytics

29 Mar
By: ClearScale
image

The Challenge Every second of every day, billions of data points are created in our online world. Making sense of all of the data and trying to find a singular piece of information that is useful, let alone actionable, is a daunting task. In the fast-moving world of news and media, being able to identify pertinent information out of all of the noise is not just important but a necessity. A large company in the global news space discovered this

Read more
Share