How to Make Big Data Analytics Efficient for Business Growth

Big data analytics analyzes millions of data points to derive meaningful patterns and information that businesses could use. According to one study, at least one third of all data that is generated will pass through the cloud. IDC (News - Alert) suggests that the total data generated at this point would be nearly 40 trillion gigabytes. Given this ocean of data to play with, how do businesses make use of technology to boost their growth? Here are a few ideas.

1. Take advantage of the cloud

SaaS (News - Alert) solutions have an easy edge over on-premises system installations to reap the benefits of big data analysis. SaaS offers rapid deployment, with no burden on the IT department to implement and provide ongoing maintenance for yet another system. It also can have a positive financial impact with no CAPEX costs and, in most cases, a lower total cost to the business. There is no hardware footprint and no software installation. An added bonus is the ability to adjust the number of users as needed.

Putting software as a service, platform as a service and data as a service together with massive upscale of data involved shows us the next big thing, Big Data-As- A-Service (BdaaS). It suitably describes the new and fast growing industry of cloud based big data services that help organizations offload a lot of their work to experts in this domain. BdaaS providers have everything setup and companies just have to rent the use of their cloud based storage and analytics engine and pay either for the time or the amount of data crunched. Providers often take on the cost of compliance and data protection as well.

A good example is IBM’s analytics for Twitter service that provides businesses with access to data and analytics on Twitter’s (News - Alert) 500 million tweets per day and 280 million monthly active users. The service provides analytical tools and applications for making sense of that messy, unstructured data and has trained 4,000 consultants to help businesses put plans into action. In sales and marketing businesses, BdaaS is increasingly playing its part. Many companies offer customer profiling services. By applying analytics to the massive amount of personal data they collect, they can more effectively profile consumers for potential leads.

2. Big Data visualization

More and more mid-sized businesses are taking a serious look at big data visualization. The reason for the increasing interest is simple - it makes it easier to identify insights in the huge amount of data tapped. Below are the points that should be considered on getting started with big data visualization:

• Promises related to improved product quality and better customer service are not enough to justify an investment in a data visualization solution. If you want to move to data driven decision making you need to think through exactly what the business benefits of better data analysis will be and how much those benefits will be worth. For example, e-commerce websites have a particularly interesting use case for big data visualization. It is important to make correlations between the purchases made by one user over a period of time with what other users have been buying. This can be leveraged for dynamic pricing and to show related products in a way that is based on the actual choices of other similar consumers. Shopping baskets that profile customer carts can also trigger emails about offers and discounts.

• Data visualization is an area where you just can’t go alone. Collaboration between business units and IT is one of the most important factors in data analytics projects. If you are a business manager, you have to get IT on-board and vice versa.

• A slow system simply cannot process the vast amount of structured/unstructured data, handle more users interested in using visual analytics or efficiently compute different types of workloads. You need to scale up when the need arises.

3. Modernization of legacy systems

Although legacy system data is an indispensable resource, IT teams struggle to find efficient and cost effective ways to access and leverage it for businesses. Four main challenges facing organizations are:

1. The cost and complexity of migrating to newer platforms is prohibitive.
2. Accessing data in legacy systems.
3. Data refreshes from legacy systems may be too slow for BI and analytics purposes.
4. Accessing data in legacy systems is often a burden for IT teams.

Legacy modernization or legacy transformation is the way out of these challenges. This is the act of reusing and refactoring existing core business logic by providing new user interfaces or by selectively moving data from legacy systems to modern data warehouse systems (where data is integrated and analyzed with data coming from modern sources). Understanding and mapping legacy systems to a modern data warehouse structure is a key success factor for a data integration project.

Big data analytics is not an easy thing to get right. Many companies hire specialists to build their ML pipelines. This is becoming increasingly important as the number of data sources, and thus the required capacity of such systems, rises. At that point, scaling factor also comes into play. Factors discussed above must be taken care of to implement efficient systems that scale well.

About the Author

Ian McGrath is a senior project manager at Pegasus Process Solutions. He has previously served as an independent consultant on cloud tech and big data and has advised several Fortune 1000 businesses in the past.