Big Data in Application and Cloud Performance

Big Data in Application and Cloud Performance- Why & How

Vikas Aggarwal

CEO, Zyrion Inc.

Always regarded as a non-critical part of day-to-day operations in the past, Big Data and its delayed analysis was relegated to batch processing tools and monthly meetings. Today, as the IT industry has snowballed into a fast moving avalanche of Cloud, virtualization, out-sourcing and distributed computing, the science of extracting meaningful intelligent metrics from Big Data has become an important and real-time component of IT Operations.

WHY BIG DATA IN CLOUD PERFORMANCE TOOLS

No longer do IT management systems work in vertical or horizontal isolation as just a few years ago. The inter-dependence between IT Services, applications, servers, cloud services and network infrastructure has a direct and measurable impact on Business Services. The amount of data generated by these components is huge and the rate at which this data is generated is so fast that traditional tools cannot keep up with any kind of real time correlation. The combined volume of data generated by this hybrid infrastructure can be huge, but if it is correlated properly, it can give mission- critical insight into:

  • the response times and behavior of an IT service or application
  • the cause of performance degradation of an IT service
  • trend analysis and proactive capacity planning
  • see if SLAs are being met for business services

This data has to be analyzed and processed in real-time in order to provide proactive responses and alerting for service degradation. The data that is being collected can be structured or unstructured, coming from a variety of systems which depend on each other to offer optimal performance, and has little to no obvious linkage or keys to one another (i.e. the data coming from an application is completely independent of the data coming from the network that it is running on). Some examples of data sources that need to be correlated are application logs, netflow, JMX, XML, SNMP, WMI, security logs, packet analysis, business service response times, weather, news, etc.

Enterprises are moving to hybrid cloud environments at an alarming rate and all customer surveys indicate that the complexity of these platforms are their biggest concern. Enterprises must adopt monitoring systems that are flexible and can handle Big Data efficiently so that they can offer real-time responses to alarms and get meaningful business impact analysis from all of the different data sources.

Contextual analytics and presentation of data from multiple sources is invaluable to IT Operations in troubleshooting poor application performance and user satisfaction. As a simple example, a user response time application could send an alert that the response time of an application is too high. Application Performance Monitoring (APM) data could indicate that a database is responding slowly to queries because the buffers are starved and the number of transactions is abnormally high. Integrating with network netflow or packet data would allow immediate drill down to isolate which client IP address is the source of the high number of queries.

HOW TO HANDLE BIG DATA FOR CLOUD PERFORMANCE

Traditional monitoring or BI platforms are not designed to handle the volume and variety of data from this hybrid IT infrastructure. The management platforms need to be designed to correlate Big Data from the IT components in real-time and provide feedback to the operations team for proactive responses. As these monitoring systems evolve, their Big Data correlation components will become richer and more analytical and will position these enterprises for the IT environments of the future.

New generation enterprise monitoring solutions that are scalable, have predictive analytics, multi-tenant and a granular security model are now available from a small number of vendors. Single use systems that are designed for just network data or just application data are trapped within the same boundaries that makes Big Data meaningless by its very nature, Big Data systems needs to be able to handle a very wide variety of data sources to provide greater uptime from faster troubleshooting and lower OpEx from correlated analysis.


Vikas Aggarwal is CEO of Zyrion Inc.,
a leading provider of Cloud and Network Monitoring software for large enterprises and Managed Service Providers.
You can read more about Zyrion’s Enterprise Monitoring Solution and how it handles Big Data on their web site.

 

 

How to Avoid the Windows 10 October 2018 Update Problems with Kaseya

The Microsoft’s Windows 10 October 2018 update, code named Redstone 5, is the most bug-prone update in the company’s history.Read More

Still Not GDPR Compliant? First Fines Expected By Year-End

General Data Protection Regulation (GDPR) came into effect on May 25, 2018, ushering in the most stringent data protection rulesRead More

What IT Service Desk Metrics Should You Track?

The IT Service Desk plays a key role in managing an enterprise’s IT systems and infrastructure. It is the primaryRead More

Healthcare IT Addresses Security, But Struggles with Backup Strategies

According to the Kaseya 2018 State of IT Operations survey report, improving IT security, reducing IT costs and delivering higherRead More

Archives

Categories