Machine Logs Analytics – Next Frontier for Data Center Infrastructure Management (DCIM)

Dinesh Katiyar

There are few detractors when it comes to the value of DCIM for an Infrastrucutre-as-a-Service (IaaS) provider. But as the data centers become more dynamic and heterogeneous, these tools will need to adept. As CDW, a leading DCIM vendor PREDICTS that cross vendor visibility and heterogeneous platform will challenge the effectiveness of these tools. Next generations of DCIM and Monitoring tools will require operational data collection from disparate platforms and applications. Machine logs analytics can help overcome this challenge.

Machine logs provide factual information. These logs are becoming increasingly valuable due to increased instrumentation of devices resulting into enriched logs data and customer demands from device manufactures to provide better predictive and proactive services. These trends will only gather steam in this era of connected devices. So what is the right solution to analyze machine logs? At first glance, current log analytics tools may suffice. But majority of current logs analytics tools were built to analyze time series event logs data (e.g. Syslogs) for narrow use cases of operational analytics. However, typical machine logs contain more complex semantics such as configuration info, CLI commands, statistics and other non-time stamped data. To make sense of the data in these logs, one needs a powerful language and processing engine to provide meaning and structure to the information..

Glassbeam’s big data machine logs analytics platform was built from ground-up to analyze complex formats of MULTI-STRUCTURED MACHINE DATA from any connected “thing”, device or network, now broadly called Internet of Things (IoT). Glassbeam launched this platform in 2009 when IoT term was still in its infancy. It leverages patent-pending Semiotic Parsing Language (SPL) that can quickly extract information from complex semantics embedded in multi-structure machine data. SPL powered technology, like Glassbeam, enables customers to quickly gain insights into machine data in days instead of months reducing time-to-value. Pay-as-you-go pricing model is designed to greatly enhance CUSTOMER ROI and offer these benefits at a fractional price of current log analytics tools.

The Glassbeam platform is proven. We work with large global IaaS providers including one with 25 global data centers, 20,000+ VM machines, 10 PB of primary storage, and 4000 customers. These IaaS vendors use Glassbeam to analyze data from storage devices (EMC, HDS, IBM, FusioIO, Xiotech etc.), networking gear (Brocade, CISCO, Juniper etc.), applications (DMBS, Linux, Apache, VMware etc.), and backup solution such as Commvault. In future, this IT data can be combined with data sources from other facility management infrastructure like power, cooling, racks, etc to make a more holistic capacity planning use case for the entire data center(s).

Current capabilities of Glassbeam solution provide the following:

  • Capacity planning
  • Proactive Notification on anomalies by setting rules on specific thresholds (upper and lower limits), rate of change (e.g. >50% change on cpu utilization)
  • Customer profiling (e.g. who are the top customers in last week, last month etc.)
  • Security profiling (e.g. malicious spawning of too many VM within a time window)
  • Financial analysis per data center, customer, organization

In addition, the Glassbeam platform offers capabilities much beyond reporting, dash boarding offered by current log analytics vendors. Glassbeam Explorer application offers robust logs search capability and used by customers for ad-hoc analysis including root cause, device failure, and anomaly detection. Machine learning and predictive analytics help IaaS providers and device manufacturers to offer proactive and predictive services to their end users. These services provide tangible benefits that can be monetized. Our customers package these services into sell through channel programs to generate incremental revenue.

Potential of business value derived from machine logs truly makes Machine logs analytics the next frontier for DCIM market. At Glassbeam, we are excited to be part of this revolution with some real life use cases in production today and a robust roadmap for future innovation with some marquee customers.