7 Must Have Items on Your Checklist Before You Consider Machine Data Analysis

Vijay Vasudevan
Sunday, June 5, 2016

These are the 7 key factors that highlight whether performing machine data analysis realizes credible value for you.

1. Prepare the raw data —

Most of machine data are in the form of logs. Industrial machines are constantly producing valuable operational data (call-home data) on configuration, performance, usage, and other important parameters that define the very life of the device in the field.

So it might be a good idea to know how the machine is configured that generates operating information (call-home data) logged in files. This is important. As knowing as much at this stage ensures we get the most out of interpreting and detecting patterns on how the machine operates.

2. Know what data the log format retains —

Call-home data contains key information about a system and its operation. All call-home data have common data retained in them. For instance, there can be specific nomenclature for naming a particular section, for instance, ‘technical support details,’ indicating information for field staff to find such sections in the log file and use it to perform root cause analysis.

3. Incomprehensible Log Formats do exist —

Some log files are absolutely of no value to analysis. Here we mean information from which machine data analysts cannot extract any real business value. Value attributes such as revenue growth, troubling shooting product malfunctions, root cause analysis, recognizing usage patterns, sales opportunities, and so on.

4. Onboarding the data —

First, you have to organize your data into directories or bundles with similar information. Bundles can contain individually compressed into their own directory structures or sub-directories under them. The key thing here to have as much metadata associated on every log bundle.

5. Expose what inside the data —

Here’s the part that gets interesting. We need to now take that raw data and built an index. Having an index makes it search to search through huge quantities of varied data. Apart from that, we could build things like faceted search mechanisms, similar to the shopping carts on an ecommerce portal.

6. Curating the search index, what should happen? —

Regular expressions as well as complex queries using logical operators help make comprehensive search criteria. That is, apart from the regular keyword search. Having a combination with logical operators helps get to the desired information really fast.

7. Log events and sections create a time-lapse sequence —

All of this indexing and sequencing of data is only going to get somewhere close to the finish line. To cross over and gain the most of it, we will need to know the patterns that don’t seem obvious. Having a dashboard that does that work for you instead of peering at just the index will be that final touch.

Let’s Dig Deeper

We’ve just skimmed the surface when it comes to understanding how and what kind of machine data is collected. I’d suggest looking at our whitepaper “Call_Home_Whitepaper_Glassbeam.pdf” for a more in-depth analysis. It is a great starting point to dig deep and learn how to instrument a device or a bunch of machines to collect and send back log data (Call-Home data) for machine data analytics.

Best of Everything, Out the Box

We’ve built Studio to ease the data preparation, data quality checks, modeling, and transformation to perform all of above in a jiffy. Studio automatically understands what kind of data is coming in, so it intelligently does the indexing by detecting any log format out there. It is also the most robust IDE, to date, to categorize machine and IoT data. Studio recognizes a large number of machine data sources.

Want to know more about Studio, check out the datasheet. Looking for a self-service trial to see how it works, click here for a demo.