Splunk is a great Operational Intelligence solution capable of processing, searching and analyzing masses of machine-generated data from a multitude of disparate sources. By complementing it with an APM solution you can deliver insights that provide value beyond the traditional log analytics Splunk is built upon:
True Operational Intelligence with dynaTrace and Splunk for Application Monitoring
Operational Intelligence: Let Your Data Drive Your Business
In a nutshell, the purpose behind Operational Intelligence is the ability to make well-informed decisions quickly based on insights gained from business activity data, with data sources ranging anywhere from applications to the infrastructure to social media platforms.
Analyze data from a multitude of disparate data sources with Splunk (courtesy of splunk.com)
Splunk’s capabilities to process and fuse large volumes of continuously streamed and discretely pushed event data with masses of historical data reliably and with low latency supports businesses to continuously improve their processes, detect anomalies and deficiencies, as well as discover new opportunities.
Many industries are realizing this insight hidden in their log data. Financial services companies, for example, use Splunk to dashboard analytics on infrastructure log files over long periods of time, understanding trends allows for smarter decisions to be made. Analysis on this infrastructure has critical impact when applications are transmitting billions of dollars a day.
Financial services companies are not the only ones taking advantage of this level of log analysis. SaaS companies are using Splunk to analyze log data from many siloed apps hosted for their customers, all with separate system profiles. Splunk allows them to set up custom views with insights and alerts on all their separate application infrastructures.
Why complement Splunk with dynaTrace?
“So, with a solution like Splunk, gaining insights from all our data will be a snap, right?” Unfortunately not. What if I told you that you are essentially building your insights on masses of machine-generated log data. Let’s discuss why this matters.
Machine-generated data in the “Big Data Pyramid” (courtesy of hadoopilluminated.com)
In Big Data parlance, machine-generated data, as opposed to human-generated data, signifies data which were generated by a computer process without human intervention, and which typically appear in large quantities. Machine-generated data originates from sources such as: applications, application servers, web servers, firewalls, etc. and thus often occur as traditional log data. However, unstructured log data is not exactly convenient to drive an analytics solution because they require you to:
1. Tell your Solution What Matters to You
Because log data is essentially unstructured, you cannot easily access the various bits and pieces of information encoded into a log message. You will need to teach your analytics solution the patterns by which any valuable information can be identified for later search and analyses:
Identify bits and pieces of valuable information inside log messages
2. Reconsider your Application Logging Strategy
While there is not much you can do about how your firewall logs data, you will need to put large efforts into designing and maintaining a thorough logging strategy that serves all the information you will want to have monitored for your application. However, you may want to contemplate whether you really want to take these efforts for a variety of reasons:
- Semantic Logging is, undoubtedly, a useful concept around writing log messages specifically for gathering analytics data that also emphasizes structure and readability. However, it can help toimprove your logging only where you own the code and thus leaves out code from any third-party libraries.
- Operational Intelligence solutions rely on you to provide context for your log messages, as outlined in Splunk’s Logging Best Practices. Only then will you be able to correlate events of a particular user transaction and understand the paths your users are taking through your application. Again, context cannot be retained easily once you leave your code.
- Efforts to establish and maintain a robust logging strategy that delivers must be aligned with ongoing development activities. You would also need to make sure that what your strategy provides is kept in sync with the expectations of your Operational Intelligence solution. If in doubt, and you better be, you will want to enforce automated testing of your strategy to verify your assumptions.
What this would mean to you? Establishing and maintaining an application logging strategy for your analytics solution that delivers actionable insights involves a lot of disciplined work from everyone involved:
- Developers: need to maintain a logging strategy whose messages are scattered all over their code base: whenever functionality is added or changed, several parts of the application need to be augmented. This makes developing a thorough logging strategy a poorly maintainable, time-consuming and thus error-prone cross-cutting concern.
- Test Automation Engineers: should enforce automated testing to assert that the assumptions of the Operational Intelligence solution on the setup of the input log data hold.
- Product Owners and Software Architects: need to cope with a decrease in velocity when they buy into developing and maintaining a thorough logging strategy. They also need to accept that the visibility into user transactions ends where the ownership of their code ends.
- Operations: continuously need to test and verify the correct functionality of the overall solution.
Why I am telling you all this? Because we have a lot of customers who were already using Splunk before they implemented dynaTrace. They had a really hard time correlating log messages due to the lack of context and were unable to answer one of the most important questions: “how many users were affected by this particular application failure?” We were able to solve their worries by delivering these features out-of-the-box:
- They could keep their talent focused on critical development, testing and operations since there is no need to change your code, no logging, testing and verification involved.
- They could quickly get to the root cause of performance issues because they had full end-to-end context for all user interactions including any third-party code which brings you full transaction visibility including method arguments, HTTP headers, HTTP query string parameters, etc.
- They had analytics customized to their critical focuses because they could decide which data needs to be captured.
Easy Steps to True Operational Intelligence with Splunk and dynaTrace
- Get and install Splunk
- Get and install the 15 Days Free Trial of dynaTrace
- Get and install the Compuware APM dynaTrace for Splunk App
- Enable the Real-Time Business Transactions Feed in dynaTrace:
Enable the Real-Time Business Transaction Feed in dynaTrace - Selectively export Business Transactions data to Splunk in dynaTrace:
Configure a particular Business Transaction to export data
That’s it. You may refer to the documentation of our dynaTrace for Splunk App for additional information. Here is a selection of insights you could already get today:
Dashboard #1: Top Conversions by Country, Top Landing- and Exit Pages
Top Conversions by Country, Top Landing- and Exit Pages
Dashboard #2: Visits Across the Globe
Visits across the globe
Dashboard #3: KPIs
KPIs: Conversion Rates, Bounce Rates, Average Visit Duration, etc.
Dashboard #4: Transaction Timeline and Details
Transactions timeline and details
However, there is more to it: should you feel the need to drill down deeper on a particular transaction to understand the root cause of an issue or precisely who was affected, you can fire up the PurePath in dynaTrace from within Splunk:
Drill down to dynaTrace from raw transactions data in Splunk
…and see deep analysis happen:
Deeply analyzing a transaction in dynaTrace
Conclusion
The road to true Operational Intelligence can be a tough one – but it does not necessarily need to be that way! By integrating dynaTrace with Splunk you won’t have to rely on application logging or require any code changes and that does not slow you down. Instead, it will help accelerate your business by providing true visibility into your applications, independent of whether it is your machine, your code or not. This level of end-user visibility enables you to communicate in terms of what matters most to your organization – customer experience.
Should you want to know more about the inherent limitations of logging, you might want to refer to one of my recent articles “Software Quality Metrics for your Continuous Delivery Pipeline – Part III – Logging”.
