There are several Log Viewer / Log Analyzer tools on the market, reflecting the fact that the standard log analytics functionality provided by most operating systems is insufficient to adequately tackle investigation of complicated software issues.
So command-line Linux / Unix OS provides powerful standard log analysis tools, such as grep and regular expressions. These are quite sufficient for analyzing single Log Line data, e.g. calculating transaction rate statistics. However, they will fall short if we, for example. Need to analyze connection between log lines, ie. recognition of logically connected lines. So for the example above, if we were to calculate system response time statistics, we first have to find the beginning and end line for each transaction. Even in Linux, this task requires advanced scripting, and in the end, the task may prove too time-consuming to complete in a reasonable amount of time.
Windows essentially uses graphical user interface, but there is no standard graphical log analyzer, and standard text editors are intended for office use and offer only simple string searching. The same goes for Mac OS, although (as it is based on Linux) it also has the option of using a command line interface. Linux environment can be simulated in Windows using Cygwin software.
In our opinion, consider 5 key questions to choose the Log Viewer best suited for your tasks.
Question # 1. What is the size of the log you need to analyze. Although in many cases software systems have a method of log rotation in place, log files can sometimes grow very large. Software professionals sometimes need to analyze files over 1 GB. Particularly in Windows, managing large files is not an easy task, so you should evaluate the performance of a tool for a log of your size.
Question # 2 Do you need to analyze binary files. Most tools only support logs in text format.
Question # 3. Do you foresee the need to merge multiple logs for analysis at longer intervals? For example, to get more reliable statistics. Not all tools support this.
Question # 4. Do you prefer a graphical analysis tool. It is well known that an image can be worth a thousand words, but is it also true in log analysis? The answer is yes. Each log line is overloaded with information, and as we saw earlier, the number of lines in the log is usually too large to easily navigate between them visually – you can’t just store as much information in your memory as it scrolls on screen. Graphics add a new dimension to the analysis, making it easier to see trends or just quicker to find what you’re looking for.
Adding graphical functionality to a Log Viewer is not an easy task. A log file does not have a regular structure: usually logs are the place where many independent processes write their messages, each using perhaps its own format. Compared to such a well-structured file such as. spreadsheets, we can’t just plot one column against another.
One way to solve this problem is to add functionality to log recognition and analysis that brings us to
Question # 5. Do you need to search for log patterns and analyze them. Log pattern is a repeated combination of keywords that can have different values associated with them.
Let’s take an example. Each web server log contains a series of repeated HTTP GET requests and 200 OK (or other code) responses. Each request has a URI that points to a specific page on the server. These 3 objects (GET, URI and 200 OK) form a natural pattern, and its occurrence as a function of time can be depicted, where the request URI and / or response code can be rendered as a heading for the graph point.
This example shows how powerful log analysis can be when log processing using patterns is combined with graphical presentation.