Linux Log Monitoring ; Monitor, Alert on & Analyze Linux / UNIX Log files the easy way ; Application Logs, Database logs, System Logs, Custom Logs, Any log file - Genenerate graphs automatically on all monitored logs - Trend any log file metric you desire - Utilize the versitility of LoGrobot to eliminate the tedious effort often required to configure log checks - Perform all log monitoring tasks with just one tool!
Some of the many labor saving capabilities and benefits of LoGrobot include:
Alert, Report, Graph Apache Access logs - Pull out as much information as you wish from access / http log files. Alert on the number of entries containing specific status codes. Graph all status codes...automatically.
Monitor log files for expected record of events - send alert notifications if those events are not found in the latest scan of all new entries added to the log!
Exclusions - Specifying a list of patterns to exclude when attempting to isolate unique entries of interest from monitored log files. When a log check reports an issue, use this option to get rid of unwanted entries from the list of error patterns detected.
Detailed Log Check Notifications - Whenever a log check detects entries from a monitored log containing the patterns you tell it to watch for, it will show all those lines in the alert notification it sends out to you. If you wish to only see the actual lines from the log that triggered the alert, add 'noextraoutput' to the logXray command.
Whenever a log check detects an issue in a monitored log, do NOT show the offending entries in alerts.
Monitor and alert on dynamic Logs - Take into account log rotation and monitor accordingly. Dynamic logs are log files that change names frequently..i.e. every minute, hour or daily.
Tail log files intelligently via time frames if the logs are date and time formatted. For example, be able to pull out log entries from 5 minutes, 10 minutes, 1 hour ago till now. Avoid grabbing random number of lines.
Use one check to monitor multiple patterns within a log file - set thresholds per pattern. Get alert notifications on a particular pattern if and only if the number of entries found containing that pattern is greater than the thresholds you specified.
Monitor a directory of log files - avoid specifying each log file separately. Monitor a specific log file across several different directories.
Check time stamps of specific log files - Set up monitoring checks to alert when those logs stop updating.
Directory File Count Monitoring - Monitor the number of files in a directory. Alert if the number of a user specified type of file in a specific directory breaches set thresholds.
Check log file size - Monitor the disk space consumption of specific log files - Alert if the size of a monitored log grows past user specified thresholds.
If you do not have Nagios, Zabbix, Zenoss or any similar monitoring application installed in your environment, that's ok. You will still be able to monitor remote log files with our custom monitoring agent, which is included in the logXray package.
NRPE - Monitor logs using the very common nagios NRPE monitoring agent. Follow these instructions to set up logXray on the remote hosts on which you have logs to monitor.
Custom log files
If you have log files that need to be monitored in a very specific way, email us to request custom development - All customers get 2 FREE requests for customization with purchase of LoGrobot.
2-day Range - Aggregate of 4 different log files on 4 different hosts. This graph shows the frequency with which each log is being written to.Preview
2-day Range - Log Size - Trends the size of log files. Allows users to keep an eye on log rotation to ensure logs are being rolled over at the expected size and time.Preview
Monitors the size of a specific log file. Alerts if log file size remains the same as it was during the last scan. This log check can also be configured to alert if growth is beneath user-specified thresholds.Preview
Monitors a log file for a specific string. When that string is found in the log file, this alert shows the actual entries from the log containing the offending pattern!Preview
What is a Log File Monitor?
A log file monitor is a utility designed and built specifically to monitor and alert on messages produced by computer systems and the applications / databases that run on them.
In UNIX, the monitoring of log files is absolutely necessary, and for good reason. You see, the time of a Unix Professional is valuable. Few, if any, can afford to spend hours each day scouring through the many log files that are generated by systems and network applications. However, if you fail to quickly recognize the abnormal or fatal events chronicled in these log files, entire networks can be abused and/or removed from service....which can cost your company dearly, monetarily speaking.
If you wish to monitor log files, there are basically [ 3 ] options available to you:
1). You can try writing your own log monitoring script and see how far that takes you (this is worth looking into if you only have a couple of logs to monitor)
2). Download any of the FREE log monitoring scripts that are available all over the internet OR
3). Purchase a professional tool that was developed specifically for situations like yours and that can easily accommodate future customizations, if necessary
If you embark on a journey to write your own script, you have to understand that it will be an endeavor that will take years to complete, and that's assuming you're a skilled programmer. Monitoring log files goes far beyond simply watching the contents of files for specific errors. As time goes on, there will be new requirements, changes, and continuous requests for modifications which in the end, if the developer isn't creative, can lead to an unusable script - one that is not user friendly.
If you choose to download the FREE log monitoring scripts that are available on the internet, you will quickly discover how ineffective they all are and how much work is necessary to get them to cooperate. If this is the option you choose to go with, you must ask yourself some very important questions:
(a). Will I be able to easily administer the creation and modification of several log checks (from a central location) using this method?
(b). Is this method scalable?
(c). Can I use this one method to monitor different logs on several hundred servers, or am I going to have to do a lot of configurations, compilations, installations, tweaking etc?
The answers to these questions are usually quite depressing. Proceed with caution.
Characteristics of the Ideal Log Monitor:
When searching for the right utility to use to monitor & alert on log files, what features should the perfect tool have?
The ideal log monitor must be able to scan and monitor log files in a very short period of time, preferably in seconds (no matter how big the log file is). At the very least, the perfect log monitor must be able to:
1). Detect abnormal usage patterns in log files
2). Recognize system or network abuse (through mathematical analysis of data)
3). Detect vulnerability scans (e.g. port scans) through the use of user-specified patterns
4). Detect intruders or attempted intrusions (through the use of user-specified patterns)
5). Detect resource shortages (e.g. slow response times, out-of-memory conditions etc)
6). Detect imminent application and system failures (this is usually in some log file on your system)
7). Scan, monitor & alert on log files of different formats (this is absolutely crucial)
While each feature listed here is important, it is worth noting that above all else, the perfect log monitoring utility must be easy to use. Users SHOULD NEVER have to spend too much time reading documentations before being able to utilize a software. The more complex a utility is, the more likely it is to be used the wrong way or abandoned altogether. Imagine having to re-read the Instruction Guide of your Television remote control each time you wanted to use it. Can you picture the annoyance of that?
When it comes to log monitoring, ease of use is essential. I cannot stress this enough. The developer(s) must focus a great deal of effort into drastically limiting or eliminating the need for configuration files. Also, the syntax of the tool must be easily comprehensible and applicable directly from the command line. This means, if a random user were to run the tool from the command line, there shouldn't be room for confusion. That user should be able to conveniently obtain whichever end result he/she was expecting WITHOUT having to read several pages of complex documentations or desperately scouring Google for help!
This is where the superiority of LoGrobot comes into play. LoGrobot is a commercial Log Monitoring utility that is very easy to utilize. It is robust, seasoned and efficiently versatile like no other tool. It understands the overriding significance of log alerting and focuses on ensuring only valid alarms are generated for the log files it monitors. Installation wise, LoGrobot does not require the addition of any nonnative modules or libraries to the system. Which means, you can install it safely on production/dev/qa servers without tampering with existing libraries or modules.
LoGrobot has a wide range of capabilities. It isn't limited to only scanning log file contents for errors. It can do virtually anything as long as it falls under the banner of log monitoring. Additionally, LoGrobot has years of real life situations, possibilities and conditions built into it, which basically means it is highly unlikely you will come up with a scenario that hasn't already been thought of and programmed into the tool. In the unlikely event that does happen, chances are, work is already in progress to address it.
When it comes to keeping an unwavering eye on all important log files in your UNIX environment, you need ONE log monitoring tool, and LoGrobot is that tool!
Aggregate graphs of multiple log files from several different hosts and/or servers into single graph(s) - See the logging activities of applications, dbs, servers all in one place. Beautify your log file activities - Visually inspect graphs / charts of all monitored logs - Easily identify anomalies (unusual rises and/or drops) in any measured log metric!
See detailed information on the log file being monitored i.e. log size, total number of entries in log, latest count of new error patterns, rate at which log file is being updated, percentage of error patterns in relation to total new entries, the scheduling consistency at which log check is running (helps identify when system is under heavy load - if a system is being taxed, checks will not run on time, and if they do, they'll take a bit longer - an additional second or two - to complete).
If you do not wish to see extra information in your log file alerts and only care for the actual entries found in the log, that is also possible. See the image below. Just add the option 'noextraoutput' to your logrobot / logXray command.
With just one log check, monitor multiple patterns within a log file - When an alert is generated, see the actual offending string(s)/pattern(s) that triggered it. Additionally, with the unique artificial intelligence built into logXray, users can now set conditional alerting on special log entries..i.e. when special entries are detected in a log, look in a certain column of those entries for a specific value. Alert as warning or critical if that value breaches set thresholds.
Provide your name (or company name) and Email Address below. Describe the situations or scenarios you wish to use logrobot for. We'll get back to you promptly!
How is LoGrobot different from any other Log File Monitoring tool?
It isn't the fact that it can alert on any entry in your logs that makes it superior. No. Any log monitoring tool can do that. The superiority of logrobot as a log monitoring software can be broken down into two major parts.
First, it is the unique ability to perform very complex actions while requiring only the simplest and the most basic information from the user, that sets loGrobot apart from its competitors. Yes, you do not have to be technically savvy to use loGrobot nor do you have to read a ton of documentation to figure out how it works. We made sure of that. loGrobot is an easy to use command line log monitoring plugin that integrates seamlessly with any monitoring application.
Secondly, loGrobot is the only log monitoring software that has several labor saving functions built into it. What that means is, with just one tool, you can do just about anything with your log files without having to use or write different scripts. And when we say 'just about everything', we really do mean that. With loGrobot, users can monitor patterns in single or multiple log files with one log check, monitor log size, growth, timestamps, directories and much, much more.
Listed below are some of the most common log monitoring tasks you can perform with loGrobot:
Who needs loGrobot? Who is it meant for?
Download LoGrobot if you wish to:
Other Specific Features:
How much is LoGrobot?
Is it easy to install LoGrobot?
Absolutely! All you have to do, after downloading the LoGrobot zip file, is run one simple command and everything will be automatically set up in place for you.
Can I use LoGrobot to monitor a huge list of strings which won't fit on the command line?
Absolutely! If your list of strings is too long or too many to fit nicely on the command line, you can instruct LoGrobot to use configuration files instead.
In what other ways can LoGrobot be used to monitor and alert on log files:
There are many different ways in which LoGrobot can be used to monitor log files.
If you need to monitor your log files in a particular way, just shoot us an email. We'll show you how to do what you want to do. And if your request is so unique, we'll develop a function to perform it for you..for free!
How easy is it to setup log checks / log monitoring for log files?
It is very easy to setup logchecks through LoGrobot. The LoGrobot tool is designed in such a way that allows it to handle, by itself, all the several little steps other tools would have had you do yourself...i.e. compilations, editing a bazillion files, searching for ReadMe text files, having to read those text files, troubleshooting issues...etc. With LoGrobot, setting up logchecks is indeed as simple as it appears!
Can LoGrobot monitor log files that do not have a dated format?
Yes! Just pass the autonda option to LoGrobot, instead of autofig. You can try either one of those options to see which one you prefer. "autonda" is a newer feature with an entirely different set of abilities.
To get familiar with all the options available in LoGrobot, at the command line, simply type:
[root@nagios-master ]# ./logrobot auto (or you can type it with no arguments)
Typing "logrobot auto" as shown above will display real life samples of how to use the different features that are available. This saves you the hassle of having to scour through documentations. The help you need will be right in front of you, directly at your fingertips.
After purchasing LoGrobot will you assist me if I need any help?
Absolutely! Support is free. It is highly unlikely you'll need help setting up LoGrobot. The tool was deliberately designed and built to be simplistic in its setup. However, if you find that you require some type of assistance, please do not hesitate to Contact Us.
Do I Have to use LoGrobot with Nagios, Zabbix, Zenoss, Graphite or any Application at all?
Absolutely not! As was shown in the preceding examples, LoGrobot runs naturally by itself. It does not rely on any other application to function. However, it is recommended to have Nagios (or a similar application) installed IF you have multiple logs to monitor on several different hosts.
Nagios (or similar applications) will allow you to manage all log file monitoring from one server through one web interface. Also, if you would like to generate excel reports on the history of all log file alerts received on your monitored list of hosts, this is possible, if using Nagios.
Graphing: If you wish to graph the contents of log files, you will need to install Cacti or Graphite.
If I choose to install Graphite (the frontend for log monitoring graphs), will you provide help with the installation?
Absolutely! We have an automated utility in our arsenal that will install the application for you, relieving you of the tedious responsibility of having to labor over complex configurations. You actually don't even need to install any of these applications.
If you sign up for our monthly service ($49.95 / month), we'll do everything for you. We'll give you the logXray tool to put on each host on which logs need to be monitored. Then, logXray will send information about any log file you tell it to watch on those hosts to us. We'll analyze all information received, alert on them, graph them and allow you to generate reports on them. Contact Us for more information.
What exactly is the NagiosLogMonitor (nlm)?
NagiosLogMonitor is the plugin that interprets the outputs produced by LoGrobot. It translates all the numbers into words so that they make sense to humans.
This plugin is the interpreter behind the screenshots on this page. Please note, you do not have to have Nagios installed in order to use it. You can run it through CRONTAB as well.
In the latest version of LoGrobot (now called logXray), NagiosLogMonitor is no longer a separate module. It has been built into the LoGrobot tool itself.
Are there any log monitoring situations LoGrobot will not be able to handle?
No. It is highly unlikely for that to happen. While we understand that there is an endless variety of custom ways some logs may need to be monitored, the fact of the matter is, LoGrobot already has years of production-based situations built into it. The tool as it is, is quite capable of handling any scenario you throw at it.
However, if you happen upon an extremely unique situation that isn't already accounted for, however remote the possibility of that may be, LoGrobot will be customized & upgraded specifically for you, at no additional cost.
Is there a Money Back Guarantee?
Yes. There is a 60 Day Return Policy. If you're not satisfied for any reason with the LoGrobot tool, by all means, please let us know. We have never had to issue a refund, but we will if requested. We stand proudly by our product and the services it delivers.
After downloading LoGrobot, can I submit requests to add custom / personalized features?
Absolutely. We usually complete custom requests within 24 to 72 hours of submission. If your request isn't of an urgent nature, please state so in your email. NON-Urgent email requests will be completed within 5 business days. Contact us for more information.
I want to monitor log files on ALL my UNIX servers - What must I do?
If using NRPE:
And that's it!
If using the Custom Monitoring Agent that comes with logXray:
Place LoGrobot on all your UNIX machines. Installation is very simple:
Basically, here is all you have to do on each of your UNIX hosts:
Explanation of the parameters:
I want to be able to also view log files on all my hosts from one web interface
LoGrobot comes with a PHP Web Interface that enables you to specify which host a log file is on, what TCP port you wish to connect to on that host, and where the log file is located on the host. The benefit of this feature is so that you DON'T have to manually log in to any server in order to view the contents of the log files on that server.
This feature basically eliminates the tedious task of having to type passwords repetitively to gain access to several servers. It eliminates the need to generate tokens or to tamper with ssh config files. None of that is necessary. If you want to view a particular log file on host x, simply bring up the PHP Web Interface, type in the host name, the location of the log file, what port to connect to, then hit submit!
Can I scan for multiple strings in a log file instead of searching for them separately?
Yes. If you want to monitor your logs for multiple strings, you can run LoGrobot this way:
Monitor log for strings "Error" and "Panic" and "Exception" and "Kernel"
./logxray localhost /var/tmp/logXray autofig /path/to/logfile 60m 'Error|Panic|Exception|Kernel' '.' 1 2 test_tag -ndshow
Whenever LoGrobot finds a log entry that contains any of these strings, it will alert. Additionally, it will show the exact line(s) found in the log that matched the strings you specified. In other words, NO, you do not have to run several instances of LoGrobot for different strings. LoGrobot can monitor them all in one go!
Is Nagios, Cacti, Graphite free to use?
Yes. The listed applications are free to download and use. They are 100% free.
What makes LoGrobot Superior to other Log Monitoring tools?
Plugin / Service / Standalone
Monitor Unlimited logs/directories on Unlimited hosts/servers!$49.95/ month
Plugin / Service / Standalone
Monitor Unlimited logs/directories on Unlimited hosts/servers!$149.95/ 6 Months
Plugin / Service / Standalone
Monitor Unlimited logs/directories on Unlimited hosts/servers!$299.95/ Year