Check a Directory of Logs

OutOfMemory Log Check

Graph Logs with Cacti

Graph Logs with Cacti

Check & Alert on Log Inactivity

Checking Directory File Count

Ensure Growth of Log File

Monitor Logs w/ LoGRobot

( Linux (all flavors), HP-UX, AIX, SunOS )

ADVANCED, DETAILED LOG MONITORING & ALERTING FOR UNIX SYSTEMS. Works on all log files regardless of Log Type, Log Format or Log Size - Designed to handle all log monitoring Requirements - No matter how complex!

Get Detailed Information on All Monitored logs with logXray

Scan, Alert, Report, Analyze, Graph - Watch logs Painlessly!

  

Functionality / Specialty:

Simplifies All Log Monitoring Tasks, Specializes mainly on the Monitoring of UNIX Log Files; Alerts directly off User-Specified Conditions and/or Automatically Identified Anomalies - Requires NO changes to your systems; Has NO Library/Module dependencies!

logXray

Log File Pattern Exclusions

Search via time-Frames

Watch for Expected Entries

Database Log File Check

Check Log Time Stamp

Advanced Stats on Logs

Alert on Log Time Stamp

Scanning & Monitoring Unix / Linux Log Files ; Monitor Logs, Analyze - Check Application Logs, Database Logs, System Logs, Custom Logs...Any Log file - Create Excel Reports on Past Log Alerts:  Regardless of the laborious nature of your log monitoring requirement, LogRobot can be trusted to handle it with convenient ease.  Each one of the NAGIOS screenshots on this page show the different ways LogRobot can be used to monitor log files quickly and easily on hundreds and even thousands of hosts.  If you do not have NAGIOS installed in your UNIX environment, please do not panic. NAGIOS can be automatically installed for you (if you want it to be) OR you can schedule LogRobot to run either as a Standalone or with other Monitoring Applications.  Alert notifications on all configured log checks can be sent via SNMP traps and/or to any email address(es) you specify.

State of Monitored Log

Log Entr(ies) Not Found

Show Errors Found in Log

Monitor Large Log Files

Monitor Dynamic Log Files

Alert when Log Not found

Alert on Log Time Stamp

Monitor One Pattern in One Log

Monitor Different Patterns in One Log

Monitor, Graph, Report, Analyze & Alert On All Log Files On Any UNIX Host:

Java, HTTP, Apache, Tomcat;catalina.out, Splunk, Mysql, Oracle, Postfix, Log4j, Mail, Weblogic, Glassfish, System Logs,

Monitor Multiple Patterns in Multiple Logs

LoGrobot Customer Feedback / Testimonials

What We Do:  LogRobot provides Reliable, Efficient and Convenient ways of monitoring logs (with or without Nagios) on UNIX Systems. It ensures Prompt and Accurate notifications on all active log checks and provides users the ability to generate professional excel spreadsheets / reports on past log file alerts. Additionally, it can be configured to fire off notifications whenever behavioral anomalies are detected in monitored logs.

 

  Monitor Specific Log Files in A Specific Directory for New Occurrences of Specific Strings 

Case Scenario:

Monitor all log files in the /var/log directory that have the word 'messages' in their names.  Check each log found matching this criteria for new entries containing the string 'ERROR'. 

If the number of entries found in any 'messages' file in the directory is less than 5, exit with an OK status.  If above 5 but less than 10, alert as Warning.  If above or equal to 10, alert as Critical.

Command:

./logxray localhost /var/tmp/logXray  autoblz  /var/log,include:messages  30m  'ERROR'  '.'  5  10  log_mon_3  -ndfoundn

 


Monitor log files for user-specified entries, then EXCLUDE specific lines from the results  

Case Scenario:

Within the last 30 minutes, find out how many lines in the log file [ /var/log/app.log ] contained both entries of "ERROR" and "Client". If any lines are found containing these two strings (ERROR.*Client), take note of that.

From the list of lines found, see if there are any lines that also contain the keywords "error 404" OR "updateNumber".  If there are, remove them from the list.  After removing them, show me what is left.  If the number of lines left is between 5 and 9, alert as WARNING.  If equal to or over 10, alert as CRITICAL.  If below 5, do not alert!

Command:

./logxray  localhost   /var/tmp/logXray  autonda  /var/log/app.log  30  ‘ERROR.*Client’  '(error 404|updateNumber)'  5  10  applog_tag  -ndshowexcl

 


  Monitor log files for certain entries - ALERT IF those entries are NOT found!  

Case Scenario:

For instance, within the last 30 minutes, if logrobot does not find at least 2 lines containing the words "Success" and "Client"  and "returned 200" OR "update:OK" in the log file, it must alert.  So in other words, the lines to search for MUST contain both words of Success & Client (Success.*Client) AND one or both of the strings returned 200 and update:OK.

Command:

./logxray  localhost   /var/tmp/logXray  /var/log/app.log  30  ‘SUCCESS.*Client’  '(returned 200|update:OK)'   2  2  expected_entry_tag  -ndnotfoundn

 


Monitor Log files for specific entries - When found, display all offending lines in alert  

 

This is particularly helpful in cases where you might want to see the actual lines that contain the patterns you instructed the tool to search for.

 

 Example:

./logxray  localhost  /var/tmp/logXray  autonda  /var/log/app.log  30  ‘ERROR.*Client’  '(returned 200|update:OK)'   5  10  error_exceptions  -ndshow

 


  Scan log files for minutes, hours, days, weeks or months worth of data  

 

For instance, to pull out 2 weeks of information from within a large log file and to find out how many lines contain certain strings and patterns, you can run a command similar to this:

 

Example:

./logxray  localhost  /var/tmp/logXray  autofig  /var/log/app.log  2w  ‘ERROR|error|panic|fail’  ‘ERROR|error|panic|fail’  5  10  -foundn

Notice the [ 2w ].  And also, notice the strings being searched for.  I repeated the strings ‘ERROR|error|panic|fail’ twice because there is no need to specify different search terms to look for.  You don't have to repeat the first string.  You can just enter a dot in its place for the second string..i.e:

./logxray  localhost  /var/tmp/logXray  autofig  /var/log/app.log  2w  ‘ERROR|error|panic|fail’  ‘.’  5  10  -foundn

From this specific example, I'm telling logrobot that I care about EVERY single line that contains any of the keywords I provided.  The [ 2w ] of course means 2 weeks. 

 

See below for the different ways of specifying the date range:

 

5m = 5 minutes (changeable to any number of minutes)

10h = 10 hours (changeable to any number of hours)

2d = 2 days (changeable to any number of days)

2w = 2 weeks (changeable to any number of weeks)

3mo = 3 months (changeable to any number of months)

 


Monitor log file for specific patterns, use user-specified strings to filter out lines to alert on:  

 

 

Command:

  • ./logxray   localhost   /var/tmp/logXray   autonda   /var/log/syslog   60m   'kernel|panic'   'abrt'   3   5   syslog_check   -ndshow

Explanation:

  1. Monitor the /var/log/syslog file

  2. Scan the log for any line containing "kernel" or "panic".

  3. When the above lines are found, from those lines select only the lines that also contain the pattern/keyword "abrt"

    • Ignore all lines which do not have 'abrt' on them

  4. If the number of lines found is less than 3, exit with an OK

  5. If the number of lines found is greater than or equal to 3, and less than 5, exit with a WARNING

  6. If the number of lines found is greater than or equal to 5, exit with a CRITICAL

  7. The name of this log check is syslog_check

  8. From the list of lines found containing 'kernel' or 'panic', exclude/ignore lines that have the string 'abrt' in them

    1. Indicated with the -ndshow

  9. While trying to scan this log file, if it is detected that the timestamp of the log itself is older than 60 minutes, abort immediately

 

 


  Reason for Specifying Exclusions in LogChecks  

Suppose you inherit a UNIX environment at a new job and you're unfamiliar on what to look for within the logs of a particular application, here's an idea; instead of worrying about what to watch for, why not force the logs to reveal their hidden contents?

 

In the example below, logrobot is instructed to search the entire messages file (denoted with the '.').  Then, it is to ignore every line that contains any one of these specific strings: 'nagios-primary nagios' OR 'not responding' OR 'synchronized to'.  Whichever lines are left after these THREE patterns are ignored should be alerted on.

 

The logic here is; if you can identify which entries in the logs are of NO importance to you, you can exclude them from being monitored.  Therefore, if a log file is stripped of the familiar and unwanted, whatever is left will be unfamiliar, thus requiring investigation.
 
[root@nagios-primary]# ./logxray  localhost  /var/tmp/logXray  autonda /var/log/messages 24h
'.' 'nagios-primary nagios|not responding|synchronized to'  1  5  syslog_check  -ndshowexcl


Jun 13 13:40:04 nagios-primary abrt[8269]: saved core dump of pid 8128 (/prod/nagios-core/sbin/status.cgi)
Jun 13 13:40:04 nagios-primary abrtd: Directory 'ccpp-2012-06-13-13:40:04-8128' creation detected
Jun 13 13:40:04 nagios-primary abrtd: Executable '/prod/nagios-core/sbin/status.cgi' doesn't belong to any
Jun 13 13:40:04 nagios-primary abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-06-13-13:40:04
Jun 14 02:20:41 nagios-primary auditd[5813]: Audit daemon rotating log files

 

Cacti Screenshots:  Graph the occurrence or lack of occurrence of specific keywords, strings or patterns - Trend log file characteristics to determine and isolate abnormal behavior in the frequency of entries logged

        

To graph log files, you can use LogRobot with either Cacti or Graphite.  In the above Cacti graphs, LogRobot is used to scan a unique log on 4 different hosts at 5 minute intervals.  The particular log being scanned is an application log which is being written to several hundred times a minute. LogRobot scrapes the application log every 5 minutes and pulls out all entries written to it within that time frame.  From the 5 minute worth of data retrieved after scanning, LogRobot graphs the number of new entries it finds that contain the user-specified strings "hostname.*SUCCESS" or "hostname.*FAILURE" ("hostname" = name of a server) NOTE:  If using Graphite, graphs for new log files are automatically generated and updated without any time consuming configuration.  Also, logs can be polled and graphed as frequently as you desire.

LogXray is a UNIX Fully Featured (upgraded version of LoGrobot) Used Specifically for:

Analyzing, Dissecting, Reporting, Monitoring and Performing Alert Notifications on log files.

LogXray provides detailed information about the log files it monitors:

During each scan of a log file, LogXray provides details on:

  • If the log file being monitored exists and is readable by the user LogXray is running under

    • If it isn't, an alert notification will be generated describing the issue

  • NEW instance(s) of the user-specified list of error patterns found in the most recent scan

  • Total number of NEW entries added to the log file since the last time a check was ran on it

  • Scan time (time range) within which the latest status of a log file check is based upon

  • The most recent size of the monitored log file during each check

  • Performance Data on the log file and each pattern being monitored in the log

  • Statistical reporting on log file behavior - Helps detect abnormal data trends

LogXray does not require any dependencies or libraries in order to function:

Whenever LogXray is introduced to a new UNIX system, here's what happens:

  • A new directory will be auto created on the host based on information provided by the user

  • The ownership of the new directory will be assigned to whichever user LogXray is first run as

    • Ownership is configurable and can be changed to that of any user

  • After the new directory is created, LogXray will run a scan on the log file given to it

  • A "NEW CHECK" alert is displayed so a record exists of when that log check was put in place

  • Subsequent checks on that log file from this point on will be to watch for legitimate issues.

LogXray combines file and directory monitoring features into one tool:

In addition to monitoring logs for error patterns, it can also be used to:

  • Monitor a directory of logs

    • Specify types of files in a directory to include in or exclude from monitoring

    • Specify different thresholds to each file type

    • Get performance data (graphs) on each log being monitored.

  • Monitor the size of specific types of files in a user specified directory

  • Monitor file timestamp (alert if file's last update-time is older than X number of minutes)

  • Monitor the size of a file (alert if file size exceeds X MB or GB)

  • Detect deviations in log growth rate, update times and content - Upgraded Feature!

    • Highly effective in helping to determine the right thresholds - Removes human factor

    • Identifies issues mathematically and can act as predictive application monitoring

Functions:

  • Uses unique algorithms to ensure the size of a log file is irrelevant

    • Log size does not affect the speed with which a log check completes!

    • Monitors files that are several gigabytes in size without imposing heavy system load

  • Allows users to specify string-file(s) when listing lengthy error patterns

  • Alerts if expected records of events are missing from monitored log files

    • Watches for those entries and IF NOT found, alert notifications are generated.

Professional Consultation for the Monitoring of Log Files

 

Professional consultation (or outsourcing) is NOT necessary in most cases.  This is because both the installation of logrobot/logxray and its day to day usage are easy to understand.  You DO NOT have to be an experienced UNIX user to get this log monitoring tool up and running.  We've already done all the hard work to make sure of that.  You only need to follow the basic instructions provided.  The few commands you will need to run are simplistic in nature and do not require any technical sophistication. 

This means, unlike other tools, there isn't a lengthy complicated manual for you to read up on. There are no classes for you to force your employees to take.  All the complex steps you would have had to carry out yourself have already been programmed into the logrobot tool.  All you have to do is run it.

While professional consultation may not be necessary in most cases, there is a variety of reasons some may still wish to delegate the task of monitoring logs to an outside entity.  Whatever the reasons may be, if you need help configuring log checks on any of your UNIX hosts, the logrobot support team will handle it for you.  We are specialized in this particular area and have in our arsenal an enormous collection of automated tools built specifically to execute, on a large scale, a wide range of tasks. 

If this option is of interest to you, Contact Us.

Back to Top

 

Quick References to some of LoGrobot's most popular functions

  1. Exclusions - Specify a List of Patterns to Exclude via Filtering

  2. Monitor Log Files for Expected Record of Events - Alert If Not Found!

  3. Detailed Alerting - Show Offending Entries from Monitored Log Files

  4. Less Detailed Alerting - Do NOT show the Offending entries in alerts

  5. Check Dynamic Logs - Take into account Log Rotation and monitor accordingly

  6. Timeframe - Pull out information from logs using user specified Time Frame

  7. Apache Log File Analysis (Nagios) - Alert / Report / Graph Apache Access logs

  8. Use one check to monitor multiple strings within a log file - Set thresholds per string

  1. Monitor Directory of Log Files - Avoid Specifying Each log file separately

  2. Setup Log Checks for Directory File Count - Monitor the number of files in a directory

  3. Automatically Generate Graphs for all Log File Monitors Configured via Nagios

  4. Check Log Time Stamps - Set up Monitoring Checks to Alert when logs stop updating

  5. Check Log File Size - Monitor the disk space consumption of specific files

  6. Automatically Install Nagios on Red Hat, CentOS, Ubuntu Hosts (avoid manual installs!)

  7. Log Analysis - Alert when a deviation is identified in overall behavior of a log file

  8. Automatically Generate Color-Coded Excel Reports on Log Alert History

Simplified Log Monitoring:  Monitor Logs of Any Application or Database regardless of Format or Size, Generate Quick & Easy Reports on Past Log Check Alerts, See Offending log entries in all dispatched Alert Notifications

Who needs logrobot?

 

Download logrobot if you wish to:

  • Monitor unlimited logs from various Application / Database servers - Alert on specific errors

  • Implement a log monitoring solution that does not require the installation of nonnative modules

  • Monitor & Alert on any log file regardless of size, date and/or time format

  • Monitor multiple log files without any complex time consuming configurations

  • Obtain an automated tool that is configured and ready to go right out of the box

  • Use Nagios or Icinga or Crontab/Emails to Manage the Monitoring of All UNIX Logs

  • Automatically generate Nagios log check configs for several log files on multiple hosts

  • Outsource Log Monitoring or do it yourself with FREE support from our 24/7 Customer Service!

    • Have a technical support team on standby to accommodate all custom requirements

  • Generate automatic color-coded Excel Reports on the alert history of all log checks

  • Avoid writing several scripts of your own, or downloading amateur scripts off the net

  • Utilize a reliable log monitor that is maintained regularly & used heavily in production environments

  • Analyze Logs: Get Notified when unfamiliar lines are introduced into system / application logs

  • Get alerts when critical log files stop getting written to after a specified period of time

  • Utilize an efficient log monitoring tool that eliminates the need to maintain several configuration files

  • Scan logs for specific entries and exclude a list of user specified patterns from the result

  • Allows for monitoring of all log files, even Windows files (mounted through NFS on a UNIX server)

  • Inform through each alert how long ago a particular string / pattern / keyword was last found in a monitored log

  • Remote Agent Included to enable monitoring of logs on several hosts FROM ONE master

    • This is for users who don't currently have NRPE installed in their environment

      • Allows complete control of log checks on all remote hosts / servers

  • Automatically figures out conditions on which to recover alerts, based on log content & other variables

  • Automatically detects log file type and format without user intervention

  • Get notified via email of all events related to your log of choice on any server(s) you specify

  • Updated regularly to meet new demands from various clients / companies

See Complete list of Features

 

Graphing Log Files:

 

 

Example 1:

Using Cacti, a log file on 6 different servers is being graphed for 500 related errors

Each server below is tagged with a different color for easy identification:

 

Example 2:

Using Cacti, a log file on 8 different servers is being graphed for 500 related errors

Each server below is tagged with a different color for easy identification:

 

Download Now

More information on Log File Monitoring using LoGrobot / LoGXray

  1. Monitoring logs - Performing reporting and analysis of log information

  2. Emails Alerts & Notifications - Monitor logs - Receive email alerts on every issue

  3. Different Scenarios for log monitoring - Use logrobot / logxray anyway to monitor logs

  4. NRPE - Monitor logs using the very common nagios NRPE monitoring agent

  1. Custom Monitoring Agent - Monitor logs with our unique Perl monitoring agent

  2. All about log monitoring - Understand the science of Monitoring UNIX log files

  3. Log Monitoring Options - Use any one of our monitoring features to alert on logs

  4. Log monitoring made tremendously easy - What makes logrobot / logxray unique!

Frequently Asked Questions

What exactly can I do with logrobot / logxray?

You can do anything with logrobot as long as it falls under log monitoring & alerting.

logrobot's massive capabilities include, but surely aren't limited to:

  1. Monitoring & Alerting on the contents of system log files (errors, strings, keywords, patterns etc)

  2. Monitoring & Alerting on custom log files (mysql, oracle, apache/http and much more)

  3. Alerting if certain keywords / patterns ARE NOT found inside a specific log within a specific timeframe

  4. Monitoring & Alerting on the timestamps of log files (verify files are being updated regularly)

  5. Monitoring several log files at the same time - (very useful if you have multiple logs to scan)

  6. The capability to monitor both live and rotated logs to ensure nothing is missed

  7. Graphing the frequency with which user-specified patterns occur in log files

    1. Or graph for anomalies

  8. Monitoring & Alerting on the size of log files (ensure logs do not consume too much disk space)

  9. Monitor log files no matter how big they are (even log files that are GIGABYTES in size)

  10. Conditional Monitoring..i.e:

    1. Alert if a certain column of a newly added log entry has a value greater than or less than x

  11. View logs on all your servers from one Web Interface (avoid having to ssh to each host)

  12. Analysis - Easily identify which minute or hour of the day had the most entries recorded

 

What is LoGrobot's Automation?

LoGrobot's Automation is a newly developed mechanism created specifically for users who do not wish to deal with any technical complexities.  With the invention of our tool "kinglazy", we handhold users through every step of the log monitoring process.

Unlike most tools out there, "kinglazy" asks limited questions and with just the stroke of a few buttons on your keyboard, it allows you to:

  • Automatically install and configure the Nagios Core Monitoring Application

    • Yes, even if you have no idea what Nagios is - This will set it all up for you!

  • Generate color-coded Excel spreadsheets on past alerts (any Nagios alert)

  • Auto Create new log check definitions in Nagios for hundreds of log files

  • Create reports on all configured service checks on a list of hosts and much more

 

How else can logrobot be used to scan, monitor and alert on log files:

There is a variety of different ways logrobot can be used to monitor and alert on log files.  The documentation provided with the tool is quite detailed and provides a huge array of examples on how logrobot can save you lots of time...and labor.
 

Can logrobot monitor date-less types of log files?

Yes! Just pass the “autonda” option to logrobot, instead of autofig. To get familiar with all the options available in logrobot, at the command line, simply type './logrobot' (without any arguments)

 

 

What is logXray?

logXray is now the latest upgraded version of LoGrobot which can be used on any UNIX system (Linux, AIX, SunOS, HP-UX).  To download it's full version, click here.

 

Is it easy & Safe to install logrobot?

Yes.

With logrobot, there's very little you have to do.  We are very proud to announce that, unlike other tools, LoGrobot does not require users to install packages, libraries or modules that are unnatural to the UNIX system in order to get it work.  It just works!

Click here to see a demonstration!

 

How Fast is logrobot?

Very fast!  LoGrobot completes its periodic scanning of log files within 0.4 to 1 second.  Listed below are some of the information provided each time a log check runs:

  • number of errors found,

  • total number of lines in the log,

  • log size at the latest log check

  • range of lines scanned within the log file

  • timeframe within which the status of the latest log check is based upon

  • seconds within which the latest errors / entries in log file were detected

  • automatically detects log rollover - prevents missing errors between checks

  • equipped with several features that does MORE than just monitor log content

 

How much is logrobot?

Please see the table at the bottom of the page for all available options.

 

After purchasing logrobot will you assist me if I need any help?

Absolutely! It is highly unlikely you'll need help setting up logrobot.  The tool was deliberately designed and built to be simplistic in its setup.  However, if you find that you require some type of assistance, please do not hesitate to Contact Us..

 

 

Can I scan for multiple strings in a log file instead of searching for them separately?

 

Yes. You can also pass individual thresholds for each string.  Click here for examples.

 

 

 

Back to Top

Licenses

LogRobot / logXray

1-50 UNIX Hosts / Servers

Monitor Unlimited Log files / Directories on each Host!

UNIX: Linux / AIX (all logs and errpt output) / SunOS / HP-UX

$119.95  Buy Now

Instant Download!

Free Support Included with Purchase!

Use ONE tool for all Monitoring related to Logs & Directories!

Unlimited UNIX Hosts / Servers

Monitor Unlimited Log Files / Directories on an Unlimited Number of Hosts!

UNIX: Linux / AIX (all logs and errpt output) / SunOS / HP-UX

$299.95  Buy Now

Instant Download!

Free Support Included with Purchase!

Download Trial Version

Contact Us

Copyright    |    Restrictions    |    Licensed Product    |    Grant of License    |    Warranty    |    License Agreement