Translations of this page:

Osmius

Log file Osmius Agent
Agent Name: osm_ag_LOG00001 Agent Code: LOG00001
Content: Log file Osmius agent user manual
Date: 12/02/2010 Revision Date:

General Information

This agent can monitor various parameters of system log file. It has been tested in different environments. However, we recommend checking its functionality before implanting it in a production environment.

The Log file Osmius agent has been developed using functionalities and enhancements of Osmius framework and ACE libraries, so it is necessary to install the ACE libraries for the proper deployment and operation of this Osmius agent. See chapter: installation.

LOG00001 agent provides up to 2 basic events with configuration parameters to allow scaling of a very simple way, in addition 9 informative events provides details of the monitored system. Events have been selected by the Research and Development Osmius Team as the most interesting for this first development.

All these events are local ones, so the agent must be running in the same server where the log files we want to monitor are. The agent opens, analyses and stores the last size closes the log files in this order for every event to monitor (actually searching strings within the log file). Tests has been made over healthy log files as well as truncated or deleted files with success.

Log File Instance

As a general rule each Osmius agent can monitor one instance type. If you are not familiar with these concepts check out the glossary. Each instance is individually defined in the configuration file (if you want further information go to agents and instances); depending on agent type is the instance type and depending on instance is the connection info.

CONNECTION_INFO

The connection information or connection_info is data that the agent needs to know to connect to the instance. (See more about the connection_info)

For this agent the connection_info would be:

CONNECTION_INFO= -i LOGFILE_ABSOLUTE_PATH

Replacing the following:

  • LOGFILE_ABSOLUTE_PATH: Complete path to the file we want Osmius to monitor.

Examples:

CONECTION_INFO = -i C:\temp\log24.txt
CONECTION_INFO = -i /var/adm/log/log01

TYPE

The type defines the instance type to be monitored. Every declared instance must be associated with a type as you can see here.

For Logfile Instace type:

TYPE= LOG00001

Event summary table

Here's briefly the capabilities of this agent, further down on this page each event is described in more detail.

EVENT DESCRIPTION c w a tseconds Extra parameters / Remarks
LOGPTSE0 Search for coincidences using regular expressions. 0 1 1 60 -S “reg_exp_to_find” [-N “reg_exp_to_exclude” -K {0|1} -R {0|1} -V {0|1}]
Silent Mode ( -s) recommended
LOGSIZE0 Returns the log file size 0 250 500 300 [-U {“B”|“K”|“M”|“G”}] These allow us to define the return units in Bytes, KiloBytes, MegaBytes y GigaBytes.
defaults to KBytes.

Information Events

Info events retieve general data about instance, usually this data doesn't change over time. This kind of events have no severity, simply provides instance details.

EVENT DESCRIPTION tseconds Remarks
LOGINFPA Path 86400 (1 día) Where the Logfile is
LOGINFHO Host Name 86400 (1 día) Hostname where the logfile is
LOGINFUI User ID 86400 (1 día) Logfile owner
LOGINFGI Group ID 86400 (1 día) Log file group
LOGINFLA Access 86400 (1 día) Last accessed time
LOGINFLM Modified 86400 (1 día) Last modified

Logfile Agent events

LOGPTSE0

This event is the one to use when searching for specific strings into log files.
Briefly what this event does is open the log file for reading and try to find the regular expresions (REGEX) defined into the event parameters.
If the read mode (-R parameter) is 0, the firts time the event is executed it will read the file from the begining to its last line, and the next times it will only read the new lines from the last read. If the file haven't changed since last read this event returns 0; if the log files has been emptied the agent will read the file again from the begining.
With the Read mode to 1 the agent will read always from the begining.
At last, the read mode 2 behaves like the 0 reading mode but keeping the md5 sum of the file to see if changes have occurred since the last read and start reading from the beginning in this case. We must be careful not to activate with big size files.
The logfile is always closed after each event execution.

Extra parameters:
This event needs 3 extra parameters:

PARAMETER MEANING Mandatory
-S -S “REGEX”.
This parameter defines the REGEX to use in the search.
We must use POSIX to define the regular expressions. See - POSIX.
yes.
-N -N “REGEX to exclude”.
This parameter defines the REGEX to exclude from those returned by the previous REGEX in the same logfile line.
We must use POSIX to define the regular expressions. See - POSIX.
No.
-K -K {0|1} This parameter defines the ignore case behaviour.
0 means the agent will ignore case differences.
1 means active and the agent will use case comparison.
Default value is i 1, so active and check case differences.
No.
-R -R {0|1} This parameter defines the Read Mode.
If 0 the agent will only read the new lines from the last read.
if the value is 1 the agent will always read the file from the begining.
2 behaves like the 0 reading mode unless the file has changed, then the agent read from the beginning of the file.
The default value is 0, inactive, so the agent will read only the changes from last time.
No.
-V -V {0|1} Information Level in the Output.
if 0 means deactivated, and the event will return the number of coincidences and the text of the last one.
1 means Active the the output is all the configuration values of the extraparameters in this list.
Default value is 0, inactive.
No.

Return Value:

VALUE MEANING
-1 Error
N Number of lines with coincences
applying the configuration in the extra parameters.

Recommended Parameters:

Comparison Direct or Inverse. The higher/lower the value the higher the severity (-c 0 or 1)
Interval 60 seconds – 1 hour –> Depends on the instance importance
Warning Threshold 1 - Contact your administrator
Alert Threshold 1 - Contact your administrator

Parameter settings example:

LOGPTSE0 = -t 60 -c 0 -w 1 -a 3 -S "Osm.*" -N "^L.*" -T “Osm+whatever excluding lines begining with L”

Comment: This event uses regex libraris to compile the regular expresions and find coincidences into the log file.
The user can define from event LOGPTSE0 to LOGPTSE9 up to 10 events of this type in the same logfile instance.

LOGSIZE0

LOGSIZE0 events retrieves the file size and returns this value converted into the units specified by the user.

Extra parameters:
This event needs 1 extra parameters:

PARAMETER MEANING Mandatory
-U -U {“B”|“K”|“M”|“G”}
Size units.
* B: Bytes
* K: KiloBytes
* M: MegaBytes
* G: GigaBytes
The default value is K KBytes.
No.

Return Value:

VALUE MEANING
-1 Error
N Integer with the size of the logfile.

Recommended Parameters:

Comparison Direct or Inverse. The higher/lower the value the higher the severity (-c 0 or 1)
Interval 60 seconds – 1 hour –> Depends on the instance importance
Warning Threshold 1 - Contact your administrator
Alert Threshold 1 - Contact your administrator

Parameter settings example:

LOGSIZE0 = -t 60 -c 0 -w 1 -a 3 -U "M" -T “Log file size in Megabytes”

Comment: The text of the event will return value and units in the text string.

Prerequisites

In order to compile, this agent requires a set of prerequisites, which are generic to compile any Osmius agent, you can see these prerequisites here.

The log file agent needs GNU regex libraries for the desired compiling platform.

For UNIX* like platforms is aeasy to find and install this packages.

For Windows yuo should download the binary GNUwin32 regex, an adapted version of the GNU package. Yuo can find it here: http://gnuwin32.sourceforge.net/packages/regex.htm . Copu the file include\regex.h into yuor Visual C++ include path, copy the “lib” folder into your Visual C++ libraries folder and copy bin\regex2.dll into one of your PATH locations and that should make it.

Makefiles and Compiling

  • Make Project Creator (MPC) is used by Osmius, so creating Makefiles is a trivial task. If you want to learn more about MPC and Osmius check out the section of Makefiles on Osmius.
  • In the particular case of Logfile Osmius agent you can easily generate Makefile as follows:

From the agent directory using console or terminal.

Linux:

$ACE_ROOT/bin/mpc.pl -type make osm_ag_log.mpc

HP-UX and Solaris:

$ACE_ROOT/bin/mpc.pl -type gnuace osm_ag_log.mpc

Windows Visual C 8:

%ACE_ROOT%\bin\mpc.pl -type vc8 osm_ag_log.mpc
  • Now that you have created the Makefile, agent compiling is extremely simple.

Linux:

make -f Makefile.Osm_Ag_Log_Osmius

HP-UX / SOlaris:

gmake -f Makefile.Osm_Ag_Log_Osmius

Windows:

Double click over Osm_Ag_Log_Osmius.vcproj.
change these two options in the project properties:
1.- Linker > General. Add to "Aditional library directories" the route to regex libraries.
2.- Linker > Input. Add "regex.lib" to Additional Dependencies
 
In the solution configuration select Release. Click on Rebuild to compile.

The binaries should be installed in bin folder within OSM_ROOT base folder.

Running the agent

The Osmius log file agent have the same running features of the other Osmius agents. You can check it out int he section Start and Stop Agents.

To run the agent without the Osmius Web Console:

osm_ag_LOG00001[.exe]1) -c osm_ag_LOG00001.ini -m MASTERAG -p 1950 -d [>> osm_ag_log00001.log]2)

Running in standalone mode

This agent, like the others Osmius agents, allows the execution in standalone mode. This option may be particularly useful when developing a new agent or to perform specific agent tests.

Basically you have to add a new value, called SNDCMD, to the agent configuration file (osm_ag_AGCODE.ini) as shown here.

Then you must run the agent setting Master Agent communications port to zero, for example:

osm_ag_LOG00001[.exe] -c osm_ag_LOG00001.ini -m 00000000 -p 0 -d

Tests list

Date: 11/12/2009
Test Results Comment
Creating an instance with all its events in silent mode OK N/A
Creating an instance with all its events with custom text OK N/A
Creating an instance with all its events but no custom text OK N/A
Declaration of 3 instances with all its events to 5 seconds and keep it
running for 48 hours
- N/A
Declare 2 instances, cause a disconnect and then reconnect OK Detects delete on instance
Declare 1 instance and test each event OK N/A
Elimination of general parameter and check unbootable OK N/A
Elimination of instance CONN_INFO and check unbootable OK N/A

APPENDIX

POSIX

For the osmius Logfile agent to understand regular expressions you must use POSIX to define those REGEX into the -S and -N parameters. Here you can peruse the most important ones:

Symbol Definition Example
| OR Operator osm|ius matches “osm” o “ius”
. one character . matches “1”, “c”, “@”, “P” …
* 0 or more times the preceeding character osm* matches “os”, “osm”, “osmm”, “osmmm” …
? 0 or 1 times the preceeding character osm? matches “os” y “osm”
+ 1 or more times the preceeding character osm+ matches “osm”, “osmm”, “osmmm”, “osmmmm”
x{m,M} x at least m times and maximun M times osm{1,3} matches “osm”, “osmosm”, “osmosmosm”
^ Starting Line position ^osm matches osm at the begining of the line
[] Matches one of the characters
between the brackets
[md]ail matches “tail” and “fail”
[^] Matches with one of the characters NOT
one of the chars between the brackets
[^osm] matches every single character except “o”, “s” o “m”
$ Position at the end of the line osmius$ matches “osmius” at the end of the line

More info in the wikipedia

Monitoring Apache Logs

This is just an example of how to monitor the apache web server log file, defined within httpd.conf using directive ErrorLog. If you want to monitor apache log files you must write down the complete path to the file. Note than the last event is the “sum” of the previous, simpler and easier to read.

[OSMIUS_INSTANCES\APACHE01]
 TYPE = LOG00001
 CONNECTION_INFO = -i /var/log/error_log 
   [OSMIUS_INSTANCES\APACHE01\EVENTS]
  LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "emerg" -T "Emergencies - system is unusable"
  LOGPTSE1 = -t 60 -c 0 -w 1 -a 1 -S "alert" -T "Action must be taken immediately"
  LOGPTSE2 = -t 60 -c 0 -w 1 -a 1 -S "crit" -T "Critical Conditions"
  LOGPTSE3 = -t 60 -c 0 -w 1 -a 1 -S "error" -T "Error conditions"
  LOGPTSE4 = -t 60 -c 0 -w 1 -a 3 -S "warn" -T "Warning Conditions"
  LOGPTSE5 = -t 60 -c 0 -w 1 -a 3 -S "notice" -T "Normal but significant condition"
  LOGPTSE6 = -t 60 -c 0 -w 1 -a 5 -S "info" -T "Informational"
  LOGPTSE7 = -t 60 -c 0 -w 1 -a 5 -S "debug" -T "Debug-level messages"
  # LINE SIMPLIFIED CONFIGURATION
  LOGPTSE9 = -t 60 -c 0 -w 1 -a 3 -S "emerg|alert|crit|error|warn|notice|info|debug" -T "Apache ErrorLog"

Monitoring Oracle Alert Logs

[OSMIUS_INSTANCES\ORALOG01]
TYPE = LOG00001
CONNECTION_INFO = -i /u01/app/oracle/admin/your_alert_log 
 [OSMIUS_INSTANCES\ORALOG01\EVENTS]
    LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "^ORA-" -T "Oracle error code"
    LOGPTSE1 = -t 60 -c 0 -w 1 -a 1 -S "cannot" -T "Oracle problem"
    LOGPTSE2 = -t 60 -c 0 -w 1 -a 1 -S "ARCx: Media recovery disabled" -T "arch process is started with the database being in noarchive log mode"

Monitorizando MySQL Logs files

[OSMIUS_INSTANCES\MYSQLLOG]
TYPE = LOG00001
CONNECTION_INFO = -i /var/log/mysql.err 
[OSMIUS_INSTANCES\MYSQLLOG\EVENTS]
	LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "Err" -K 0 -T "Error detected in MySQL log"

Monitoring dmesg in RedHat

Usually linux boxes wite the log files in this folder /var/log.
Common names of these logfiles are: boot.log, dmesg, messages y syslog
Use this agent to monitor patterns of yuor interest.

[OSMIUS_INSTANCES\RHDMESG_]
TYPE = LOG00001
CONNECTION_INFO = -i /var/log/dmesg 
[OSMIUS_INSTANCES\RHDMESG_\EVENTS]
	LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "abort|fail|error" -K 0 -T "Review RedHat dmesg"

Monitoring HP-UX syslog

[OSMIUS_INSTANCES\HPUXSYSL]
TYPE = LOG00001
CONNECTION_INFO = -i /var/adm/syslog/syslog.log 
[OSMIUS_INSTANCES\HPUXSYSL\EVENTS]
	OLD POSIX REGULAR EXPRESSION SYNTAX
	LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "error|fail|warn|alert" -K 0 -T "Error at HP-UX syslog"

Monitoring Tomcat localhost.log

[OSMIUS_INSTANCES\TCATLHST]
TYPE = LOG00001
CONNECTION_INFO = -i /catalinahome/logs/localhost.log 
[OSMIUS_INSTANCES\TCATLHST\EVENTS]
	LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "SEVERE|WARNING" -T "Error at TomCat localhost.log"

Monitoring TomCat catalina.log file

[OSMIUS_INSTANCES\TCATCATL]
TYPE = LOG00001
CONNECTION_INFO = -i /catalinahome/logs/catalina.log 
[OSMIUS_INSTANCES\TCATCATL\EVENTS]
	LOGPTSE0 = -t 60 -c 0 -w 1 -a 1 -S "SEVERE|WARNING" -T "Error TomCat catalina.log"

We must commnet that the Osmius logfile agent each instance represents a different logfile, and you need to declare one instance per logfile to monitor, as yuo can see in the Tomcat example above.

1) In windows system you must add .exe
2) Optional, to store agent messages into a file
 
en/agentes/log00001.txt · Last modified: 2012/12/05 19:21 by osmius
 
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki