Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Deduplication

News Event Correlation Recommended Links Message Keys Automatic acknowledgement  
Default Policy Groups Policies Correlation Composer ECS Designer Humor Etc

Note: HP renamed the product called now HP operations manager way too many times. Also it is very inconsistent with using abbreviations. Here we will assume that the term "HP Operations manager" and abbreviations HPOM, OMU, and OVO  mean the same thing :-)

There are many configuration tasks for the management server. After the initial installation, customize the server environment. A few of the most popular initial configuration tasks are described in this section.

The management server produces messages immediately after installing the agent software, policies, actions, monitors, and commands. The agent software running on the server generates the messages. Many of the messages are duplicates; this happens if the HPOM agent detects the same error condition multiple times. If you disable the policies, you will not see the messages from that node in the message browser. Initially, you just want to suppress the duplicate messages.

After a new installation, you may find it necessary to reduce the number of duplicate messages that appear in the message browser. You can reduce the amount of redundant information appearing in the message browser by changing the server configuration with the duplicate message suppression feature.

Messages for duplicate suppression are grouped using so called message key which can be concatenation of several message attributes and variables. Message key represent the signature (primary filter) based on event attributes. Further processing takes place when an event matches all attributes set in the message key .

Deduplication is performed set of messages with identical message keys only.

Example Server Duplicate Message Suppression

Select Actions→Server→Configure. Select duplicate message suppression
 

This configures the server to display and increment a counter for duplicate messages in the message browser. If there is information that should accompany a message in the form of a note, it is referred to as an annotation. You can either discard the duplicate messages or add them as annotations to the original message.

 

The runtime engine for HPOM event-correlation is available for the HP Operations management server and the HP Operations agent. See the

HPOM Installation Guide for the Management Server for a list of platforms on which the runtime engine currently runs.

For more information about the concepts behind event correlation, as well as the way event correlation works in HPOM, see the HPOM Concepts Guide.

Configuring Event Correlation for HPOM

The HPOM message-source policy allows you to specify which conditions generate a message and whether or not the generated message is copied or diverted to the message stream interface (MSI) from where it may be passed to and processed by the event correlation template. Follow the procedure below to configure the event correlation:

1. Enable output from the HPOM internal message stream to the MSI as required at either (or both) management server or managed node level as follows:

2. Set up the HPOM message-source policy so that the message conditions that you are interested in output messages as intended. For each condition statement (policy block starting with CONDITION), make sure that: You specify a message-type attribute in the CONDITION or SET block of the policy body; attribute can be any of the allowed attributes specified in the policy grammar. For information on policy grammar, see the HPOM Concepts Guide.

The specified message-type attribute matches the corresponding attribute referenced in the Input node that starts the flow in the event-correlation circuit which you want to process the message in question.

3. Enable the Copy/Divert to MSI option for each condition that you wish to output a message, using one of the following keywords in the SET section(s): MPI_SV_COPY_MSG Copy message to MSI and pass it on to HPOM server processes. MPI_SV_DIVERT_MSG Send message to MSI and remove it from HPOM processing chain on the management server. MPI_AGT_COPY_MSG On a managed node, copy

message to MSI and pass it on to HPOM server processes. MPI_AGT_DIVERT_MSG On a managed node, send message to MSI and remove it from HPOM processing chain on the management server.

4. Enable, if required, the logging options for each template, by using one or more of the following keywords in the policy body, before specifying conditions: LOGMATCHEDMSGCOND Logs messages matching message conditions (section starting with MSGCONDITIONS). LOGMATCHEDSUPPRESS Logs messages matching suppress conditions (section starting with SUPPRESSCONDITIONS). LOGUNMATCHED Logs unmatched messages.

Suppressing Duplicate Messages

In general terms, duplicate messages are messages that report the same event or a similar event. For example, each time a user switches to another user, HPOM generates a message. If the user always changes to the same user, you could consider this information superfluous, declare it an identical event, and have HPOM suppress further messages relating to it. In addition, HPOM lets you decide how often or for how long further messages are suppressed before a “new” duplicate message is sent. NOTE If an incoming, duplicate message has a different severity or message text than the existing duplicate messages, you can configure HPOM to display the new values instead of the previous data. For more information, see “Updating Severity and Message Text of Duplicate Messages” on page 375.

You can configure HPOM to suppress duplicate messages on the managed node or on the management server. Filtering out (or suppressing) messages on the managed node reduces network traffic and keeps the management server free for other tasks. Because suppressing messages on the management node is more beneficial to performance, HPOM offers a variety of suppression types and settings on the managed node, but only a global setting on the management server. Use the configuration variable OPC_MAX_DUPL_ANNO to limit the number of duplicate messages for which annotations are added.

Verifying Suppression Types

If the condition event matches the condition, HPOM verifies that one of the following suppression types has been selected:

If either message does not have a message key, the following message attributes must be identical: If an identical message already exists, and the second event occurs within a specified time interval or below a configured counter, the second message is suppressed.

The log file of the syslog daemon, /var/adm/syslog/syslog.log on HP-UX, illustrates suppressing duplicate messages with the Suppress Identical Output Messages option. For example, consider the following lines from the syslog log file:

>Mar 14 14:39:01 server inetd[9900]: telnet/tcp:
Connection from node1 at Tue Mar 14 14:39:01 2009
Mar 14 12:46:02 server inetd[9005]: login/tcp: Connection
from node2 at Tue Mar 14 12:46:02 2009
The pattern-matching text might look like this:
"inetd\[<#>\] <@.service>: Connection from <@.from_node>"
The important characteristics of the inetd connection messages are the local node, the node from which the connection is made, and the inetd service. The syslog time stamp, the PID, and the connection time are not important.

A message key might look like this: inetd_connect_from:<$MSG_NODE_NAME>:<from_node>: <service>

This message key would suppress all messages from the same nodes that connect to the same node and connect for the same service. Duplicate message suppression processes only those messages that are generated by the same condition. If you want to suppress duplicate messages that are generated by different conditions, enable this functionality on the management server. For details, see “Suppressing Duplicate Messages on the Management Server” on page 374.

NOTE Duplicate message suppression on managed nodes is not available for threshold monitor policies. To find out how to suppress duplicate messages from threshold monitor policies, see “Suppressing Duplicate Messages on the Management Server” on page 374.

You can specify the type of duplicate message suppression, as well as the time interval and counter or threshold settings. There are three types of duplicate message suppression. Use the following keywords to achieve the desired results:

SUPP_DUPL_COND Suppresses all messages that match the same condition.
SUPP_DUPL_IDENT Suppresses all messages that have same original message text (input).
SUPP_DUPL_IDENT_OUTPUT_MSG Suppresses all messages that have the same output message text.

The value following these keywords represents the time interval within which messages are suppressed. If instead of a time value, a keyword COUNTER_THRESHOLD (followed by a number) is used, the messages are suppressed until the defined number of messages are received, after which one message is sent and the counter is reset. Using keyword RESET_COUNTER_INTERVAL will set a time interval after which the counter is reset regardless of the number of messages received. Keyword RESEND sets the time limit after which messages are being sent again. For more information, see the policy body grammar in Appendix A, “Policy Body Grammar,” on page 475.

Type of Suppression Settings

You can choose between the following suppression settings:

Suppression Based on Time

Figure 4-10 illustrates suppression based on time. For suppression of identical input events, use SUPP_DUPL_IDENT keyword instead. For suppression of identical output messages, use SUPP_DUPL_IDENT_OUTPUT_MSG keyword.

  1. The first event (E1) matches a condition. A message is sent. The timer is started.
  2. A second event (E2) occurs 25 seconds later. This event occurs less than 30 seconds after the first event. So it is suppressed.
  3. A third matching event (E3) occurs less than 30 seconds after the second event. So it is also suppressed.
  4. The next matching event (E4) occurs less than 30 seconds after the third event. But it also occurs more than 60 seconds after the first event. So a new message is sent.
E1 E2 E3 E4
1m = 60s
25s 25s 25s
No message is sent.
Message is sent.
Ex Events that are identical.
SUPP_DUPL_COND 30s
RESEND 1m
Implementing Message Policies
Strategies for Optimal Message Filtering
Chapter 4 373
Suppression Based on Counter
Figure 4-11 illustrates suppression based on counter.
Figure 4-11 Suppression Based on Counter
1. The first event (E1) matches a condition. The counter increments to
one. No message is sent because the threshold of two has not yet
been reached.
2. A second matching event (E2) occurs 25 seconds later. The counter
increments to two. A message is sent. The counter resets to zero.
3. A third matching event (E3) occurs. The counter increments to one.
No message is sent.
4. The next matching event (E4) occurs more than 30 seconds after the
third event. At 30 seconds, the counter was reset to zero. So the
counter now increments to one. No message is sent.
E1 E2 E3 E4
25s 25s 35s
No message is sent.
Message is sent.
Ex Events that are identical.
No message is sent.
COUNTER_THRESHOLD 2
RESET_COUNTER_INTERVAL 30s
Suppressing Duplicate Messages on the Management Server Suppression of duplicate messages can also be configured for the management server. By suppressing duplicate messages on the management server, you can significantly reduce high system loads caused by large numbers of messages. In addition, you can correlate messages from more than one managed node. The suppression method used by the management server is identical to the Suppress Identical Output Messages method used on the managed node. HPOM compares the message attributes of incoming messages with the message attributes of existing messages. If either message does not have a message key, the following message attributes must be identical:

 

  • Severity
  • Node
  • Application
  • Message group
  • Object
  • Message text
  • Service name

     

    If an identical message already exists, HPOM suppresses the duplicate message and all subsequent duplicate messages. HPOM also starts a counter of duplicates on the first message. This counter is displayed in the browser windows of the responsible operator, as well as in the Message Properties window. The counter gives an indication of how often the problem occurred. The Message Properties window also shows when the last duplicate message was received (and suppressed). If suppression is enabled, HPOM stores information about suppressed duplicate messages in annotations to the first message.

    NOTE Automatic actions started by duplicate messages on the managed node are still executed. Any action responses, however, are lost.

    Enabling Duplicate Message Suppression on the Management Server

    Duplicate message suppression on the management server can be enabled with the opcsrvconfig -dms command. You can also specify that the duplicated messages are added as annotations. For example:

    # opcsrvconfig -dms -enable anno

    NOTE Duplicate message suppression is a global setting that affects all messages and operators.

    For more information, see the opcsrvconfig(1m) manpage. Suppressing Duplicate Messages in Flexible Management Environments

    In flexible MoM environments, each management server is responsible for counting the messages it receives from its direct managed nodes. This means that messages received from other management servers are not aggregated into one message but are displayed as multiple messages, each with the count as received from the originating management server.

    If you do not want a management server to send or receive message count events, set the following variables on the HP Operations management server by using the ovconfchg command-line tool:

    Updating Severity and Message Text of Duplicate Messages

    If an incoming, duplicate message has a different severity or message text than the already existing messages, you can configure HPOM to display the new values instead of the previous data:

    Logging Messages

    The message handling facility of HPOM produces the following types of message results:

    If no filtering occurs, unmatched messages can still be routed into HPOM. Typically, unmatched messages are messages you have not seen before, so you have not yet set up filtering conditions for them. You can choose to log these unmatched messages locally on the managed node, or forward them to the management server for processing. If you forward them to the management server, you can then choose to display them in the Java GUI Message Browser, or to put them directly into the history database. If they are displayed in the browser, they are identified by an X in the U column of the Java GUI Message Browser.

    You can set up logging options after you have defined message source policies and their message and suppress conditions. You specify how you want message types logged by using the following keywords in the policy body:

    LOGMATCHEDMSGCOND
    LOGMATCHEDSUPPRESS
    LOGUNMATCHED
    NOTE If you switch on logging for any of the event correlation policies, logging is automatically enabled for all other event correlation policies.

    After a message has been integrated and filtered by HPOM, you can change its current default message group on the management server. You can customize message groups specifically suited to an operator’s tasks and responsibilities. This means that you are not required to change the conditions for a message source. And you are not required to redistribute the policies to move messages from one message group to another.

    NOTE For information about the use of the service name attribute in regroup conditions, see the Service Navigator Concepts and Configuration Guide. Message conditions and attributes are processed as follows:

    Defining a Regroup Condition

    If you want to define a regroup conditions, you need to use the Administrator’s GUI (CVPL). See the CVPL user documentation available for download in the HP Operations Manager for UNIX directory at: http://support.openview.hp.com/selfsolve/manuals. NOTE If you regroup messages into a message group that does not exist, all messages relating to this message group belong, by default, to the message group Misc until that message group is created. To check the original message group of a message currently assigned to Misc, use the Java GUI Message Properties window of the Java GUI Message Browser.

    Alternatively, you can use the regroup conditions APIs. For more information, see the HPOM Developer’s Reference.

    Examples of Regroup Conditions

    The message conditions shown in earlier examples form the basis for the regroup condition example shown in this section. All messages filtered into HPOM have been forwarded to the message group FINANCE.

    Now you want to split the messages into two groups:

    The regroup conditions for these two groups are listed below: Regroup Condition No. 1
    Application: idris4|idris5
    Application: FINANCE PAYROLL
    Text Pattern: ^***PAYROLL:[ERROR|WARNING]
    New Message Group: payroll
    
    Regroup Condition No. 2
    Application: idris4|idris5
    Application: FINANCE ACCOUNTING
    Text Pattern: ^***ACCOUNTING:[ERROR|WARNING]
    New Message Group: accounting
    Implementing Message Policies
  •  


    Top Visited
    Switchboard
    Latest
    Past week
    Past month

    Old News ;-)

     



    Etc

    FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

    ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

    Society

    Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

    Quotes

    War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

    Bulletin:

    Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

    History:

    Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

    Classic books:

    The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

    Most popular humor pages:

    Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

    The Last but not Least


    Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

    The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

    Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

    FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

    This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

    You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

    Disclaimer:

    The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

    Last modified: August 05, 2013