lhcblogo.gif (8544 bytes)

Experiment Control System

lbLogErrHandler and Central Log Viewer



The lbLogErrHandler is one of the basic components in the LHCb framework which has to be installed in every PVSS control project. It provides two main funtionalities. The first one consists in publishing the PVSS log messages via DIM and the second one is storing this messages into an Oracle database. (This second functionality is not yet set up!)
In parallel to this component, there is a Central Log Viewer which collects and displays all published log messages and allows to query the error messages stored in the database.


lbLogErrHandler


Installation

The installation of the component has been done centrally by the online group. To check if the component is already installed, look for the manager PVSS00DimErrInfo in the project console.

The package is also available in the common software/pvss area so it can be reinstalled at any time.
In this case,

  1. Use the fwInstallation tool to install the components
    • Select the place to "Look for new components":
      - /sw/pvss/fwLogErr/lbLogErrHandler_0.5.2.1
    • Check the installation directory:
      - /group/<group_id>/pvss/fwComponents_<projName>
  2. Restart the project when the installation procedure asks for it (needed!).
    Remark: A new manager PVSS00DimErrInfo should have been added (normally at the end of the console).
  3. When the project is running again, make sure that the manager PVSS00DimErrInfo is running and check with the DIM browser that the server is also running (LHCb_<systemName>_ERRORINFO).

Please be aware that it is intend to be used only on the production projects.


Start using

During installation of the component, a new manager is added to the pvss console of the project, PVSSDimErrInfo. This manager acts as a DIM server and publishes the pvss error messages as services via DIM. Each server will be named LHCb_<systemName>_ERRORINFO. You can check this with the DID browser pointing to DIM_DNS_NODE=ecs03.

By default, the log messages printed with the pvss function DebugN are not published (because they are not PVSS error messages). Nevertheless, for those messages which are interesting to understand the overall behaviour of the system, the component provides a function to publish them: lbErr_send.
The idea is that users go through their code (panels, libs, etc.) and make a replace between DebugN and this funtion (while the messages will still be printed in the local Log Viewer); but keep in mind that this should only be done for those messages relevants from a run point of view.
On the other hand, the DebugN function can always be used for debugging purposes, since the usual PVSS Log Viewer will still be available.

    DebugN("No RPC_in received. Check DIM communication");
    lbErr_send("No RPC_in received. Check DIM communication");

In addition to the message to report, this function accepts two optional parameters:

    lbErr_send(msg, PRIORITY, note);

The priority by default is info, but can be chosen between info, warning or error: lbINFO, lbWARNING, lbERROR respectively.
For more information about priority use cases and LHCb convention in terms of error reporting, please refer to the document LHCb ECS Error Reporting Guidelines

The messages generated with this mechanism, will content a prefix indicating the sub-system the message comes from (e.g. OT, TFC, LHC, etc). This is usefull for the advanced filtering on the Central Log Viewer side, which will be presented in the next section.

There is a second function available which allows the user to use an specific error-message catalog, it means, a catalog written by the user. In this case, note that the error code and prefix convention should be followed.

And do not forget to include the line #uses "lbLogErrHandler.ctl" in scripts, libs, etc.

NOTE: there is a known bug being investigated: only an even number of resarts makes the manager work (if you stop and start the manager, it will not work. You have to restart it again).


Central Log Viewer


The PVSS Log Viewer is the place where the control messages are collected. Every PVSS system collects and stores its messages locally, so that an expert needs to log into every machine to check and correlate this messages.
There is a new tool which collects and shows all the PVSS error messages coming from the distributed control systems: the Central Log Viewer. It acts as a DIM client subscribing to the services published by the PVSSDimErrInfo manager.

Different instances of the Central Log Viewer can run at a time, being configured with different settings. Like this, sub-systems can run its own Central Log Viewer and configure it for its own purposes. E.g. the RICH expert may want to see all messages coming from a RICH project but not from any other subdetector. The shortcuts for sub-detectors (VELO, IT, TT, OT, RICH1, RICH2, CALO, ECAL, HCAL, PRS, MUON, TPU) have been created and can be found at:

    /group/online/ecs/pvss/Shortcuts38/<SD>/centralLog

Messages can be filtered by manager type, severity, timestamp, etc. This can be done online (by clicking the specific options) or applying predefined advanced filters which can be edited on the advance tab.

In the Control Room, runs a Central Log Viewer configured to show only user defined messages (the ones sent with the lbError_send function):
G:\online\ecs\Shortcuts38\OTHER\centralLog\lbCentralLogViewer_shift.bat
This tool should help the shift crew to follow and understand any eventuality related with a run controls.




Presentation about the fwLogErrHander is available here.

For any component related report please contact Alba Sambade Varela.