Category Archives: Log Management

Log management comprises an approach to dealing with large volumes of computer-generated log messages (also known as audit records, audit trails, event-logs, etc.). Log management covers log collection, centralized aggregation, long-term retention, log analysis (in real-time and in bulk after storage) as well as log search and reporting. Log management is driven by reasons of security, system and network operations (such as system or network administration) and regulatory compliance. Effectively analyzing large volumes of diverse logs can pose many challenges — such as huge log-volumes (reaching hundreds of gigabytes of data per day for a large organization), log-format diversity, undocumented proprietary log-formats (that resist analysis) as well as the presence of false log records in some types of logs (such as intrusion-detection logs). Wikipedia definition.

ArcSight Logger L750MB now for free !

Log Management,

Good news for the Log Management market, ArcSight is now offering his downloadable software Logger L750MB version for free ! You don’t have to pay 49$ per year to install and fully use this Log Management solution. You can download this version from ArcSight website, but downloadable version can only be downloaded from the following countries: Australia, Belgium, Canada, Denmark, Finland, France, Germany, Hong Kong, India, Israel, Italy, Kuwait, Malaysia, Netherlands, Norway, Poland, Russia, Saudi Arabia, Singapore, South Africa, Spain, Sweden, Taiwan, Thailand, Turkey, United Arab Emirates, United Kingdom and United States.

If you want to install and play with ArcSight Logger L750MB, I have write some blogposts to help you :

ArcSight SmartConnectors silent mass upgrade

Event Management, Log Management,

Since the Jun 2, ArcSight has release a new version of the free ArcSight Logger L750MB (5.1.0.5887.0) and for related SmartConnectors (5.1.3.5875.0). You can download these updates from ArcSight Download Center.

In my previous blogpost we have document on how silently mass install SmartConnectors, in this new blogpost we will see on how upgrade these SmartConnectors also silently. This blogpost is only applicable if you have standalone SmartConnectors, not connected to a Connectors Appliance, L3x00 Logger Appliance serie (whow integrate a Connectors Appliance) or directly connected to ArcSight ESM SIEM.

The previous installed SmartConnectors version was 5.0.2.5703.0, you can check your SmartConnector version by executing this command.

First of all you need to completely install, for example, a Windows Syslog SmartConnector as described in my previous “Syslog SmartConnector and Snare installation“.

After the SmartConnector 5.0.2.5703.0 installation start to install the 5.1.0.5887.0 version. You will see this following screen, just click “OK” to continue.

Later during the installation process, if you see the following screen just click on the “Cancel” button.

Open a command prompt and go to the SmartConnector installation directory (ex : C:\Program Files\ArcSightSmartConnectors) and execute the following command.

The “recorderui” start option will allow you to record the installation process in order to create a mass installation properties configuration file.

The installation wizard will propose you to select a “Silent Properties File Name” and a typical “Installation Target Folder“.

Create an “installer.properties” file on your desk and select it in the wizard. Also select a default installation folder, for example “D:\ArcSightSmartConnectors“. Now you can continue your typical SmartConnector configuration.

When you see this screen, check the “I do not want to change any settings” as described in the following screenshot.

Finish the setup, and you can examine your “installer.properties” file to adapt the properties with your needs.

Adapt the “USER_INSTALL_DIR” and “ARCSIGHT_AGENTSETUP_PROPERTIES” variables to your needs.

Now you can upgrade all your SmartConnectors in silent mode. If a properties file named either “installer.properties” reside in the same directory as the installer, it will automatically be used, overriding all other command line options, unless the “-f” option is used to point to another valid properties file.

The “-f” option can be used by following command line.

Free Novell Sentinel Log Manager 25 features and limits

Log Management,

Novell provide a free version of they Log Management solution named Novell Sentinel Log Manager 25.This version is not a trial version. A good occasion to discover this Log Management solution and to better know the provided features and limits of this product. You only need to register on the Novell Website in order to be allowed to download the free version.

From the Novell Download Center, you will be able to download :

  • The latest version of Novell Sentinel Log Manager as ISO, VMWare or Xen image.
  • All Sentinel Collectors how permit you to parse the data from a variety of event sources into the normalized Sentinel event structure.
  • All Sentinel Connectors how permit you to facilitate the connectivity between Sentinel Collectors and event or data sources.
  • Solution Packs, such as “Identity Tracking“, “PCI-DSS“, “SAP” or “Sentinel Core“.
  • Actions Plug-ins how are remediation actions that can be triggered by correlation rules, such as “Generic Event Forwarder“, “Create Remedy Ticket“, etc.
  • Integrators how provide integration with external third-party products that can be easily called from Action Plug-ins, such as “Remedy“, “SMTP“, “SOAP“, etc.
  • Utilities how provide additional functionality for Sentinel and/or are commonly used by several other Plug-ins.
  • The documentations are available online or in PDF format.

Supported Platforms & Browsers

Supported Operating Systems for Novell Sentinel Log Manager installation are :

  • 64-bit SUSE Linux Enterprise Server 11 SP1 for Sentinel Log Manager versions 1.2 and later.
Supported Operating Systems for Collector Managers are :
  • SUSE Linux Enterprise Server 10 SP2 (32-bit and 64-bit)
  • SUSE Linux Enterprise Server 11 (32-bit and 64-bit)
  • SUSE Linux Enterprise Server 11 SP1 (32-bit and 64-bit) for Collector Managers 1.2 and later.
  • Windows Server 2003 (32-bit and 64-bit)
  • Windows Server 2003 SP2 (32-bit and 64-bit)
  • Windows Server 2003 R2 (32-bit and 64-bit)
  • Windows Server 2008 (64-bit)
  • Windows Server 2008 R2 (64-bit)
Supported Virtual Environment are :
  • VMWare ESX/ESXi 3.5/4.0 or higher
  • VMPlayer 3 (for demo only)
  • Xen 3.1.1
I have test this Log Management solution with VMWare Fusion 3.
Supported browsers are :
  • Mozilla Firefox 3.6 for Linux
  • Mozilla Firefox 3 for Windows
  • Microsoft Internet Explorer 8
For Hardware requirement the VMWare and Xen version are pre-configured with :
  • 4 GB RAM, cause SLES is a 64-bit version
  • 48 GB hard disk
  • 1 VCPU

Sentinel Collector Managers

The Collector Managers manage all data collection and data parsing for Novell Sentinel Log Manager. One Collector Manager is installed by default during the Sentinel Log Manager installation. However, you can install multiple Collector Managers in a distributed setup, but take care on the 25 EPS limits. A Collector Manager is communicating with the Novell Sentinel Log Manager on port 61616/TCP. All installed Collectors and Connectors will communicate with the Collector Manager and forward the collected data to Novell Sentinel Log Manager for storage and processing.

Sentinel Connectors

Connectors facilitate the connectivity between Sentinel Collectors and event or data sources. All Collectors should communicate with a determined Connector. Example of Connectors : Novell Audit, Database, File, LEA, IBM Mainframe, Process, Syslog, SNMP, Windows WMI, etc. You can find a complete list of all supported Connectors by registering on Novell Website.

Sentinel Collectors

Collectors are scripts that parse the data from a variety of event sources into the normalized Sentinel event structure, or in some cases collect other forms of data from external data sources. Each Collector should be deployed with a compatible Connector. Novell support around 100 collectors such as Check Point, Cisco, Apache, Microsoft, McAffee, Juniper, etc. All of them are free to use with the 25 EPS version of Novell Sentinel Log Manager. You can find a complete list of all supported Collectors by registering on Novell Website.

Novell Sentinel Log Manager 25 Features, Limitations and Restrictions

Features :

  • Compression rate of 10:1
  • Unlimited number of Collectors. Allows you to use any Collector without any restriction
  • Unlimited EPS Limited Event Store. Provides the ability to persist with the event flow even if the event view is restricted because of the expired or not licensed Event Store feature.
  • Unlimited Embedded Database. Stores the Sentinel Log Manager configuration data.
  • Unlimited usage of the reporting module.
  • Authorized network storage (NFS, CIFS, locally mounted SAN).

Limits :

  • EPS rate limited to a maximum of 25  after 60 days, unlimited before. So you will have a maximum of 2 160 000 events per day (25 * 86 400).
  • With the VMWare or Xen appliance, 48 GB hard disk, how represent 480 GB with a 10:1 compression rate. The storage and the compression rate will permit you to store around 1 717 986 918 events ((480 x 1024 x 1024 x 1024) / 300). If you are fully using the 25 EPS rate, you will have a maximum retention of 795,36 days.
  • Rules limited to 60 days. You can configure rules to filter events based on one or more of the searchable fields. Each rule can be associated with one or more of the configured actions. After the free license expiration the Rules are no more executed.
  • Actions limited to 60 days. You can configure actions to deliver an event to one or more actions when it meets the criteria specified by one of the rules (ex: Email, Log to file, Log to Syslog, Send to Sentinel Link). After the free license expiration the Actions are no more executed.
  • Distributed Searches limited to 60 days. Enables you to search events and report event data not only on your local Sentinel Log Manager server, but also on other Sentinel Log Manager servers distributed across the globe. After the free license expiration you can still add, modify, and delete the search target configurations. However, only the local event store is used for searches and reports while you are using an expired license. This applies to all distributed searches and reports, even if they were scheduled before the license expired.
  • Event Store limited to 60 days. Enables you to view the details of all the events regardless of the EPS rate. The authorized event rate after the license expires is 25 EPS. Any events received while the system averages more than 25 EPS are stored, but the details of those events are not displayed in the search results or reports. These events are tagged with the OverEPSLimit tag.
  • Sentinel Link limited to 60 days. Provides the ability to hierarchically link multiple Sentinel systems, including Sentinel Log Manager and the two Sentinel Security Information Event Management (SIEM) systems, Novell Sentinel and Novell Sentinel Rapid Deployment systems. After the free license expiration the events cannot be sent or received by using Sentinel Link.
Novell Sentinel Log Manager 25 is really a good solution for SMB companies how don’t require a big EPS throughput. Maybe your Log Management scope will not be complete, but you can select different sources du to the available number of Sentinel Collectors.

ArcSight SmartConnectors silent mass installation

Event Management, Log Management,

With your free ArcSight L750MB Logger you can mass install ArcSight SmartConnectors with a silent properties configuration file. If you have to install, for example, 10 or more Syslog SmartConnectors, you will win time by reading this blog post.

First of all you need to create a properties configuration file template by installing a typical SmartConnector, with typical settings. Just start to install, for example, a Windows Syslog SmartConnector as described in my previous “Syslog SmartConnector and Snare installation“.

During the installation process, if you see the following screen just click on the “Cancel” button.

Open a command prompt and go to the SmartConnector installation directory (ex : C:\Program Files\ArcSightSmartConnectors) and execute the following command.

The “recorderui” start option will allow you to record the installation process in order to create a mass installation properties configuration file.

The installation wizard will propose you to select a “Silent Properties File Name” and a typical “Installation Target Folder“.

Create an “installer.properties” file on your desk and select it in the wizard. Also select a default installation folder, for example “D:\ArcSightSmartConnectors“. Now you can continue your typical SmartConnector configuration.

You can examine your “installer.properties” to adapt the properties with your needs.

For each SmartConnector you have to install you need to adapt into your “installer.properties” file :

– The SmartConnector name : AgentDetailsPanel.agentname
– The optional SmartConnector location : AgentDetailsPanel.agentlocation
– The optional device location : AgentDetailsPanel.devicelocation
– The optional comment : AgentDetailsPanel.comment

Now you can install all your SmartConnectors in silent mode. If a properties file named either “installer.properties” reside in the same directory as the installer, it will automatically be used, overriding all other command line options, unless the “-f” option is used to point to another valid properties file.

The “-f” option can be used by following command line.