ArcSight Logger L750MB – Syslog SmartConnector and Snare installation

In my previous blog post, I have detail how to install ArcSight Logger L750MB (the 49$ one) on a Centos 5.x. In this post I will explain you how to install a Windows Syslog ArcSight Connector, how to collect Windows events with Snare and how to push all these events to your Logger.

ArcSight Logger L750MB – network flows

As described in my “ArcSight Logger L750MB features and limits” blog post, this product version of ArcSight Logger has some limitations.

10 devices maximum could send they logs to the L750MB Logger. A device is the IP of the event generator. For example, you could have a Linux RHEL 5 server with Snort, the 2 products will be considered coming from the same source IP. Also, a device on the Logger, is a combination of a event source IP and a “Receiver“. We recommend you to use the same “Receiver” for common products on the same source IP.

With the L750MB version you will be allowed to install SmartConnectors to support these products :

  • Cisco PIX/ASA
  • Cisco IOS Routers and Switches
  • Juniper Network and Security Manager (NSM)
  • Juniper JUNOS Routers and Switches
  • Red Hat Enterprise Linux
  • SNARE
  • Snort

They are 2 provided SmartConnectors in ArcSight Download Center, one to install a Syslog server under Windows, and the other one to install a Syslog server under Linux. All the supported products should send they’re logs to the SmartConnector in Syslog 514/UDP or 514/TCP. When the products logs are received by the ArcSight SmartConnector, these logs are normalized in CEF (Common Event Format) and send encrypted to the Logger in SmartMessage format on port 9000/TCP. ArcSight SmartConnector is not only normalizing logs in CEF, but we will not explain in this blog post all the SmartConnectors capabilities. Here under a small representation of the network flows.

ArcSight Logger L750MB Network Flows
ArcSight Logger L750MB Network Flows

As you know, UDP is not a protocol what we can trust for delivering informations. UDP does not provide guarantee of delivery which can cause data to go missing. When considering connection problems or missing data, the TCP connection is much more desirable. Also, if you need to encrypt the data connection, you should use TCP. So we strongly recommend you to communicate with the ArcSight Syslog SmartConnector in TCP protocol. But the free version of Snare for Windows only support UDP protocol, so we will do this demonstration with UDP. If you want to have TCP support for Snare, you need to buy Snare server.

ArcSight Logger L750MB – Receiver configuration

First of all we need to create a “Receiver” on the Logger. This “Receiver” will be a SmartMessage one, in order to have the possibility to receive events in CEF.

To do this configuration, you only need to identify you on your Logger and to go in the “Configuration” menu, then click on the “Event Input/Output” sub-menu. The receiver will listen on port 9000/TCP, same port as for the Logger Web interface. We will name the receiver “SmartMessageReceiver01“, his type will be “SmartMessage Receiver” and his encoding “UTF8“.

ArcSight Logger L750MB SmartMessage Receiver Configuration part 1
ArcSight Logger L750MB SmartMessage Receiver Configuration

Once saved, you will find your configured receiver in the “Receivers” list. On the right of the receiver, in the receiver list, you need to active it by clicking on extreme right icon. When the receiver will be actif the extreme right icon will be replaced by the following one.

ArcSight Logger L750MB - Receiver startupWe will not pre-configure the devices, cause they will be auto-discovered by the Logger. And we will also not pre-configure a “Devices Group” and a “Storage Rule“.

ArcSight Syslog SmartConnector installation

Now we will install the Syslog SmartConnector on a dedicated box in order to centralize the communication from all the devices and to the Logger. The Syslog SmartConnector will use the TCP protocol. In our example, we will install the Windows Syslog SmartConnector (ArcSight-5.0.2.5703.0-Connector-Downloadable-Logger-Win.exe).

Execute the installation executable, and choose a proper installation folder, we recommend you to not install the SmartConnector on the C drive. Then choose a typical install and the installation executable will install the software and launch the SmartConnector wizard. Choose “ArcSight Logger SmartMessage (encrypted)” as destination and provide all the required informations for the SmartConnector and Logger interconnexion. The “Receiver Name” should be the same as declared on the Logger, here “SmartMessageReceiver01“.

ArcSight Logger L750MB SmartConnector interconnexion configuration
SmartConnector interconnexion configuration

The next screen will indicate you that a Syslog SmartConnector will be installed, and you will have to configure the syslog server by providing the listener IP address (here 192.168.178.66) and the related protocol (here UDP).

Syslog SmartConnector configuration
Syslog SmartConnector configuration

Now give a name (here WinSyslogSC01), a SmartConnector location (here Rack 10-01), a device location (here Rack 11) and a comment (here All rack 11 servers). These informations are optionals.

SmartConnector optional informations
SmartConnector optional informations

You will see in the last screen, that it is possible to reconfigure the SmartConnector with the “bin/runagentsetup.bat” script. Choose now if you want to install the SmartConnector as a standalone application or as a service, and if the service should start automatically.

Syslog SmartConnector as a service
Syslog SmartConnector as a service

The Syslog SmartConnector installation is now finished, but don’t forget to start the Syslog SmartConnector service 🙂

Snare Event Log Agent for Windows installation

Download Snare Event Log Agent for Windows and install it one every Windows server or station you want, but don’t forget that you are limited to 10 devices maximum.

Execute the installation binary, accept the agreement, choose an installation folder, let Snare to take over control of the EventLog configuration, enable the Snare remote control interface with a password and finish Snare installation. Launch “Snare for Windows” from you “Program” menu, you will be connected in the web remote control interface on port 6161/TCP.

Snare web remote control interface
Snare web remote control interface

In the “Network Configuration” menu, configure the “Destination Snare Server” (here 192.168.178.66), the “Destination Port” (here 514),  and click the checkbox for “Enable SYSLOG Header“, the save the configuration.

Snare for Windows configuration
Snare for Windows configuration

To reload the Snare configuration just click on the “Reload Settings” in the “Apply the Latest Audit Configuration“. And here we go, the Windows events are send to the Logger.  For further instructions on how to configure Snare we recommend you to read the Snare documentation 🙂

Windows Events in your Logger

In ArcSight Logger you will now have all the Windows Events directly accessible by the “Analyze” menu and “Search” sub-menu.

ArcSight Logger Windows Events Snare
ArcSight Logger Windows Events Snare

 

 

ArcSight L750MB Logger Centos installation

Since ArcSight Protect 2010 in September 2010, the Logger model L750MB has been integrated in the ArcSight Logger product catalog. In our previous blog post we have analyse the Logger L750MB features and limits. We will resume here some of the features and limitations provided by L750MB and provide an installation guide for Centos 5.x

Features :

Connector appliance features are disabled.
Alerting module features are enabled.
Reporting module features are enabled.
SAN storage feature is disabled.
Logger peering features are disabled.

Limits :

10 devices maximum supported.
Maximum number of daily collected data is 750 MB
EPS rate is limited to a maximum of 60
maximum data retention is 50 GB

OS & Hardware requirements

You can install L750MB Logger on these following certified operating systems :

Red Hat Enterprise Linux (RHEL), version 5.4, 64-bit
Oracle Enterprise Linux (OEL) 5.4, 64-bit
CentOS, version 5.4, 64-bit

or on these others supported operating systems :

Red Hat Enterprise Linux (RHEL), version 4.x, 64-bit
CentOS, version 4.x, 64-bit

Virtual Machine installation of the above listed OS is supported. You will not able to install the Logger on an existing machine how is running MySQL or PostgreSQL. We recommend you a complete dedicated operating system for the installation. You will also need a synchronized NTP for all your infrastructure. A synchronized time is a key factor for Log Management. After the installation you will need one of the following supported browser, with Adobe Flash Player plug-in :

Internet Explorer: Versions 7 and 8
Firefox: Versions 3.0 and 3.5

For hardware requirements we recommend you :

CPU : 1 or 2 Core
Memory : 4 – 12 GB
Disk Space : 120 GB

Storage strategy & retention policy requirements.

As ArcSight Logger L750MB has a limit of 50GB maximum data retention, your storage strategy and retention policy will be simple to define, just follow the ArcSight recommended installation,  and then we will change it by the ArcSight Logger Web interface.

By the recommended installation ArcSight Logger will initialize the Storage Volume to the maximum authorized, aka 50 GB, and the Storage Volume has to be on local disk, on a NFS, or SAN mount point. You will not be able to increase the size of the Storage Volume above 50GB with the L750MB, and once the Storage Volume size is configured the only way to resize the Storage Volume is to reinstall every thing.

Also, with the recommended installation ArcSight Logger will initialize the maximum of 6 Storage Groups. Two of these Storage Groups are inherent to the Logger and are named “Default Storage Group” and “Internal Event Storage Group“. if you choose to not create the maximum of 6 Storage Groups, you will not further able to create more Storage Groups. Here under the default Storage Groups configuration :

[TABLE=14]

You will be able to resize all Storage Groups, we recommend you to, until you understand the concept of “Devices”, “Device Groups” and “Storage Rules”, to not touch the “Internal Event Storage Group” definition and to provided the maximum size to the “Default Storage Group“. You will have then this configuration :

[TABLE=15]

Installation

First of all you will need an updated Centos 5.4 installation, just follow the Centos installation procedures. You will need to configure IP addresses, DNS and NTP configuration before starting the Logger installation procedure. As ArcSight Logger

Create an arcsight user and group :

ArcSight user add
ArcSight user add

Give a password to the arcsight user :

ArcSight user password
ArcSight user password

Upload “ArcSight-logger-5.0.0.5355.2.bin” installation binary and your “arcsight_logger_license.lic” license file in the arcsight home directory.

Make the installation binary executable :

ArcSight Logger chmod
ArcSight Logger chmod

To install the Logger in console mode execute the following command :

ArcSight Logger Console mode installation
ArcSight Logger Console mode installation

On the first prompt press enter to display the license agreement and accept the terms of agreement.

ArcSight Logger licence agreement
ArcSight Logger licence agreement

Provide the installation directory, in “/home/arcsight”, and then press enter to begin the continue the installation.

ArcSight Logger installation folder
ArcSight Logger installation folder
ArcSight Logger installation
ArcSight Logger installation

After the end of the installation, you will need to press “enter” to initialize the Logger. This initialization may take several minutes.

ArcSight Logger initialization
ArcSight Logger initialization
ArcSight Logger successful initialization
ArcSight Logger successful initialization

When initialization is done you will have to configure the Logger, by a configuration wizard. To start this wizard in console mode, please type the following command.

ArcSight Logger configuration
ArcSight Logger configuration
ArcSight Logger configuration in console mode
ArcSight Logger configuration in console mode

The license file location will be asked.

ArcSight Logger License file
ArcSight Logger License file

Choose the typical installation type if you are not familiar with ArcSight Logger indexing, storage groups, and storage volume. Also don’t forget that the L750MB will not permit you to go above a theoretically 50GB storage. As described above we will change to Storage Groups settings further.

ArcSight Logger installation type
ArcSight Logger installation type

When the complete configuration is finished we recommend you to not start directly the logger and reboot the server.

ArcSight Logger installation startup
ArcSight Logger installation startup

After the reboot log you on the server with the arcsight user to start the logger with the following commands.

ArcSight Logger startup after reboot
ArcSight Logger startup after reboot

The “loggerd” command is located in “/home/arcsight/current/arcsight/logger/bin” directory. If the startup is successful you will have this return.

ArcSight Logger loggerd status
ArcSight Logger loggerd status

The “loggerd” command can have these following arguments.

ArcSight Logger loggerd arguments
ArcSight Logger loggerd arguments

Now you can log in ArcSight Logger Web interface on port 9000 with https and you will have the following login page.

ArcSight Logger login page
ArcSight Logger login page

The default login is “admin“, and the default password is “password“, please change it 🙂 To change your password just go in the “System Admin” menu, then in the “Change Password” sub-menu.

ArcSight Logger Web user interface
ArcSight Logger Web user interface

To change the Storage Groups settings just go in the “Configuration” menu, then in the “Storage” sub-menu.

You have now an up and running logger, in a next blog post we will install the L750MB SYSLOG SmartConnector on a dedicated Linux server and the “SNARE” software on Windows to have  our first events.

ArcSight L750MB Logger features and limits

ArcSight L750MB Logger is now for free, since 16 August . A good occasion to discover this Logger version and to better know the provided features and limits of this product.

After the registration, no CCN is asked, with the promotional code, you will receive two separate emails. One will give you access to the ArcSight Download Center, and one hour latter you will receive your free licence key attached to the second email. Hopefully since, two or three weeks registrations are also allowed for users how are outside US & Canada.

From the ArcSight Download Center, you will be able first to download :

  • The latest version of ArcSight L750MB Logger – 5.0 Patch 2 (5.0.0.5355.2) – 438.8 MB.
  • All L750MB related documentations (Administrator Guide, Quick Start, Release Notes & the Software Licence Agreement).
  • Limited ArcSight Syslog SmartConnectors for Linux and Windows (229.4 MB for Linux and 187 MB for Windows).
  • All the limited Syslog SmartConnectors documentations.

Supported Plateforms & Browsers

Certified Operating Systems for ArcSight L750MB Logger installation are :

  • Red Hat Enterprise Linux (RHEL), version 5.4, 64-bit
  • Oracle Enterprise Linux (OEL) 5.4, 64-bit
  • CentOS, version 5.4, 64-bit

Other supported Operating Systems are :

  • Red Hat Enterprise Linux (RHEL), version 4.x, 64-bit
  • CentOS, version 4.x, 64-bit

Virtual Machine installation of the above listed OS is supported. You will not able to install the Logger on an existing machine how is running MySQL or PostgreSQL. We recommend you a complete dedicated Operating System for the installation.

Supported browsers are, with Adobe Flash Player plug-in :

  • Internet Explorer: Versions 7 and 8
  • Firefox: Versions 3.0 and 3.5

For the Hardware requirement :

  • 100 GB disk space will be enough cause the L750MB Logger has a 50 GB maximum compressed log (10:1) restriction.
  • Minimum of 2 GB memory, but better 4 GB to gain the advantages of the 64 bits OS.
  • 1 to 2 CPU cores are enough due to others L750MB limitations.

L750MB provided SmartConnectors

SmartConnectors will provide you the ability to send CEF normalized and aggregated events to the Logger. To use SmartConnectors with ArcSight Logger you need to create a “Smart Message Receiver” on the Logger.

I was surprised to see that the SmartConnectors provided with the L750MB Logger version are very limited in term of number. Normally ArcSight cover with his SmartConnectors technology around 250 different products. With L750MB you are able to install and use the following SmartConnectors :

  • Cisco PIX/ASA (an average of 200 bytes per event)
  • Cisco IOS Routers and Switches (an average of 150 bytes per event)
  • Juniper Network and Security Manager (NSM) (an average of 300 bytes per event)
  • Juniper JUNOS Routers and Switches (an average of 300 bytes per event)
  • Red Hat Enterprise Linux (an average of 150 bytes per event)
  • SNARE (an average of 800 bytes per event, depending on the Windows OS)
  • Snort (an average of 200 bytes per event)

In addition or without SmartConnectors you could use the following others Logger Receivers :

  • Syslog (UDP or TCP)
  • File Transfer in SCP, SFTP or FTP.
  • CEF TCP or UDP

The File Receiver is disabled cause the L750MB Logger don’t allow you to mount NFS, CIFS or SAN shares.

L750MB Features, Limitations and Restrictions

Features :

  • Connector appliance features are disabled.
  • Alerting module features are enabled.
  • Reporting module features are enabled.
  • SAN storage feature is disabled.
  • Logger peering features are disabled.

Logger limits :

  • 10 devices maximum could send they logs to the L750MB Logger. A device is the IP of the event generator. For example, you could have a Linux box with Syslog and Snort, the 2 events sources will be considered coming from the same generator IP. Also, the device, on the Logger, is a combination of a generator IP and a Receiver. We recommend you to use the same Receiver for common events sources on the same generator IP.
  • The maximum number of daily collected data is 750 MB. The sum of the size of the original events is used to determine this value. 750 MB represent, with a 300 bytes event average size, 2 621 440 events per day ((750 * 1024 * 1024) / 300).
  • The EPS rate is limited to a maximum of 60. So you will have a maximum of 5 184 000 events per day (60 * 86 400). But the 750 MB limit will stop you before you will reach the 60 EPS if you are running SmartConnectors like SNARE or Juniper.
  • The maximum data retention is 50 GB with a compression rate of 10:1 (500 GB). But the Logger need an “Internal Event Storage Group” of 5 GB, so your total of maximum data retention is more less than 45 GB (450 GB).  This compression rate will permit you to store around 1 610 612 736 events ((450 x 1024 x 1024 x 1024) / 300). This total amount of events compared to the 750 MB limit will permit you to have a retention period of 614,4 day’s (1 610 612 736 / 2 621 440).

If the limit of 750 MB per day is exceeded, the software version of Logger continues to collect and store events. However, if this limit is exceeded 5 times (5 days) in a 30 day sliding windows, you will no more able to run “Searches” or run “Reports” on the collected events until the 30 day sliding window contains 4 or less data limit violations. A warning message will be displayed when a data limit violation occurs. You can also view the data limit violation information on the License information page.

ArcSight L750MB Logger is really a good solution for SMB companies how don’t have necessary lot of devices, but the limitation in supported SmartConnectors is a point how will maybe encourage IT guys to select a product with more native supported products.

Thanks for Christophe Briguet for its Log Caliper Iphone/Ipad application 🙂

ArcSight annonce confusing L5GB, or L750MB, Log Management solution

ArcSight Inc. has annonce the release of a new version of his Log Management solution, version 5.0 of ArcSight Logger. The entry price for this update is announced at 49$.

In the “ArcSight Logger – Universal Log Management Products Descriptions” section of ArcSight Web site, we have two tabs one “DOWNLOAD” and one “SPECS“.

In the “DOWNLOAD” tab, the new product is named L750MB, and has a limit of 750 MB of logs per day, with a total searchable space of 50 GB.

In the “SPECS” tab, the another product, or the same product, is named L5GB, and has a limit of 5G of logs per day, with a maximum of 50 devices connected to the Log Management solution. No total searchable space is given for the L5GB Logger.

With an average compression rate of 10:1(dependent on data type and data source), the L750MB has the following data’s :

  • 750 MB of daily raw logs represent 75 MB daily compressed logs.
  • With a total searchable space of 50 GB, the log retention policy could be theoretically be 682 days.

In the ArcSight Logger “Product Brief” PDF. They’re is no motion of the L5G product, but only the L750MB one. The characteristic’s, in the “Product Brief” PDF file, for the L750MB are :

  • Daily Limit on Log Data : 750 MB.
  • Total searchable Space (Compressed) : 50 GB.

In the 49$ product “Terms and Conditions”, the mentioned product is the L750MB, no trace of the L5G.

  • Daily Limit on Log Data : 750MB – 3.3 (viii)
  • Total searchable Space (Compressed) : 10 GB3.3 (ix)
  • 10 devices max. – 3.3 (x)
  • 2 physical CPU max (2 dual cores, or 2 quad cores, or 2 etc.) – 3.3 (xi)

With 10 GB total searchable space (compressed) the retention policy is downgraded from 682 days to 136 days ! We can also discover new restrictions for devices and CPU.

3.3. Restrictions
(viii) process more than 750MB of incoming data per day with the Software;
(ix) use the Software to store more than 10GB of data;
(x) use the Software with more than 10 devices,

Also, what is interesting in the “Terms and Conditions” is the title of the document “THE ONE-YEAR TRIAL VERSION OF LOGGER“. Is the 49$ product an one year trial ?

Another fact how is interesting in the “Terms and Conditions”, is the restriction “3.3 (ii)“. As MSSP you don’t have the possibility to distribute, sell, sub license, rent, lease the product.
But in the press release, ArcSight has target the SMB market with this new solution. Is this product not usable by MSSP ?

The new ArcSight offer seem to be quiet interesting, but clearly confusing. Maybe the rush of ArcSight Protect 10 is the cause of all these confusions.