Tag Archives: ArcSight

ArcSight SmartConnector commands and features

03/06/2011Event Management, Log ManagementArcSight, Logger, SmartConnectorwow

If you have download for free the ArcSight Logger L750MB version, follow the installation guideline under Centos and install Windows Snare with ArcSight Syslog SmartConnector, you have now an operational lab or production environment. In this post we will describe you some SmartConnector commands and features. These commands and features are not documented in the provided ArcSight Logger L750MB documentation.

Starting the SmartConnector

If you don’t have specify, during the setup, to run the SmartConnector as a service, you will ne to start it manually. To start the SmartConnector you need to go in the “$ARCSIGHT_HOME/current/bin” directory and execute the “arcsight” script for Linux, or “arcsight.bat” script under Windows, with the following argument.

Once started, to confirm that the SmartConnector is working properly, you will have to check these outputs.

ArcSight SmartConnector starting outputs

“Eps” will give you the actual EPS throughput, and “Evts” the total number of events how have been processed by the SmartConnector. “ET” and “HT” should have twice the “Up” value in order to validate the the SmartConnector connexion with the Logger is working properly. If they are any communication troubles between the SmartConnector and the Logger you will have these kind of outputs.

ArcSight SmartConnector and Logger communication troubles

Checking SmartConnector availability

To valide that the SmartConnector is up and running, you can use the following command.

If the SmartConnector is down, you will have this result.

This command will not validate that the communication between the SmartConnector and the Logger is up and running.

Restarting the SmartConnector

To restart the SmartConnector you will have to use the following command.

Stopping the SmartConnector

If you have start the SmartConnector in the standalone mode, a simple CTRL+C will terminate the activities. But you can also stop the activities with the following command :

Checking SmartConnector status

To check the complete SmartConnector status use the following command.

The output will provide you some useful informations about the SmartConnector activities (memory usage, agent type, agent version, processed events, EPS, last event processed date, etc.)

Checking SmartConnector DNS resolution

To verify that the SmartConnector is able to do DNS resolution you can execute the following command.

ArcSight Agent FlexAgent Regex Tester

ArcSight provide, with the SmartConnector, a tool how will permit you to create and test regex for your logs. SmartConnectors delivered for ArcSight Logger L750MB will not allow you to add FlexConnectors (custom agents), but you can still use this regex GUI for personal purposes. To start the regex GUI execute the following command.

For example, I have test the regex tool, with the following postfix log entry.

May 12 04:14:13 logger sendmail[3457]: p4C2EDU2003456: to=<[email protected]>, ctladdr=<[email protected]> (0/0), delay=00:00:00, xdelay=00:00:00, mailer=local, pri=31483, dsn=2.0.0, stat=Sent

The regex tester will provide you a solution on how to parse this log.

Download ArcSight Logger for free and get your Love Thy Logs t-shirt

24/04/2011Log ManagementArcSight, Loggerwow

ArcSight propose you, until first June 2011, to download for free it’s ArcSight Logger L750MB version. If you register the free version of ArcSight Logger L750MB with the “FreeLogger4WebA” promo code, you will also receive a “Love they logs” t-shirt.

We have several blog posts how could help you install and configure your free ArcSight Logger.

ArcSight Logger L750MB – Syslog SmartConnector and Snare installation

14/04/2011Event Management, Log ManagementArcSight, Logger, SmartConnector, SNARE, Syslogwow

In my previous blog post, I have detail how to install ArcSight Logger L750MB (the 49$ one) on a Centos 5.x. In this post I will explain you how to install a Windows Syslog ArcSight Connector, how to collect Windows events with Snare and how to push all these events to your Logger.

ArcSight Logger L750MB – network flows

As described in my “ArcSight Logger L750MB features and limits” blog post, this product version of ArcSight Logger has some limitations.

10 devices maximum could send they logs to the L750MB Logger. A device is the IP of the event generator. For example, you could have a Linux RHEL 5 server with Snort, the 2 products will be considered coming from the same source IP. Also, a device on the Logger, is a combination of a event source IP and a “Receiver“. We recommend you to use the same “Receiver” for common products on the same source IP.

With the L750MB version you will be allowed to install SmartConnectors to support these products :

Cisco PIX/ASA
Cisco IOS Routers and Switches
Juniper Network and Security Manager (NSM)
Juniper JUNOS Routers and Switches
Red Hat Enterprise Linux
SNARE
Snort

They are 2 provided SmartConnectors in ArcSight Download Center, one to install a Syslog server under Windows, and the other one to install a Syslog server under Linux. All the supported products should send they’re logs to the SmartConnector in Syslog 514/UDP or 514/TCP. When the products logs are received by the ArcSight SmartConnector, these logs are normalized in CEF (Common Event Format) and send encrypted to the Logger in SmartMessage format on port 9000/TCP. ArcSight SmartConnector is not only normalizing logs in CEF, but we will not explain in this blog post all the SmartConnectors capabilities. Here under a small representation of the network flows.

As you know, UDP is not a protocol what we can trust for delivering informations. UDP does not provide guarantee of delivery which can cause data to go missing. When considering connection problems or missing data, the TCP connection is much more desirable. Also, if you need to encrypt the data connection, you should use TCP. So we strongly recommend you to communicate with the ArcSight Syslog SmartConnector in TCP protocol. But the free version of Snare for Windows only support UDP protocol, so we will do this demonstration with UDP. If you want to have TCP support for Snare, you need to buy Snare server.

ArcSight Logger L750MB – Receiver configuration

First of all we need to create a “Receiver” on the Logger. This “Receiver” will be a SmartMessage one, in order to have the possibility to receive events in CEF.

To do this configuration, you only need to identify you on your Logger and to go in the “Configuration” menu, then click on the “Event Input/Output” sub-menu. The receiver will listen on port 9000/TCP, same port as for the Logger Web interface. We will name the receiver “SmartMessageReceiver01“, his type will be “SmartMessage Receiver” and his encoding “UTF8“.

ArcSight Logger L750MB SmartMessage Receiver Configuration part 1 — ArcSight Logger L750MB SmartMessage Receiver Configuration

Once saved, you will find your configured receiver in the “Receivers” list. On the right of the receiver, in the receiver list, you need to active it by clicking on extreme right icon. When the receiver will be actif the extreme right icon will be replaced by the following one.

ArcSight Logger L750MB - Receiver startup We will not pre-configure the devices, cause they will be auto-discovered by the Logger. And we will also not pre-configure a “Devices Group” and a “Storage Rule“.

ArcSight Syslog SmartConnector installation

Now we will install the Syslog SmartConnector on a dedicated box in order to centralize the communication from all the devices and to the Logger. The Syslog SmartConnector will use the TCP protocol. In our example, we will install the Windows Syslog SmartConnector (ArcSight-5.0.2.5703.0-Connector-Downloadable-Logger-Win.exe).

Execute the installation executable, and choose a proper installation folder, we recommend you to not install the SmartConnector on the C drive. Then choose a typical install and the installation executable will install the software and launch the SmartConnector wizard. Choose “ArcSight Logger SmartMessage (encrypted)” as destination and provide all the required informations for the SmartConnector and Logger interconnexion. The “Receiver Name” should be the same as declared on the Logger, here “SmartMessageReceiver01“.

ArcSight Logger L750MB SmartConnector interconnexion configuration — SmartConnector interconnexion configuration

The next screen will indicate you that a Syslog SmartConnector will be installed, and you will have to configure the syslog server by providing the listener IP address (here 192.168.178.66) and the related protocol (here UDP).

Now give a name (here WinSyslogSC01), a SmartConnector location (here Rack 10-01), a device location (here Rack 11) and a comment (here All rack 11 servers). These informations are optionals.

You will see in the last screen, that it is possible to reconfigure the SmartConnector with the “bin/runagentsetup.bat” script. Choose now if you want to install the SmartConnector as a standalone application or as a service, and if the service should start automatically.

The Syslog SmartConnector installation is now finished, but don’t forget to start the Syslog SmartConnector service 🙂

Snare Event Log Agent for Windows installation

Download Snare Event Log Agent for Windows and install it one every Windows server or station you want, but don’t forget that you are limited to 10 devices maximum.

Execute the installation binary, accept the agreement, choose an installation folder, let Snare to take over control of the EventLog configuration, enable the Snare remote control interface with a password and finish Snare installation. Launch “Snare for Windows” from you “Program” menu, you will be connected in the web remote control interface on port 6161/TCP.

In the “Network Configuration” menu, configure the “Destination Snare Server” (here 192.168.178.66), the “Destination Port” (here 514), and click the checkbox for “Enable SYSLOG Header“, the save the configuration.

To reload the Snare configuration just click on the “Reload Settings” in the “Apply the Latest Audit Configuration“. And here we go, the Windows events are send to the Logger. For further instructions on how to configure Snare we recommend you to read the Snare documentation 🙂

Windows Events in your Logger

In ArcSight Logger you will now have all the Windows Events directly accessible by the “Analyze” menu and “Search” sub-menu.

ArcSight L750MB Logger Centos installation

10/04/2011Log ManagementArcSight, Loggerwow

Since ArcSight Protect 2010 in September 2010, the Logger model L750MB has been integrated in the ArcSight Logger product catalog. In our previous blog post we have analyse the Logger L750MB features and limits. We will resume here some of the features and limitations provided by L750MB and provide an installation guide for Centos 5.x

Features :

Connector appliance features are disabled.
Alerting module features are enabled.
Reporting module features are enabled.
SAN storage feature is disabled.
Logger peering features are disabled.

Limits :

10 devices maximum supported.
Maximum number of daily collected data is 750 MB
EPS rate is limited to a maximum of 60
maximum data retention is 50 GB

OS & Hardware requirements

You can install L750MB Logger on these following certified operating systems :

Red Hat Enterprise Linux (RHEL), version 5.4, 64-bit
Oracle Enterprise Linux (OEL) 5.4, 64-bit
CentOS, version 5.4, 64-bit

or on these others supported operating systems :

Red Hat Enterprise Linux (RHEL), version 4.x, 64-bit
CentOS, version 4.x, 64-bit

Virtual Machine installation of the above listed OS is supported. You will not able to install the Logger on an existing machine how is running MySQL or PostgreSQL. We recommend you a complete dedicated operating system for the installation. You will also need a synchronized NTP for all your infrastructure. A synchronized time is a key factor for Log Management. After the installation you will need one of the following supported browser, with Adobe Flash Player plug-in :

Internet Explorer: Versions 7 and 8
Firefox: Versions 3.0 and 3.5

For hardware requirements we recommend you :

CPU : 1 or 2 Core
Memory : 4 – 12 GB
Disk Space : 120 GB

Storage strategy & retention policy requirements.

As ArcSight Logger L750MB has a limit of 50GB maximum data retention, your storage strategy and retention policy will be simple to define, just follow the ArcSight recommended installation, and then we will change it by the ArcSight Logger Web interface.

By the recommended installation ArcSight Logger will initialize the Storage Volume to the maximum authorized, aka 50 GB, and the Storage Volume has to be on local disk, on a NFS, or SAN mount point. You will not be able to increase the size of the Storage Volume above 50GB with the L750MB, and once the Storage Volume size is configured the only way to resize the Storage Volume is to reinstall every thing.

Also, with the recommended installation ArcSight Logger will initialize the maximum of 6 Storage Groups. Two of these Storage Groups are inherent to the Logger and are named “Default Storage Group” and “Internal Event Storage Group“. if you choose to not create the maximum of 6 Storage Groups, you will not further able to create more Storage Groups. Here under the default Storage Groups configuration :

[TABLE=14]

You will be able to resize all Storage Groups, we recommend you to, until you understand the concept of “Devices”, “Device Groups” and “Storage Rules”, to not touch the “Internal Event Storage Group” definition and to provided the maximum size to the “Default Storage Group“. You will have then this configuration :

[TABLE=15]

Installation

First of all you will need an updated Centos 5.4 installation, just follow the Centos installation procedures. You will need to configure IP addresses, DNS and NTP configuration before starting the Logger installation procedure. As ArcSight Logger

Create an arcsight user and group :

Give a password to the arcsight user :

Upload “ArcSight-logger-5.0.0.5355.2.bin” installation binary and your “arcsight_logger_license.lic” license file in the arcsight home directory.

Make the installation binary executable :

To install the Logger in console mode execute the following command :

ArcSight Logger Console mode installation

On the first prompt press enter to display the license agreement and accept the terms of agreement.

Provide the installation directory, in “/home/arcsight”, and then press enter to begin the continue the installation.

After the end of the installation, you will need to press “enter” to initialize the Logger. This initialization may take several minutes.

ArcSight Logger successful initialization

When initialization is done you will have to configure the Logger, by a configuration wizard. To start this wizard in console mode, please type the following command.

ArcSight Logger configuration in console mode

The license file location will be asked.

Choose the typical installation type if you are not familiar with ArcSight Logger indexing, storage groups, and storage volume. Also don’t forget that the L750MB will not permit you to go above a theoretically 50GB storage. As described above we will change to Storage Groups settings further.

When the complete configuration is finished we recommend you to not start directly the logger and reboot the server.

After the reboot log you on the server with the arcsight user to start the logger with the following commands.

The “loggerd” command is located in “/home/arcsight/current/arcsight/logger/bin” directory. If the startup is successful you will have this return.

The “loggerd” command can have these following arguments.

Now you can log in ArcSight Logger Web interface on port 9000 with https and you will have the following login page.

The default login is “admin“, and the default password is “password“, please change it 🙂 To change your password just go in the “System Admin” menu, then in the “Change Password” sub-menu.

To change the Storage Groups settings just go in the “Configuration” menu, then in the “Storage” sub-menu.

You have now an up and running logger, in a next blog post we will install the L750MB SYSLOG SmartConnector on a dedicated Linux server and the “SNARE” software on Windows to have our first events.

Eric Romang Blog

aka wow on ZATAZ.com