ArcSight Logger configuration backup and restoration

With your ArcSight Logger L750MB you have maybe create some particular settings, some groups with associated users, filters, saved searches, customized report queries, report templates and dashboards. It is important to have regulate backups of all these stuffs. This blog post, will explain you on how to setup “One time only” and “Scheduled” backup of your ArcSight Logger configuration.

An important thing to know is that “Configuration Backup don’t include backup of the received events.

Configuration Backup” can only be made on a different host than the Logger and only by SSH SCP. So you will need to have a system user on a server how has a valid SSH connexion, also you will need to create a folder in this user home directory in order to receive the “tar.gz” backup file. In our example this folder will be named “backup“.

One time only” or “Scheduled” configuration backup

To configure an “One time only” or a “Scheduled” backup you will to log in the Logger Web administration and go in the “Configuration -> Configuration Backup” menu.

Edit the existing “Configuration backup” entry by clicking on the edit button and complete the fields.

Port : The port on which the SSH server is listening (by default 22)
IP/Host : IP address or host name of the SSH server.
User : The remote SSH user.
Password : The remote SSH password
Remote directory : The remote directory how the backup will be deposited.
Schedule : For “One time only” backup, let the check box be checked. For “Scheduled” backup, choose “Everyday” or “Days of Week” (Example : Su, M, T, W, Th, F, Sa), and “Hour of day” (in 24 hour format, example : 1, 4, 7, 12, 23), or “Every Hours” (in 24 hour format, example : 1, 4, 7, 12, 23) or “Every Minutes” (Example : 15, 20, 30, 59). For the “Every Minutes” setting you can not a value less than 15 minutes.

Backup content :All” for all the configurations or “Report Content only” for reports, queries, parameters, dashboards and templates.

Then click on “Save” button to save your “Configuration Backup” settings.

To start the backup click twice on the extreme right icon of the “Configuration Backup” Web page. One time to deactivate the backup and one other time to reactivate the backup. If you don’t do this, the backup will not be done.

 

One the remote server, in the “$HOME_SSH_USER/backup” directory, you will see  a file with a unique name (ex : 26Jun11_183551.configs.tar.gz).

Scheduled “Configuration Backup” specificities

Scheduled “Configuration Backup” appear in the “Scheduled Tasks” page, accessible from the “Configuration” menu.

You can, from this page, edit the “Configuration Backup” settings, delete the “Configuration Backup“, enable or disable the schedule of the “Configuration Backup“.

Also, you can verify that the “Configuration Backup” has occur successfully by verifying the “Finished Tasks“.

If your scheduled “Configuration Backup” has not occur successfully you can also find all the outputs in the “Finished Tasks”.

Another way to check the scheduled tasks results is to read the “$ARCSIGHT_HOME/current/arcsight/logger/logs/logger_server.out.log” file. For people how have an Logger appliance you can download the logs files from the “Configuration -> Retrieve Logs” menu.

Unfortunately they are no CEF event generated when a scheduled task has occur successfully or failed. So no way to have a clear view on scheduled tasks activities.

“Configuration Backup” restoration

When you restore your “Configuration Backup” all existing content are not preserved and deleted, also you can only restore a “Configuration Backup” from the same operating system and version of Logger.

To restore your backup, you only have to log in the Logger Web administration and go in the “Configuration -> Configuration Backup” menu. Then click on the “Restore” button and upload your configuration backup. Once the “Configuration Backup” is restored the Logger will reboot. So plan your restore 🙂

ArcSight Logger and SmartConnectors Questions and Answers

I receive questions about ArcSight Logger and SmartConnectors, you will find here under some answers. I will add more questions and answers in future. Don’t hesitate to add your questions as comments on this blog post.

Is ArcSight Logger L750MB still free for download ?

ArcSight Logger L750MB is now for free, since 17 August. You don’t have to pay 49$ per year any more.

Is ArcSight Logger L750MB available as ISO or virtual appliance ?

ArcSight Logger L750MB is not provided as ISO or as virtual appliance (VMWare image, Xen, VirtualBox, KVM, etc.). The Logger is available as a binary file how will install the software on an existing operating system.

Where can I find an ArcSight Logger demo ?

ArcSight has publish a Logger demonstration video on YouTube.

Where can I find an ArcSight SmartConnector list ?

For Logger L750MB, you can find all the supported products list in my previous blog post. For a complete list of ArcSight SmartConnector supported products, a PDF is available on ArcSight web site.

Where can I download CEF (Common Event Format) specifications ?

ArcSight doesn’t provide direct access to the CEF open log management standard. You have to contact ArcSight through this Web page.

What are Logger and SmartConnector default ports ?

For ArcSight Logger it is depending if you have acquire a software or appliance version. If you have a software version, the default port will be 9000/TCP to access to the Logger Web interface and to configure the destination port of your SmartConnector. If you have an appliance version, the default port will be 443/TCP to to the Logger Web interface and to configure the destination port of your SmartConnector.

For ArcSight ESM, all communication are done on port 8443/TCP by default.

What is ArcSight Logger administration default URL ?

Default administration URL is https://$LOGGER:9000 for a software version, or https://$LOGGER:443 for an appliance version. Replace the $LOGGER variable with the hostname or IP address of your Logger.

What are ArcSight Logger default login and password ?

ArcSight Logger default login and password are “admin” / “password” 🙂

How many Storage Groups are available in ArcSight Logger ?

ArcSight Logger propose 6 Storage Groups, one of them is reserved for internal activities and one will be created by default. You have to create the 4 others Storage Groups during the Logger setup, after the installation you will no more able to create additional Storage Groups.

Do an ArcSight SmartConnector require a server ?

Depending on your architecture, you will require or not a server to host an ArcSight SmartConnector.

Cases you don’t need a server to host a SmartConnector :

  • You have a L3x00 serie Logger. These Logger series have an embedded SmartConnector appliance, so you will be able to manage embedded SmartConnectors and a certain number of remote SmartConnectors directly from the Logger.
  • You have a SmartConnector appliance. SmartConnector appliances are able to manage a certain number of embedded SmartConnectors, so you will be able to manage embedded SmartConnectors and a certain number of remote SmartConnectors directly from the Logger.
Cases you will need a server to host a SmartConnector :
  • You have a software Logger (L750MB or L5GB). Software Logger doesn’t provide any embedded SmartConnector appliance, you will not be able to manage remote SmartConnectors through the Logger.
  • You have a L7x00 serie Logger. These Logger series doesn’t provide any embedded SmartConnector appliance, you will not be able to manage remote SmartConnectors through the Logger.
  • You have ArcSight ESM. ESM don’t provide any embedded SmartConnectors, but you will able to manage remote SmartConnectors.

ArcSight SmartConnector commands and features

If you have download for free the ArcSight Logger L750MB version, follow the installation guideline under Centos and install Windows Snare with ArcSight Syslog SmartConnector, you have now an operational lab or production environment. In this post we will describe you some SmartConnector commands and features. These commands and features are not documented in the provided ArcSight Logger L750MB documentation.

Starting the SmartConnector

If you don’t have specify, during the setup, to run the SmartConnector as a service, you will ne to start it manually. To start the SmartConnector you need to go in the “$ARCSIGHT_HOME/current/bin” directory and execute the “arcsight” script for Linux, or “arcsight.bat” script under Windows, with the following argument.

Starting ArcSight SmartConnector
Starting ArcSight SmartConnector

Once started, to confirm that the SmartConnector is working properly, you will have to check these outputs.

ArcSight SmartConnector starting outputs
ArcSight SmartConnector starting outputs

Eps” will give you the actual EPS throughput, and “Evts” the total number of events how have been processed by the SmartConnector. “ET” and “HT” should have twice the “Up” value in order to validate the the SmartConnector connexion with the Logger is working properly. If they are any communication troubles between the SmartConnector and the Logger you will have these kind of outputs.

ArcSight SmartConnector and Logger communication troubles
ArcSight SmartConnector and Logger communication troubles

Checking SmartConnector availability

To valide that the SmartConnector is up and running, you can use the following command.

ArcSight SmartConnector agent up
ArcSight SmartConnector agent up

If the SmartConnector is down, you will have this result.

ArcSight SmartConnector down
ArcSight SmartConnector down

This command will not validate that the communication between the SmartConnector and the Logger is up and running.

Restarting the SmartConnector

To restart the SmartConnector you will have to use the following command.

ArcSight SmartConnector restart
ArcSight SmartConnector restart

Stopping the SmartConnector

If you have start the SmartConnector in the standalone mode, a simple CTRL+C will terminate the activities. But you can also stop the activities with the following command :

Stopping ArcSight SmartConnector
Stopping ArcSight SmartConnector

Checking SmartConnector status

To check the complete SmartConnector status use the following command.

ArcSight SmartConnector status
ArcSight SmartConnector status

The output will provide you some useful informations about the SmartConnector activities (memory usage, agent type, agent version, processed events, EPS, last event processed date, etc.)

Checking SmartConnector DNS resolution

To verify that the SmartConnector is able to do DNS resolution you can execute the following command.

ArcSight SmartConnector DNS test
ArcSight SmartConnector DNS test

ArcSight Agent FlexAgent Regex Tester

ArcSight provide, with the SmartConnector, a tool how will permit you to create and test regex for your logs. SmartConnectors delivered for ArcSight Logger L750MB will not allow you to add FlexConnectors (custom agents), but you can still use this regex GUI for personal purposes. To start the regex GUI execute the following command.

ArcSight Agent FlexAgent regex tester
ArcSight Agent FlexAgent regex tester

For example, I have test the regex tool, with the following postfix log entry.

May 12 04:14:13 logger sendmail[3457]: p4C2EDU2003456: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=local, pri=31483, dsn=2.0.0, stat=Sent

The regex tester will provide you a solution on how to parse this log.

ArcSight regex tester example
ArcSight regex tester example

Download ArcSight Logger for free and get your Love Thy Logs t-shirt

ArcSight propose you, until first June 2011, to download for free it’s ArcSight Logger L750MB version. If you register the free version of ArcSight Logger L750MB with the “FreeLogger4WebA” promo code, you will also receive a “Love they logs” t-shirt.

We have several blog posts how could help you install and configure your free ArcSight Logger.