Tag Archives: ArcSight

ArcSight SmartConnectors Disk Size and Memory Requirements

Event Management, Log Management,

If you plan to install an ArcSight SmartConnector for you free ArcSight Logger, you will need to have the following disk space and memory size requirements.

Disk space

The basic SmartConnector installation take around 500 MB, if you configure the SmartConnector cache with the default setting then you will need 1 GB more, plus the SmartConnector generate some logs (10 x 10 MB) and thread dumps in case of crash. I recommend you to have minimum 3 GB disk space dedicated to the SmartConnector.

You can specify some parameters in SmartConnector configuration file in order to customize the SmartConnector log files to your needs :

  • “log.channel.file.property.package.com.arcsight=1“. This parameter will define the loglevel how will be recorded into log files (0 for Debug, 1 for Info, 2 for Warning, 3 for Error and 4 for Fatal). Anything upper or equal to the specified level will be recorded. By default 1.
  • “log.channel.file.property.path=agent.log“. This parameter will allow you to define the log file name, by default “agent.log“.
  • “log.channel.file.property.maxsize=10MB“. This parameter will allow you to define the maximum file size of the log before log rotated. By default 10MB.
  • “log.channel.file.property.maxbackupindex=10“. This parameter will allow you to define the maximum number of log backup files. By default 10.

Attention, don’t modify the “agent.defaults.properties” file located in “$ARCSIGHT_HOME/current/config/agent” folder, but add the parameters to “$ARCSIGHT_HOME/current/user/agent/agent.properties“. Also you will need to restart the SmartConnector to apply the new parameters.

Memory size

By default the SmartConnector is configured to use a minimum and maximum of 256MB of RAM, but you can adjust this value in “$ARCSIGHT_HOME/current/user/agent/agent.wrapper.conf“. “agent.wrapper.conf” file doesn’t exist by default, you will have to create it and add “wrapper.java.maxmemory=512” in the file, if you would to increase the maximum memory attributed to the SmartConnector.

ArcSight SmartConnector Configuration User Guide – Part 1

Event Management, Log Management,

With the free ArcSight Logger L750MB, you have download some associated SmartConnectors, Snare SmartConnector, Cisco IOS SmartConnector, Unix Auditd SmartConnector, etc. The configuration of each SmartConnector is customizable in order to activate batching, time correction, caching, QoS (Quality of Service), aggregation or filtering.

In this blog post we will show some examples on how to configure your SmartConnector.

SmartConnector Configuration setup

Under Windows, you will need to run the “runagentsetup.bat” script and under Linux the “runagentsetup.sh” script. The scripts are located into your “$ARCSIGHT_HOME/current/bin” directory.

Changing SmartConnector parameters

By selecting “0 – I want to change SmartConnector parameters“, you will be able to change, for example on a Syslog Daemon SmartConnector, the network port, listener IP address and associated protocol. You will also able to switch between a standalone application installation or as a service installation.

Changing SmartConnector service settings

By selecting “1 – I want to change SmartConnector service settings“, you will be able, same as for the first option, to switch between a standalone application installation or as a service installation.

Adding/Removing/Modifying SmartConnector Destinations

By selecting “2 – I want to add/remove/modify ArcSight Manager destinations“, you will be able to modify your initial destination configuration, in our case a L750MB Logger, or to add a new destination, for example another Logger or ESM (Enterprise Security Manager)

This configuration option will allow you to add a fail over destination or also to re-register a SmartConnector.

Modifying actual SmartConnector destination parameters

If you select your actual destination you will be able to modify your actual destination parameters “0 – Modify destination parameters” :

  • Host name or IP address of the actual destination
  • Port of the actual destination
  • Receiver Name of the actual destination
  • Enable or not compression mode


Compression mode will allow the SmartConnector to send events to the destination in a compressed format and lowers the network bandwidth usage.

Adding a fail over destination to your actual SmartConnector

If you need to have a fail over destination for your actual SmartConnector configuration you will have the choice between an ESM or Logger destination. The setup of the fail over destination is the same as creating a first destination, you will have to provide the following information’s :

  • Host name or IP address of the fail over destination (Manager or Logger)
  • Port of the actual destination (Manager or Logger). 9000 for software Logger, 443 for appliance Logger and 8443 for ESM.
  • Receiver Name of the actual destination (Logger)
  • Enable or not compression mode (Logger)

Some specific configurations are necessary for an ESM destination like “AUP Master Destination“, “Filter Out All Events“, ESM user name and password, but we will not describe these configurations settings.

Configure multiple parallel destinations

ArcSight SmartConnector is also able to send a copy of events to each additional configured destinations. You could for example send you events in parallel to two Logger’s, or to one Logger and one ESM, or to two ESM. This could be useful to have a copy of the events or to setup a lab environment in parallel of your production environment. You can configure additional destinations by selecting “1 – Add new destination“.

Modifying batching destination settings

SmartConnectors allow you to batch all processed events, in order to increase performance and bandwidth usage, by three options :

  • Batching per events : The connector will wait that the provided value of events is reached and then send the block of events to your destination. The default configured value is 100 events.
  • Batching per seconds : The connector will wait that the provided value in seconds is reached and then send the block of events to your destination. The default configured value is 5 seconds.
  • Batch by time based or severity based : The “Time Based” selection will allow the SmartConnector to send batches as they arrive, how is the default configuration. The “Severity Based” selection will allow the SmartConnector to send batches based upon the events severity (highest first then lowest).

If you have high volume of events, you should prefer the “Severity Based” selection in order to have a focus on high severity events first.

Modifying time correction settings

ArcSight SmartConnector will allow you to do time corrections. For example, if you have a remote SmartConnector how has a valid NTP synchronization :

  • The end device could have time troubles, cause no NTP configured or bad NTP configuration.
  • The end device could be in a different timezone than the remote SmartConnector.

If the device is in the past, you should normally receive this kind of event:

deviceEventClassID : agent:012
name : Device Receipt Time from [centos5-6-1.zataz.loc|192.168.178.76|Unix|auditd] may be incorrect - Device Receipt Time is smaller than Agent Receipt Time (Events are in the past)
deviceEventCategory : /Agent/Time/Device?Failure
deviceReceiptTime : The device time how is in the past.
agentReceiptTime : The time of the SmartConnector.

If the device is in the future, you should normally receive this kind of event:

deviceEventClassID : agent:012
name : Device Receipt Time from [centos5-6-1.zataz.loc|192.168.178.76|Unix|auditd] may be incorrect - Device Receipt Time is greater than Agent Receipt Time (Events are in the future)
deviceEventCategory : /Agent/Time/Device?Failure
deviceReceiptTime : The device time how is in the futur.
agentReceiptTime : The time of the SmartConnector.

By setting “Use Connector Time as Device Time” to yes, the SmartConnector will override the reported device time by using the SmartConnector time.

Also you will able to provide a “Device Time Correction” in seconds for all devices how are sending they’re events to this SmartConnector. You will also be able to provide a “SmartConnector Time Correction” in seconds, if the server where the SmartConnector is installed don’t has a synchronized NTP.

Another example of time correction is if the device doesn’t have a valid configured timezone or if the device events don’t report the timezone, you can through this option specify the desired timezone on the SmartConnector.

In “$ARCSIGHT_HOME/logs/agent.log” file, you can see that these settings are activated by :

But take care, time correction is not a setting to configure without knowing what you are doing. If you modify the timestamp of the events, you will :

  • Gain the ability to do events searches, generate reports or valid correlation rules on the ESM or Logger based on the same timestamp.
  • Loose the “real” timestamp of the events, if you don’t preserve raw events. In case of forensics purpose to loose the “real” timestamp could be dramatically.

Modifying time auto-correction settings

With time auto-correction you will be able to specify forward and backwards time limits for your events. If the values are exceeded the SmartConnector will automatically correct the time with the SmartConnector time. By default the “-1” values are disabling the auto-correction, you can specify values in seconds for the forward and backwards time limits, and also filter by devices separated by commas.

In “$ARCSIGHT_HOME/logs/agent.log” file, you can see that these settings are activated by :


Modifying time checking settings

These settings will allow you to specify the time threshold and frequency for device time checking. By the SmartConnector will forward events how are 300 seconds in the future and forward events how are 3600 seconds in the past, compared to the SmartConnector time.

Modifying Cache settings

If the configured events destinations are down, or if the number of EPS are to high to be forwarded by batching, the SmartConnector will begin to cache the events. With the cache settings you will be able to adjust your cache size from 5MB to 50GB (by default 1GB), after how many events the SmartConnector will trigger a notification, by default 10 000 events, and the frequency of these notifications from 1 minute to 60 minutes, by default every 10 minutes.

The cache will work in a FIFO mode (First in, First out) if the cache is full, so you have to adjust you cache depending on the number of EPS of the device. For example a device with 25 EPS, and with an average size of 300 bytes per event, you will have the default 10 000 events threshold limit reached in 400 seconds, and the default 1 GB cache size reached in 143 165 seconds (39 hours). But you have also to take into account the aggregation settings how will reduce drastically the size of events in your cache.

Cache files are stored in “$ARCSIGHT_HOME/current/user/agent/agentdata” folder.

If the SmartConnector is caching you will have this kind of event :

deviceEventClassID : agent:019
name : Agent is currently caching events
message : Agent cache contains [100] events
deviceEventCategory : /Agent/Cache/Caching

If the SmartConnector is emptying his cache :

deviceEventClassID : agent:020
name : Agent cache empty
deviceEventCategory : /Agent/Cache/Empty

In the next blog post we will continue to describe SmartConnector network, field based aggregation, filter aggregation, processing and filters settings.

Unix Auditd Authentication Events Analysis and Visualisations with ArcSight Logger

Log Management, Security visualization,

With the free ArcSight Logger L750MB, you can in combination with auditd SmartConnector, gather some useful informations in order to have a better overview on your Unix infrastructure Access Management or to respond to some compliances (ex : PCI-DSS, etc.).

In this blog post we will show some examples on how use ArcSight Logger search engine and visualisation capabilities.

Unix Auditd authentication events analysis

We will first analyse Unix auditd authentication events in order to understand what useful informations we can find.

We assume that login under root account is not allowed and that 500 represent the first usable user account.

We will describe here under different SSH authentication auditd events with they results. You can find in bold some important keywords.

  • Successful SSH authentication auditd events
type=USER_AUTH msg=audit(1316335353.969:348): user pid=32370 uid=0 auid=4294967295 msg='PAM: authentication acct="eromang" : exe="/usr/sbin/sshd" (hostname=macbook.zataz.loc, addr=192.168.178.25, terminal=ssh res=success)'

type=USER_LOGIN msg=audit(1316335353.991:355): user pid=32370 uid=0 auid=500 msg='uid=500: exe="/usr/sbin/sshd" (hostname=macbook.zataz.loc, addr=192.168.178.25, terminal=/dev/pts/0 res=success)'
  • Failed SSH authentication, for a valid user account, auditd events
type=USER_AUTH msg=audit(1316335438.989:362): user pid=32395 uid=0 auid=4294967295 msg='PAM: authentication acct="eromang" : exe="/usr/sbin/sshd" (hostname=macbook.zataz.loc, addr=192.168.178.25, terminal=ssh res=failed)'

type=USER_LOGIN msg=audit(1316335438.990:363): user pid=32395 uid=0 auid=4294967295 msg='acct="eromang": exe="/usr/sbin/sshd" (hostname=macbook.zataz.loc, addr=192.168.178.25, terminal=sshd res=failed)'
  • Failed SSH authentication, for non existing user account, auditd events
type=USER_AUTH msg=audit(1316335506.816:372): user pid=32403 uid=0 auid=4294967295 msg='PAM: authentication acct="?" : exe="/usr/sbin/sshd" (hostname=macbook.zataz.loc, addr=192.168.178.25, terminal=ssh res=failed)'

type=USER_LOGIN msg=audit(1316335506.817:373): user pid=32403 uid=0 auid=4294967295 msg='acct="invaliduser": exe="/usr/sbin/sshd" (hostname=macbook.zataz.loc, addr=192.168.178.25, terminal=sshd res=failed)'
  • Successful TTY authentication auditd events
type=USER_AUTH msg=audit(1316338747.834:381): user pid=2678 uid=0 auid=4294967295 msg='PAM: authentication acct="eromang" : exe="/bin/login" (hostname=?, addr=?, terminal=tty1 res=success)'

type=USER_LOGIN msg=audit(1316338747.867:437): user pid=2678 uid=0 auid=500 msg='op=login id=500 exe="/bin/login" (hostname=?, addr=?, terminal=tty1 res=success)'
  • Failed TTY authentication, for a valid user account, auditd events
type=USER_AUTH msg=audit(1316338880.990:495): user pid=32452 uid=0 auid=4294967295 msg='PAM: authentication acct="eromang" : exe="/bin/login" (hostname=?, addr=?, terminal=tty1 res=failed)'

type=USER_LOGIN msg=audit(1316338880.990:496): user pid=32452 uid=0 auid=4294967295 msg='op=login id=500 exe="/bin/login" (hostname=?, addr=?, terminal=tty1 res=failed)'
  • Failed TTY authentication, for non existing user account, auditd events
type=USER_AUTH msg=audit(1316338916.360:497): user pid=32452 uid=0 auid=4294967295 msg='PAM: authentication acct="?" : exe="/bin/login" (hostname=?, addr=?, terminal=tty1 res=failed)'

type=USER_LOGIN msg=audit(1316338916.360:498): user pid=32452 uid=0 auid=4294967295 msg='op=login acct="invaliduser" exe="/bin/login" (hostname=?, addr=?, terminal=tty1 res=failed)'

For USER_LOGIN auditd type events you will have these results :

  • Count of successful and failed authentication attempts (number of lines in the log).
  • Targeted username through auditd “acct” value, but only for failed attempts.
  • Targeted UID for successful and failed attempts through auditd “uid“, “id” and “auid” values.
  • Targeted device address and hostname (box hosting the auditd.log file, or IP/hostname present in SYSLOG log file).
  • Source address and hostname, only for remote authentication, through auditd “hostname” and “addr” values.
  • Associated terminal through “terminal” value.

FOR USER_AUTH type you will have these results :

  • Count of successful and failed authentication attempts (number of lines in the log).
  • Targeted user name through auditd “acct” value, but only valid user accounts.
  • Targeted device address and hostname (box hosting the auditd.log file, or IP/hostname present in SYSLOG log file).
  • Source address and hostname,  only for remote authentication, through auditd “hostname” and “addr” values.
  • Associated terminal through “terminal” value.

In order to provide Access Management analysis, we will focus on USER_AUTH type. With this type we can detect regular authentications and authentication brute force attacks on valid users, with valid user name display, “acct” value.

Unix Auditd authentication events searches in ArcSight Logger

On ArcSight Logger we can now play with searches in order to retrieve and analyse auditd authentication events.

The following filter will provide you all Unix auditd events.

deviceVendor = "Unix" AND deviceProduct = "auditd"

The following filter will provide you all auditd USER_AUTH type events.

deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH"

In order to analyse Access Management activities, we need to understand how the SmartConnector is categorizing the auditd USER_AUTH event and gather the most important fields.

  • baseEventCount : The number of occurrence of the event if aggregation is active on the SmartConnector.
  • deviceAddress : Targeted device address
  • deviceHostName : Targeted device hostname.
  • sourceAddress : The attacker address.
  • sourceHostName : The attacker hostname, if available.
  • sourceUserName : The targeted authentication username.
  • destinationProcessName : Targeted process name. “/usr/sbin/sshd” for SSH or “/bin/login” for TTY console in our cases.
  • deviceCustomString3 : The authentication result. “success” or “failed” in our cases.
  • deviceCustomString6 : For the associated terminal or tty. “ssh” or “ttyx” in our cases.
  • deviceReceiptTime : The event associated timestamp.

In the following scenario, the target was under SSH brute force attack, and the attacker has gain access to the box. We will conduct an analysis, provide you some useful search queries and operators, and try to demonstrate you that using ArcSight Logger for forensics analysis is quiet easy.

This search query is based on the last 24 hours, and we can directly see through the Logger radar that a lot of USER_AUTH requests have been made. In order to only focus on the specific fields, we will execute the following query.

deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH"  | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime baseEventCount

“cef” search operator will extracts values for specified fields from matching CEF events. To view only the extracted values, select “User Defined Fieldsets” in the search drill down menu. “cef” search operator will also allow you to use other search operators such as “sort“, “chart“, etc.

To focus on failed authentication attempts, execute the following query.

deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH"  AND deviceCustomString3 = "failed" | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString6 deviceReceiptTime  baseEventCount

To focus on successful authentication, execute the following query.

deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH"  AND deviceCustomString3 = "success" | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString6 deviceReceiptTime  baseEventCount

Browsing in the events is a quiet boring, ArcSight Logger provide you a query operator how will permit you to create visualisations with your search results.

  • Chart creation to count sourceUserName occurrences

Don’t forget that USER_AUTH auditd type don’t provide the targeted user name for non existing users, so we will only focus on existing sourceUserName.

deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName IS NOT NULL | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by sourceUserName

Associated to the result chart, you will also have a result table.

You can also, change the type of chart, by clicking on the upper right corner icon of the result chart frame. The available chart type are column, bar, pie, area, line, stacked column or stacked bar.

  • Chart creation to count Terminal occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName IS NOT NULL | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by deviceCustomString6

We can see that majority of the authentications are through SSH and that some are through TTY.

  • Chart creation to count sourceUserName and Terminal occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName IS NOT NULL | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by sourceUserName deviceCustomString6

  • Chart creation to count sourceUserName and authentication occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName IS NOT NULL | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by sourceUserName deviceCustomString3

We can see that only user name “eromang” has successful login.

  • Chart creation to count sourceUserName “eromang“, authentication and Terminal occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName = "eromang" | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by sourceUserName deviceCustomString3 deviceCustomString6

We can discover that user name “eromang” has successful login through “ssh” and “tty” terminals.

  • Chart creation to count sourceUserName “eromang” and successful authentication occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName = "eromang" AND deviceCustomString3 = "success" | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by sourceUserName deviceCustomString3 deviceCustomString6

  • Char creation to count sourceUserName “eromang”, successful “ssh” authentication and sourceAddress occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH" AND sourceUserName = "eromang" AND deviceCustomString3 = "success" AND deviceCustomString6 = "ssh" | cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by sourceAddress

We can discover two different source IP addresses.

  • Chart creation to count “ssh” authentications for both sourceAddress occurences
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH"  AND (sourceAddress = "192.168.178.25" OR sourceAddress = "192.168.178.21")  AND sourceUserName = "eromang" AND deviceCustomString6 = "ssh"
| cef deviceAddress deviceHostName sourceAddress sourceHostName sourceUserName destinationProcessName deviceCustomString3 deviceCustomString6 deviceReceiptTime  baseEventCount | chart sum(baseEventCount) AS Total by deviceCustomString3 sourceAddress

We can discover that sourceAddress “192.168.178.21” has hundreds of failed login and one successful login.

  • Query to find the exact time sourceAddress has breach the target
deviceVendor = "Unix" AND deviceProduct = "auditd" AND deviceEventCategory = "USER_AUTH"  AND sourceAddress = "192.168.178.21" AND sourceUserName = "eromang" AND deviceCustomString6 = "ssh" AND deviceCustomString3 = "success" | cef deviceReceiptTime

As you can see by this example, ArcSight Logger has some powerful search engine queries and visualisation outputs how will help you to do forensics investigations for your assets.

Cisco Smart Business Architecture (SBA) guides for SIEM solutions integration

Event Management, Log Management, , , , , ,

Cisco provide some useful Smart Business Architecture (SBA) guides for SIEM solutions integration how will helps you to design and deploy best practices that include Cisco switching, routing, security and wireless technologies.

Actually the SBA guides are covering the following solutions :

  • SBA guide how provides a general overview of SIEM technology, as well as best practices, use cases, and deployment considerations for using a SIEM with Cisco infrastructure (click here to read). Cisco products logging retrieval methods,
  • SBA guide for ArcSight SIEM plateform (ESM, Logger, Express, SmartConnectors and Content Pack) integration (click here to read).
  • SBA guide for Loglogic MX Series SIEM product integration (click here to read).
  • SBA guide for netForensics nFX Cinxi One SIEM product integration (click here to read).
  • SBA guide for RSA enVision SIEM product integration (click here to read).
  • SBA guide for Splunk security management solution (click here to read).