Honeynet Forensic Challenge 5 Log Mysteries – Analysis – Part 1

The Honeynet Project has launch a new forensic challenge, for year 2010, called “Challenge 5 – Log Mysteries“. This challenge is provided by Raffael Marty, Anton Chuvakin and Sebastien Tricaud and take the participant into the world of virtual systems and confusing log data. In this challenge, the participant should figure out what happened to a virtual server using the logs from a possible compromised server.

I will maybe officially participate to this challenge, but I will post on my blog all analysis I done around it. If you have any suggestions or corrections don’t hesitate to post a comment on the blog, or contact me by twitter.

Part 1 : SSH activities forensics

All SSH activities are stored into the “auth.log” file with “sshd” string a primary key for all searches.

  • Gathering and visualizing all SSH failed access attempt

Failed SSH access on the box are identified by string “Failed password for invalid user xxx” or “Failed password for xxx“, where xxx is representing the targeted user account. The source IP addresses are also provided by the logs.

To get a list of all failed access source IPs and the number of failed access occurrences by source IPs, just run the following commands. We will mark the failed access as “FAILED” into a CSV “ssh_failed_sips.csv” file for further usages.
cat auth.log | grep sshd | grep "Failed password for invalid user" | awk '{print$13",FAILED"}' > ssh_failed_sips.csv
cat auth.log | grep sshd | grep "Failed password for" | grep -vw 'invalid' | awk '{print$11",FAILED"}' >> ssh_failed_sips.csv

To create a representation of all these failed access, we will use Afterglow in two node mode. The source IPs will be colored in “lightskyblue1“, the event (here FAILED) will be colored in “red” color and represented as a “box“.

cat ssh_failed_sips.csv | afterglow.pl -c color.properties -a -d -e 3 -t | neato -Tpng -o ssh_failed_sips.png

The Afterglow “-d” option will display the node count.

The Afterglow visualization of the SSH failed access attempt is available here.

With this visualization we can see, by the node count, that some “SSH Brute force” attempts were done on the server from some source IPs.

  • Gathering and visualizing all SSH success access

Accepted SSH access on the box are identified by string “Accepted password for xxx“, where xxx is representing the targeted user account. The source IP addresses are also provided by the logs.

To get a list of all success source IPs and the number of success access occurrences by source IPs, just run the following commands. We will mark the success access as “SUCCESS” into a CSV file “ssh_success_sips.csv” for further usages.

cat auth.log | grep sshd | grep "Accepted password for" | awk '{print$11",SUCCESS"}' > ssh_success_sips.csv

To create a representation of all these success access, we will use Afterglow in two node mode. The source IPs will be colored in “lightskyblue1“, the event (here SUCCESS) will be colored in “green” color and represented as a “box“.

cat ssh_success_sips.csv | afterglow.pl -c color.properties -a -d -e 3 -t | neato -Tpng -o ssh_success_sips.png

The Afterglow “-d” option will display the node count.

The Afterglow visualization of the SSH success access attempt is available here.

We can see a large amount of successful access on the box from different source IPs.

  • Merging successful and failed SSH access

Now we have all successful and failed SSH access, we will merge all these datas into another CSV file “ssh_failed_success_sips.csv” to have a global view.

cat ssh_failed_sips.csv > ssh_failed_success_sips.csv
cat ssh_success_sips.csv >> ssh_failed_success_sips.csv

To create a representation of all these access, we will use Afterglow in two node mode. The source IPs will be colored in “lightskyblue1“, the success event (SUCCESS) will be colored in green, the failed event (FAILED) will be colored in red, and represented as a “box“.

cat ssh_failed_success_sips.csv | afterglow.pl -c color.properties -a -d -e 3 -t | neato -Tpng -o ssh_failed_success_sips.png

The Afterglow visualization of the SSH all access attempt is available here.

Here under a Google gadget for all SSH success or failed access with count by result.

We can see that some source IPs have both failed and success attempts, some of them with “brute force attacks”. Other source IPs have only success access to the box. Now we will create a CSV file “ssh_success_unique_sips.csv” with only the source IPs how have successfully access the box. This new CSV file will be used to create another CSV file “ssh_sips_stats.csv” representing all SSH success and failed access attempt, based on the unique source IPs how have successful access the box.

for i in `cat ssh_success_unique_sips.csv`; do success=`cat ssh_failed_sips.csv | grep $i | wc -l`; failed=`cat ssh_success_sips.csv | grep $i | wc -l`; echo $i,"FAILED",$success >> ssh_sips_stats.csv; echo $i,"SUCCESS",$failed >> ssh_sips_stats.csv; done

All the results have been imported into a Google gadget available here under.

With the CSV file “ssh_failed_success_sips.csv“ we will create a new CSV file “all_ssh_sips.csv” containing all source IPs how have try or access the box.

cat ssh_failed_success_sips.csv | awk -F "," '{print$1}' | sort -u > all_ssh_sips.csv

Based on this new CSV file “all_ssh_sips.csv” and the existing CSV file “ssh_success_unique_sips.csv” we will now create a dedicated CSV file “ssh_failed_sips.csv” for source IPs how have fail to access the box and never succeeded.

comm -13 ssh_success_unique_sips.csv all_ssh_sips.csv > ssh_failed_sips.csv

By executing this script, you will have stats for each source IPs how have fail to access the box and never succeeded. This script will create a CSV file called “ssh_failed_sips_count.csv“.

./ssh_failed_sips_count.sh

Here under you can get a Google gadget with all these datas.

  • Detailed statistics for success source IPs

I developed a bash script to get all detailed statistics for the source IPs how have successfully SSH access to the box. This script “ssh_sips_detailled_stats.sh” give you detailled results “ssh_sips_detailled_stats.txt” (First seen, last seen, number of events, number of different logins, first successful SSH connexion, last successful SSH connexion, timeline of all successful SSH connexions, number of failed connexions). Another datas could be provided by executing the following command how will create a new CSV file “ssh_timeline.csv” with all the success SSH access timeline. Here under a Google gadget with all the timeline results.

More in the next post…

ArcSight annonce confusing L5GB, or L750MB, Log Management solution

ArcSight Inc. has annonce the release of a new version of his Log Management solution, version 5.0 of ArcSight Logger. The entry price for this update is announced at 49$.

In the “ArcSight Logger – Universal Log Management Products Descriptions” section of ArcSight Web site, we have two tabs one “DOWNLOAD” and one “SPECS“.

In the “DOWNLOAD” tab, the new product is named L750MB, and has a limit of 750 MB of logs per day, with a total searchable space of 50 GB.

In the “SPECS” tab, the another product, or the same product, is named L5GB, and has a limit of 5G of logs per day, with a maximum of 50 devices connected to the Log Management solution. No total searchable space is given for the L5GB Logger.

With an average compression rate of 10:1(dependent on data type and data source), the L750MB has the following data’s :

  • 750 MB of daily raw logs represent 75 MB daily compressed logs.
  • With a total searchable space of 50 GB, the log retention policy could be theoretically be 682 days.

In the ArcSight Logger “Product Brief” PDF. They’re is no motion of the L5G product, but only the L750MB one. The characteristic’s, in the “Product Brief” PDF file, for the L750MB are :

  • Daily Limit on Log Data : 750 MB.
  • Total searchable Space (Compressed) : 50 GB.

In the 49$ product “Terms and Conditions”, the mentioned product is the L750MB, no trace of the L5G.

  • Daily Limit on Log Data : 750MB – 3.3 (viii)
  • Total searchable Space (Compressed) : 10 GB3.3 (ix)
  • 10 devices max. – 3.3 (x)
  • 2 physical CPU max (2 dual cores, or 2 quad cores, or 2 etc.) – 3.3 (xi)

With 10 GB total searchable space (compressed) the retention policy is downgraded from 682 days to 136 days ! We can also discover new restrictions for devices and CPU.

3.3. Restrictions
(viii) process more than 750MB of incoming data per day with the Software;
(ix) use the Software to store more than 10GB of data;
(x) use the Software with more than 10 devices,

Also, what is interesting in the “Terms and Conditions” is the title of the document “THE ONE-YEAR TRIAL VERSION OF LOGGER“. Is the 49$ product an one year trial ?

Another fact how is interesting in the “Terms and Conditions”, is the restriction “3.3 (ii)“. As MSSP you don’t have the possibility to distribute, sell, sub license, rent, lease the product.
But in the press release, ArcSight has target the SMB market with this new solution. Is this product not usable by MSSP ?

The new ArcSight offer seem to be quiet interesting, but clearly confusing. Maybe the rush of ArcSight Protect 10 is the cause of all these confusions.

Honeynet Forensic Challenge 5 Log Mysteries – Switched Off

On demand of the Honeynet Team, and to be fair, I will switch off all Blog posts related to the Log Mysteries Challenge. All the analysis will be posted at the end of the challenge, the 30 September 2010.