The Honeynet Project has launch a new forensic challenge, for year 2010, called “Challenge 5 – Log Mysteries“. This challenge is provided by Raffael Marty, Anton Chuvakin and Sebastien Tricaud and take the participant into the world of virtual systems and confusing log data. In this challenge, the participant should figure out what happened to a virtual server using the logs from a possible compromised server.
I will maybe officially participate to this challenge, but I will post on my blog all analysis I done around it. If you have any suggestions or corrections don’t hesitate to post a comment on the blog, or contact me by twitter.
Part 1 : SSH activities forensics
All SSH activities are stored into the “auth.log” file with “sshd” string a primary key for all searches.
- Gathering and visualizing all SSH failed access attempt
Failed SSH access on the box are identified by string “Failed password for invalid user xxx” or “Failed password for xxx“, where xxx is representing the targeted user account. The source IP addresses are also provided by the logs.
cat auth.log | grep sshd | grep "Failed password for invalid user" | awk '{print$13",FAILED"}' > ssh_failed_sips.csv
cat auth.log | grep sshd | grep "Failed password for" | grep -vw 'invalid' | awk '{print$11",FAILED"}' >> ssh_failed_sips.csv
To create a representation of all these failed access, we will use Afterglow in two node mode. The source IPs will be colored in “lightskyblue1“, the event (here FAILED) will be colored in “red” color and represented as a “box“.
cat ssh_failed_sips.csv | afterglow.pl -c color.properties -a -d -e 3 -t | neato -Tpng -o ssh_failed_sips.png
The Afterglow “-d” option will display the node count.
The Afterglow visualization of the SSH failed access attempt is available here.
With this visualization we can see, by the node count, that some “SSH Brute force” attempts were done on the server from some source IPs.
- Gathering and visualizing all SSH success access
Accepted SSH access on the box are identified by string “Accepted password for xxx“, where xxx is representing the targeted user account. The source IP addresses are also provided by the logs.
To get a list of all success source IPs and the number of success access occurrences by source IPs, just run the following commands. We will mark the success access as “SUCCESS” into a CSV file “ssh_success_sips.csv” for further usages.
cat auth.log | grep sshd | grep "Accepted password for" | awk '{print$11",SUCCESS"}' > ssh_success_sips.csv
To create a representation of all these success access, we will use Afterglow in two node mode. The source IPs will be colored in “lightskyblue1“, the event (here SUCCESS) will be colored in “green” color and represented as a “box“.
cat ssh_success_sips.csv | afterglow.pl -c color.properties -a -d -e 3 -t | neato -Tpng -o ssh_success_sips.png
The Afterglow “-d” option will display the node count.
The Afterglow visualization of the SSH success access attempt is available here.
We can see a large amount of successful access on the box from different source IPs.
- Merging successful and failed SSH access
Now we have all successful and failed SSH access, we will merge all these datas into another CSV file “ssh_failed_success_sips.csv” to have a global view.
cat ssh_failed_sips.csv > ssh_failed_success_sips.csv cat ssh_success_sips.csv >> ssh_failed_success_sips.csv
To create a representation of all these access, we will use Afterglow in two node mode. The source IPs will be colored in “lightskyblue1“, the success event (SUCCESS) will be colored in green, the failed event (FAILED) will be colored in red, and represented as a “box“.
cat ssh_failed_success_sips.csv | afterglow.pl -c color.properties -a -d -e 3 -t | neato -Tpng -o ssh_failed_success_sips.png
The Afterglow visualization of the SSH all access attempt is available here.
Here under a Google gadget for all SSH success or failed access with count by result.
We can see that some source IPs have both failed and success attempts, some of them with “brute force attacks”. Other source IPs have only success access to the box. Now we will create a CSV file “ssh_success_unique_sips.csv” with only the source IPs how have successfully access the box. This new CSV file will be used to create another CSV file “ssh_sips_stats.csv” representing all SSH success and failed access attempt, based on the unique source IPs how have successful access the box.
for i in `cat ssh_success_unique_sips.csv`; do success=`cat ssh_failed_sips.csv | grep $i | wc -l`; failed=`cat ssh_success_sips.csv | grep $i | wc -l`; echo $i,"FAILED",$success >> ssh_sips_stats.csv; echo $i,"SUCCESS",$failed >> ssh_sips_stats.csv; done
All the results have been imported into a Google gadget available here under.
With the CSV file “ssh_failed_success_sips.csv“ we will create a new CSV file “all_ssh_sips.csv” containing all source IPs how have try or access the box.
cat ssh_failed_success_sips.csv | awk -F "," '{print$1}' | sort -u > all_ssh_sips.csv
Based on this new CSV file “all_ssh_sips.csv” and the existing CSV file “ssh_success_unique_sips.csv” we will now create a dedicated CSV file “ssh_failed_sips.csv” for source IPs how have fail to access the box and never succeeded.
comm -13 ssh_success_unique_sips.csv all_ssh_sips.csv > ssh_failed_sips.csv
By executing this script, you will have stats for each source IPs how have fail to access the box and never succeeded. This script will create a CSV file called “ssh_failed_sips_count.csv“.
./ssh_failed_sips_count.sh
Here under you can get a Google gadget with all these datas.
- Detailed statistics for success source IPs
I developed a bash script to get all detailed statistics for the source IPs how have successfully SSH access to the box. This script “ssh_sips_detailled_stats.sh” give you detailled results “ssh_sips_detailled_stats.txt” (First seen, last seen, number of events, number of different logins, first successful SSH connexion, last successful SSH connexion, timeline of all successful SSH connexions, number of failed connexions). Another datas could be provided by executing the following command how will create a new CSV file “ssh_timeline.csv” with all the success SSH access timeline. Here under a Google gadget with all the timeline results.