Embed Size (px)
Citation preview
CDX Event Report Blue Angels Team, University of Cincinnati
Tyler Brisbin, Jacob Chesley,
Suprabha Hegde, Andrew Rose
4 PM Dec. 2, 2015 to 12 PM Dec. 5, 2015
Table Of Contents Table Of Contents
Overview
1 Tools Used
2 Division of Labor
2.1 Tyler Brisbin
2.2 Jacob Chesley
2.3 Suprabha Hegde
2.4 Andrew Rose
3 Problems
Planning
1 Initial Plan
2 Hosting
3 The Contest
4 The Report
VM Hardening
1 PreEvent
2 The Actual Event
2.1 10.8.0.220 & 10.8.0.221
2.2 10.8.0.222
Attack Analysis
1 Spooky Skilenton SSH Banner Attack
1.1 Reconnaissance
1.2 Install and Actions on Objectives
1.3 Discovery and Teams Actions to Fix
2 Password Reset Attack
2.1 Reconnaissance
2.2 Install and Actions on Objectives
2.3 Discovery and Teams Actions to Fix
3 Wordpress Replaced with Cat Picture Attack
3.1 Reconnaissance
3.2 Weaponization of Exploit (Including Delivery)
1
3.3 Install and Actions on Objectives
3.4 Discovery and Teams Actions to Fix
4 Sudo Removal and Lockout Attack
4.1 Install and Actions on Objectives
4.2 Discovery and Teams Actions to Fix
5 Nyan Cat Attack
5.1 Weaponization of Exploit (Including Delivery)
5.2 Install and Actions on Objectives
5.3 Discovery and Teams Actions to Fix
Mistakes and Other Ideas
1 Mistakes
2 Ideas That Never Got Implemented
3 Future CDX Suggestions
3.1 Our Suggestions
3.2 Class Suggestions
References
Appendix A: Tap0 PCAP Statistics
2
Overview
1 Tools Used Oracle VM VirtualBox Used to run the 3 VM’s required for this exercise.
Jacob’s Sony Vaio Laptop Used to host the VM’s using virtualbox. Host OS is Linux
Ubuntu 15.04, with Oracle VirtualBox version 5.02.
Wireshark Used during/after the event to record and analyze pcap data from tap0 on
the host machine.
Dumpcap Part of Wireshark and used to continuously capture network traffic on the
tap0 interface.
Tcpflow Used to reconstruct data from the tcp streams in the pcap files.
Bro Used to analyse the pcap at the end of the contest.
Armitage and Metasploit Used for penetration testing of the vulnerable VM’s.
Outlook Email (UC Campus Email) Used to communicate before, during, and after the
contest.
Google Drive Used to store project information (Report, Notes, etc.)
2 Division of Labor 2.1 Tyler Brisbin Tyler reported on information remotely during the exercise as well as provided advice and
guidance on hardening the contest VMs based on hardening of Tyler’s own given VM. During
the exercise Tyler accessed VMs via SSH and checked for services which were running or not
during the exercise. Tyler also provided the space for reports during the exercise plus the space
to write the final organized report. Before the exercise Tyler prepared with a hardened VM for
the exercise yet issues occurred during the process of exporting and cloning the appliance as a
.ova to other computers which could better sit in one place and stay consistently connected
during the exercise. Contributed postexercise to the final report through documentation of
attacks as well as general editing and proofreading.
3
2.2 Jacob Chesley Jacob was in charge of hosting the virtual machines on his laptop at his home. He set up the
VM’s with the team, after the CDX had started. Once the machines were set up and online with
the team on campus, Jacob took his laptop home with him and quickly got the VM’s up and
running at his house. He coordinated with the team on attacks as he found them, and worked
with teammates to limit the attacks and stop them as they were happening. When the
bootloader went down, Jacob worked with Andrew on Friday evening from 9:30 pm to 12:30 AM
recreating and hardening the VM’s to bring them back online as quickly as possible. Since
Jacob had physical access to the machine that was hosting the VM’s, he was able to restart the
VM’s and enter a special boot loader command to start the VM’s in a root shell immediately to
reset passwords that had been changed by attackers. Jacob also used his host machine with
dumpcap to continuously log network traffic and record as much PCAP data as he could. Jacob
also helped to analyze the pcap data for the final report and to provide details on the attacks he
had discovered and/or fixed.
2.3 Suprabha Hegde Helped Jacob in setting up the Virtual Machines (10.8.0.220, 10.8.0.221, 10.8.0.222). Initially
during the exercise Suprabha was not able to remotely access the Virtual Machines as she
couldn’t get the openvpn working, but she got it fixed later. So she was able to monitor the
Virtual Machines from Dec 4th onwards. She constantly kept checking which services were up
and working and if the services weren’t she would immediately report to Jacob as he had
physically set up all the VMs’ on his laptop.
2.4 Andrew Rose Worked extensively at the beginning of the contest to secure VM’s after the group's failure to do
so before the start of the contest. Helped Jacob get the hosting set up on his machine initially
and set up a new user on his machine so that the group could ssh into it and then into the
contest VM’s. Once the machines were up and semisecure Andrew continued to monitor and
improve security and fix breaches through the end of the contest on Saturday at noon.
Contributing significantly to the repairing of locked sudoers files and correcting permission
issues with the wordpress data files. Andrew worked closely with Jacob when the machines
were bricked to get the new ones up and secure as fast as possible. For the final report, Andrew
4
set up the formatting so that everything had a consistent look and feel to it. After that he wrote
up a large amount of the boilerplate sections. Andrew then turned his attention to analysing the
pcap file that was captured during the contest. To this end he used tcpflow and bro to
reconstruct the streams back into their original content for further analysis. Throughout the
entire contest Andrew kept the team up to date on his findings via a group email chain and
posting to the log everyone was contributing to.
3 Problems A lack of proactive hardening and adjustments before the exercise as well as initial connectivity
issues to the CDX server made things problematic during the first hours of the exercise. Once
the issues of connectivity were solved the next issues involved who could best physically host
the VMs and what to do to harden them against attacks. During the contests the continued
stream of attacks brought about urgent and varied responses to the attacks on various systems
of the VMs. A source of difficulty after the contest was finding tools that could handle the large
pcap files that had been generated during the contest.
5
Planning 1 Initial Plan The initial plan was to secure the VM’s ahead of time and set up proper logging and hosting
solutions so as to be up as long as possible during the actual contest. Due to several factors
this did not work out as planned. Primarily it was an issue of miscommunication and incorrect
assumptions being made about team member’s abilities to perform certain actions. Another
contributing factor was that we did not realize how much time it was going to take to initially set
up the VM’s securely. During the first class taking place inside the exercise any attempt at
exporting a hardened VM failed and required the use of VMs without proactive measurements,
having to add these measurements in after the fact.
2 Hosting Initially we thought that Tyler Brisbin was going to host the VM’s, but it turned out that he was
unable to get the VMs properly set up on his Windows machine. A large part of the issue was
the use of Windows as the host OS instead of Ubuntu. This led to a large part of the confusion
at the beginning of the contest. Once we all learned of the problems Tyler was facing we
decided to switch to running the VM’s on Jacob Chesley’s machine immediately before the start
of the contest. Due to unforeseen complications with exporting the VM appliance we were
unable to transfer the machine Tyler had worked on securing over to Jacob’s computer.
Therefore we had to create three brand new VM’s on Jacob’s computer. These three VM’s
remained online for a bulk of the contest. Jacobs computer was chosen since he did not need to
use it and could leave it at home connected to the internet 24/7.
3 The Contest During the contest our plan was simple: continue to try and secure the VM’s using the guide at
www.hardenubuntu.com. Each of us was supposed to log into different VM’s and work on them.
At the same time we were going to monitor the VM’s for any malicious activity. As we saw
anything we would fix it and report it to the other team members. If anything came up requiring
physical access to the machine Jacob Chesley was on call 24/7, since it was his machine
hosting the VM’s. Anything anyone did to a VM was supposed to be recorded in a document
6
hosted on Google Drive by Tyler. Jacob also used wireshark to record pcap data from tap0 on
his host machine for a vast majority of the contest.
4 The Report The report was compiled using Google Drives document functionality. In this way all the team
members were able to contribute to the report at times convenient to them. The report was
greatly aided by a log kept of everything we did during the contest which was another Google
Drive document. Analysis of the pcap data collected proved to be extremely challenging and
only partially aided the team in the finding and understanding of the various attacks.
7
VM Hardening 1 PreEvent Tyler Brisbin was in charge of this and unfortunately was unable to get the VM’s successfully
attached to the CDX network from his machine. The team was also unable to make duplicates
of the VM to put on a separate host machine.
The VM’s provided by Professor Franco had several undocumented misconfigurations. The first
thing done, was to change the passwords from the weak default of ‘student’ to stronger
passwords that would make it harder for the red team to break. Ubuntu allows for randomly
generated new passwords rated at either fair or good; adding further mixed characters (letters of
upper or lower case, symbols, and numbers) created a strongrated password. From there the
website www.hardenubuntu.com contained measures to protect software components from
intruders.
The first steps involved basic “sudo aptget” commands, in order: update, upgrade, autoremove,
and autoclean. These four would update systems on the OS, upgrade installed packages to
newer versions, and remove unnecessary packages which could leave vulnerabilities for
intruders. The remainder of the guide focused on specific applications to install or properly
configure.
2 The Actual Event Once the event began and the team realized they had no secured VM’s to run it became a
group effort to try and quickly get new ones online and secured. To this end we spent most of
the evening Friday after 4:00pm getting the VM’s set up on Jacob’s machine. Once the
UbuntuCDX.OVA was downloaded and installed into 3 seperate VM’s we immediately brought
them online and began to harden.
Unfortunately due to limitations of the virtualized hardware and the setup of the CDX network it
was difficult to connect to both it and the internet at the same time. So we had to take the
machines off the CDX network one by one and run the basic aptget commands to update the
machine. Fearing that we would fall too far behind we did not take the time to install any other
software. This became extremely problematic later in the event as the lack of other tools left
vulnerabilities for the red team to exploit.
8
Once we updated the systems we were each supposed to take a VM and use the guide at
www.hardenubuntu.com to harden the systems. To this end we set up a new user on Jacobs
computer that we could all ssh into. From there we could access the three VM’s without the
network congestion problems observed when trying to log straight into the VM’s (We later
learned the congestion issue was a problem with the MAC addresses). Since the machines
were hosted on Jacob’s machine he was going to bounce between all three.
2.1 10.8.0.220 & 10.8.0.221 The default password of the machine was changed when Jacob initially set them up. No one
else had time to go through and secure these VM’s before we noticed the first attacks on them.
2.2 10.8.0.222 Andrew took this VM and tried to harden it using the www.hardenubuntu.com guide from about
11:00pm Dec. 2nd to 2:00am Dec. 3rd. Here’s a list of the steps that were completed:
Fixed the /etc/sudoers by changing “Student ALL=(ALL) NOPASSWD: ALL” to “Student ALL=(ALL:ALL) ALL”.
Disabled the shell account for “yourmom”. Disabled IPv6 in both /etc/sysctl.conf and /etc/default/bind9 . Attempted to harden the appache.conf, but this led to problems seen later (and rolled
back) by Jacob. Disabled ETAG support. Disabled Trace HTTP Requests. Disabled SymLinks. Attempted to hide header information. Created a new user and group,“webuser”, and forced apache to use it instead of
Student. Attempted to restrict ‘/’ Directory access (Undone by Jacob because it caused
issues with the wordpress). Disabled IP spoofing. Hardened MySQL.
Changed root user to “YouKnowWho” and password to “ThisIsStrongPasswordOne”
Disabled local infile. Stopped SHOW DATABASES from working.
9
Tried to set up ssh keys to disable password login, but could never get it working
correctly.
Andrew sent an email to the team at 1:30am Dec. 3rd letting them know what he had done for
the VM and asking Jacob to test the wordpress stuff on the localhost.
10
Attack Analysis 1 Spooky Skilenton SSH Banner Attack The red team managed to breach the 10.8.0.220 VM and add code to the .bashrc script for the
user students so that it displayed the following image on login from SSH:
1.1 Reconnaissance At this point we did not yet have the logging system active. So we are not 100% sure how they
managed to do this. Our best guess is that the anonymous ftp had access to the student home
folder and they were able to pull the file modify it and replace it.
1.2 Install and Actions on Objectives The attack really did not do anything specifically to harm the system. It seemed to be something
along the lines of a warning shot. The red team was probably able to download other important
files off the system at this time as well as demonstrate that they could gain control of the system
and alter files.
11
1.3 Discovery and Teams Actions to Fix This was discovered around 8:53pm Dec. 2nd when Andrew was testing the SSH setup into the
VM’s from Jacob’s computer. Andrew had a general idea of what was happening and first
checked the sshd_config to see if a banner was being set in it. Upon finding no such banner
listing he checked the .bashrc file to see if anything was running on login and found the code to
display the skeleton image. He removed the code and tested to make sure the image was gone.
It was not until the next day that the team realized the ftp configuration needed to be secured.
We tried to secure the ftp by adding/uncommenting the following lines in the proftpd.conf file
and restarting the service:
DefaultRoot ~ HideUser root HideUser student <Directory *>
<Limit WRITE> DenyAll
</Limit> </Directory>
2 Password Reset Attack We noticed that the password for 10.8.0.221 had its password reset and the desktop wallpaper
was changed to that of the cat below:
12
2.1 Reconnaissance Unfortunately at the time of this attack we did not have the logging system up. From file
modification times we can tell however that the attacks occurred around 11:23pm Dec. 2nd.
2.2 Install and Actions on Objectives The password was reset to its default of ‘student’ to make it easier for the attackers to get back
in and the desktop wallpaper was changed. The image used for the wallpaper was part of a
newly created folder called “Wallpapers”.
2.3 Discovery and Teams Actions to Fix Jacob discovered the attack around 12:40am Dec. 3rd and was able to reset the password back
to the password it had been. The reason the attacker was able to change the password so
easily was because no one had fixed the /etc/sudoers file on that machine to force sudo to
require a password. Jacob realized this and made the following modification to both 10.8.0.220
and 10.8.0.221: “student ALL=(ALL) NOPASSWD:ALL” to “student ALL=(ALL:ALL) ALL”.
Andrew had already made the change to 10.8.0.222 while going through the
www.hardenubuntu.com guide. Jacob also started to capture network traffic occurring on tap0 to
a pcap file for examination at this time.
3 Wordpress Replaced with Cat Picture Attack The wordpress on 10.8.0.220 was displaying nothing, but the following photo of a cat:
13
3.1 Reconnaissance It appears as though the red team was requesting the wordpress site quite often during the time
period of this attack and even managed to bring up the wordpress admin console. Having been
informed that the username was ‘student’ and the password was initially ‘student’ they probably
proceeded with the attack.
3.2 Weaponization of Exploit (Including Delivery) The red team was able to use the admin console of the wordpress page to upload the picture of
the cat and alter the way the site was displaying the information. Since the username/password
were left at the default at this point the attackers just used them to log into the admin console.
3.3 Install and Actions on Objectives This attack replaced the localhost wordpress page with a picture of a cat bringing down the
teams wordpress service.
14
3.4 Discovery and Teams Actions to Fix The attack was noticed by Jacob around 11:30pm Dec. 3rd. Jacob noticed that the file kitten.jpg
had been uploaded to /var/www/html/kitten.jpg and the wpload.php file had been modified
around 2:10pm Dec. 3rd. So he reverted the wpload.php file back to its original contents by
removing obfuscated code that linked to the uploaded kitten image. Jacob then hardened the
wordpress by running the following commands on all three machines:
find /path/to/your/wordpress/install/ type d exec chmod 755 \; find /path/to/your/wordpress/install/ type f exec chmod 644 \;
4 Sudo Removal and Lockout Attack The removal of sudo privileges from the student accounts on the VM and locking out
Wordpress.
4.1 Install and Actions on Objectives The red team gained access to the OS and removed sudo privileges from the student accounts
on the VMs. Before removing sudo the red team modified the permissions on
/var/www/html/wordpress to make it owned and readonly by root. This caused the wordpress
site to display a permission denied message when accessed. Therefore, our wordpress site
appeared as down.
4.2 Discovery and Teams Actions to Fix At 7:52pm Dec. 3rd, Andrew lost sudo ability on all three machines. In order to fix the wordpress
sites we needed to gain access to the sudo file to change the permissions back. After several
failed attempts at remotely solving the problem by Andrew, Jacob eventually arrived home and
was able to reboot the machines into recovery mode where upon he added student back to the
sudoers file and rebooted. His exact steps were:
Turn off the VM Turn on the VM, edit GRUB options under regular kernel, and add “rw init=/bin/bash” at end of line containing linux build. Boot the VM. Run the commands:
passwd student <LONG PASSWORD HERE> passwd root <LONG PASSWORD HERE>
15
Exit the VM. Turn on the VM. Go to linux recovery. Run disk check to enable read write mode. Bring up root shell. Log in with new password from root account above. Run commands:
sudo cp /etc/sudoers /etc/sudoers.backup sudo nano /etc/sudoers
Modify sudoers file and save.
Once rebooted Andrew successfully fixed the wordpress permission errors and everything was
up and running again by 9:00pm Dec 3rd. The commands used to fix the wordpress site were:
sudo chown wwwdata R /var/www/html/wordpress sudo chmod 0755 R /var/www/html/wordpress sudo chmod g+s R /var/www/html/wordpress
5 Nyan Cat Attack Alteration of the GRUB bootloader to show a “nyan cat” and preventing a proper boot loading of
the VM.
5.1 Weaponization of Exploit (Including Delivery) The payload carried the name “nyan.mbr”, transferred via ftp to the VM at address 10.8.0.220.
16
5.2 Install and Actions on Objectives The red team made one final attack on the VMs and installed a modified bootloader. They then
rebooted the machines and the machines bricked.
5.3 Discovery and Teams Actions to Fix By 9:24pm on Dec. 4th, all of the machines had been modified to lock out the blue team. SSH or
attempting to use the terminal was met with a locked screen stating the service had been
disabled. Andrew found through experimentation that the lock screen could be defeated with a
simple ctrlc. At the same time the attackers had installed a new bootloader onto the systems.
While Andrew was trying to regain control of the VM’s through SSH, Jacob was able to briefly
communicate with the red team through an open text editor in one of the VM’s before the red
team decided to show off what they had done and reboot all of the machines.
Fixing the bootloader would have been possible through the use of a secondary VM that could
load the drive of the other VM’s and repair the bootloader section, but it would have been
difficult and taken most of the night. Instead the team received permission from the instructor
(Dr. Franco) to restart with brand new VM’s again and try to keep them alive for the remainder of
the contest. Jacob created the new VM’s over the top of the bricked machines (since we could
no longer access them there was nothing to be gained by keeping them). Then Andrew and
Jacob spent most of the night from 10:00pm Dec. 4th to about 2:00am Dec. 5th securing all
three of the new VM’s. Andrew then spent another couple of hours monitoring them and
attempting to add to their security. No further attacks occurred and the VM’s were live until the
end of the contest at noon saturday Dec. 5th.
17
Mistakes and Other Ideas 1 Mistakes
Verify that people have done what they said and don’t just trust.
Not communicating more extensively before the contest about issues and achievements.
Not taking the time initially to install proper tools for logging on the VM’s.
Underestimating the time it would take to fully and appropriately harden the VM’s.
Not following the instructions when choosing the host OS.
2 Ideas That Never Got Implemented Create hundreds of new users with basically zero access to anything to make it more
difficult for an attacker to figure out who was who.
SSH RSA key authentication instead of passwords.
Make the VM’s accessible on the CDX network and internet at the same time.
Setup logging software on the VM’s themselves.
3 Future CDX Suggestions 3.1 Our Suggestions
Have the contest over a weekend instead of during the week. Perhaps from like noon on
Friday to noon on Monday. That would cause less interference with classes/homework
and students could more fully focus on the contest. It would also interfere less with work
or activities of other courses.
Perhaps have a Q&A lecture with DSU at some point in the semester to get an idea of
what it's like on their end beyond the lessons taught about penetration testing and red
team capabilities.
Create an online forum where the entire class can discuss what they’ve done ahead of
time. Learning from each other and getting feedback from others about various security
precautions they’ve taken.
Fix the score system and make it more transparent.
18
3.2 Class Suggestions Provide the faulty VM at the beginning of the semester.
Lecture on hardening an OS earlier in the semester.
Midterm inclass CDX with the class split into red and blue teams for attack and defense
using the improved VMs.
Make the video one of the labs.
Open up the CDX network so UC teams can attack each other before the actual event
with DSU, serving as a practice round before the main challenge.
19
References www.hardenubuntu.com
http://gauss.ececs.uc.edu/Courses/c6055/labs/final.html
http://gauss.ececs.uc.edu/Courses/c6055/videos/25Nov15_6055.ogv
http://www.servercobra.com/bulletproofproftpd/
http://security.stackexchange.com/
http://www.bro.org
20
Appendix A: Tap0 PCAP Statistics
21
