Network Forensics

Understanding Network Forensics Analysis in an Operational Environment

Memory Forensics

Detecting Malware With Memory Forensics

Android Forensics

Andriller - Android Forensic Tools

Sunday, December 20, 2015

Cuckoo Sandbox :: Installation Without Errors

System Requirements

This is what you will need as a minimum:
Essential
  • Reasonably powerful x86 hardware. Any recent Intel or AMD processor should do.
  • 4GB RAM (this is dependent on how many virtual machines you plan to run)
  • Large HDD (virtual machines can take up a lot of space)
  • Ubuntu LTS (14.04 is recommended)
Optional
  • x86 CPU capable of supporting Intel VT-x (or AMD-V for AMD CPU's)
    • This is to enable support for 64 bit virtual hosts
    • You need make sure the extensions are enabled in BIOS (they are usually disabled by default)

Preparing the system for Cuckoo

Update operating system


To make sure your Ubuntu system is fully up to date, open Terminal and run the command:
$ sudo apt-get update && sudo apt-get dist-upgrade -y
Or run the Software Updater (click on Ubuntu icon in the taskbar in top left corner) and type Software Updater and click on the icon:
Select Updater
It will check for updates and if any is found, it will present you with this option:
Software Updater

Select "Install Now". If it ask you to reboot, select "Reboot Now".

Virtualization Software


Cuckoo will work with VMWare Workstation, Oracle VirtualBox and KVM. For this guide, we will use Oracle VirtualBox.
To install, open Terminal and run the following commands:
sudo -s
echo deb http://download.virtualbox.org/virtualbox/debian trusty contrib >> /etc/apt/sources.list 
wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | apt-key add - 
apt-get update && apt-get install virtualbox-4.3 dkms 
adduser your_username vboxusers
exit
This will install the latest version of VirtualBox from Oracle (4.3.20 r96996 at time of posting) and add your primary account to the vboxusers group, allowing you to use the VirtualBox host kernel modules (vboxdrv, vboxnetflt and vboxnetadp). Log out and back in before using VirtualBox.

Required software for Cuckoo Sandbox

Installing Essential Packages


Cuckoo require at a minimum these packages:
  • Python 2.7
  • SQLAlchemy
  • Python BSON
Ubuntu 14.04 does come with Python 2.7 by default, to confirm, open Terminal and run the command:
ubuntu:~$ python --version
You should get Python 2.7.6 as the output. If it say command not found, run the command:
sudo apt-get install python-minimal
Then run the previous command again to confirm.
Next install SQLAlchemy and Python BSON:
sudo apt-get install python-sqlalchemy python-bson

Installing Optional Packages


The optional packages are:
  • Dpkt (Highly Recommended): for extracting relevant information from PCAP files.
  • Jinja2 (Highly Recommended): for rendering the HTML reports and the web interface.
  • Magic (Optional): for identifying files’ formats (otherwise use “file” command line utility)
  • Pydeep (Optional): for calculating ssdeep fuzzy hash of files.
  • Pymongo (Optional): for storing the results in a MongoDB database.
  • YARA and Yara Python (Optional): for matching Yara signatures (use release 1.7.2 or above or the svn version).
  • Libvirt (Optional): for using the KVM machine manager.
  • Bottlepy (Optional): for using the api.py or web.py utility (use release 0.10 or above).
  • Django (Optional): for using the web interface (use release 1.5 or above).
  • Pefile (Optional): used for static analysis of PE32 binaries.
  • Volatility (Optional): used for forensic analysis on memory
  • MAEC Python bindings (Optional): used for MAEC reporting (use a release >=4.0, but <4.1).
  • Chardet (Optional): used for detecting string encoding.
Most of them can be installed via Ubuntu's repository. Open Terminal and run the command to install all of these packages or edit according to your needs:
sudo apt-get install python-dpkt python-jinja2 python-magic python-pymongo python-gridfs python-libvirt python-bottle python-pefile python-chardet volatility
Two packages (Pydeep & YARA) cannot be installed via the repository and must be installed manually.

PYDEEP


Pydeep depends on ssdeep 2.8+. Ubuntu's repository for ssdeep is version 2.7 so this will not be installed. instead ssdeep need to be compiled from source and Pydeep via Python PIP.
Install ssdeep from source
ubuntu:~$ sudo apt-get install build-essential git python-dev -y
ubuntu:~$ wget http://sourceforge.net/projects/ssdeep/files/ssdeep-2.12/ssdeep-2.12.tar.gz/download -O ssdeep.tar.gz
ubuntu:~$ tar -xf ssdeep.tar.gz
ubuntu:~$ cd ssdeep-2.12
ubuntu:~$ ./configure
ubuntu:~$ make
ubuntu:~$ sudo make install
ubuntu:~$ ssdeep -V
2.12
Install pydeep via pip
sudo apt-get install python-pip -y
sudo pip install pydeep
The current stable release of pydeep is version 0.2.
ubuntu:~$ pip show pydeep
---
Name: pydeep
Version: 0.2
Location: /usr/local/lib/python2.7/dist-packages
Requires:

YARA


Prep the system first with the command:
sudo apt-get install build-essential git python-dev libjansson-dev libmagic-dev libtool eclipse-cdt-autotools -y
Download the latest YARA fromhttps://github.com/plusvic/yara/releases/latest which at time of writing is 3.2.0.
tar -zxf yara-3.2.0.tar.gz
cd yara-3.2.0./
./bootstrap.sh
./configure --enable-cuckoo --enable-magic
make
sudo make install
Then to install the yara-python extension (do not do this until you have installed YARA as above first):
cd yara-python
python setup.py build
sudo python setup.py install

Configure tcpdump


Cuckoo is run under a standard user account. To allow Cuckoo to use tcpdump without root access, run the command in Terminal:
sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump
To confirm this is working, under a standard user (ie your own account), this is the expected output:
getcap /usr/sbin/tcpdump
Expected output should be:
/usr/sbin/tcpdump = cap_net_admin,cap_net_raw+eip

Install Cuckoo Sandbox


The last step is to install Cuckoo itself. This is quite very simple. Decide on a location where you want Cuckoo to go such as in /opt. In this case we'll be using ~/bin:
mkdir ~/bin
cd ~/bin
wget http://downloads.cuckoosandbox.org/cuckoo-current.tar.gz
tar -xf cuckoo-current.tar.gz
This will create a new directory cuckoo in ~\bin and Cuckoo is now ready to use. To start Cuckoo:
cd ~/bin/cuckoo
./cuckoo.py
However at this stage, Cuckoo will give an error about not finding the bind server, which is to be expected. The next step is to build your virtual machine and configure Cuckoo to use this virtual machine.

Creating a Virtual Machine


This guide is based on creating a Windows XP SP3 virtual machine however Cuckoo does work with Windows Vista and Windows 7. Please note that if you decide on using Windows Vista or Windows 7, you need to disable UAC.
Some assumptions:
  • You have a licenced copy of Windows XP
  • You have the installation media available as an ISO format on the computer
  • You do know how to install Windows XP

Installing Windows XP


It is very important you follow the instructions below, especially regarding the installation of Python and the Cuckoo agent file
Open VirtualBox via the Ubuntu icon in top left corner:
Opening VirtualBox
Click on the Oracle VM VirtualBox icon as shown above.
Click on New icon in the toolbar.
The following screenshots shows some suggested options for creating a Windows XP SP3 image:
Name and operating system
Make sure if it is 32 bit or 64 bit version of Windows XP to be used.
Memory size
512MB is allocated. Assign more if necessary - ie you are planning on testing malware attacking a number of applications such as Microsoft Office.
Hard driveHard drive file typeThe default option is VDI.
Storage on physical hard driveFile location and sizeVirtual Machine settings page
The last step is to add an ISO image of Windows XP SP3. Make sure the boot order is for CD first (this is the default option).
Adding an ISO
Select the option "Choose a virtual CD/DVD disk file..." On starting the VM, continue to setup Windows.
Installing Windows XP
In order to have a working network connection for the Windows XP image - change the network adaptor to PCnet-FAST III and select Host-Only networking.
Network settings
Install the VirtualBox Guest Additions - this install the necessary network and video and other hardware drivers.
Guest Additions

The guest need to be prepared before Cuckoo can work with it. It need a copy of Python 2.7 to be running on the guest. Install the Python Imaging Library as well - this allows for the taking of screenshots.

NETWORKING


The installation of the virtual machine isn't done at this point however the virtual network is vital so this section focus on the correct setup of the network:
Make sure you are using Host-Only networking on the virtual machine.
Assumptions:
  • ON YOUR MACHINE ETH0 IS YOUR OUTGOING INTERFACE
  • VBOXNET0 IS YOUR VIRTUAL INTERFACE
  • 192.168.56.0/24 IS YOUR SUBNET ADDRESS FOR VIRTUALBOX HOST-ONLY NETWORK - THIS IS THE DEFAULT SUBNET
Setup iptables with forwarding and filtering on the Host. Run the command in Terminal:
sudo iptables -A FORWARD -o eth0 -i vboxnet0 -s 192.168.56.0/24 -m conntrack --ctstate NEW -j ACCEPT
sudo iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT sudo iptables -A POSTROUTING -t nat -j MASQUERADE sudo sysctl -w net.ipv4.ip_forward=1
By using iptables - you can control the network traffic that is going to the virtual machine here.
The IP address of the test virtual machine is assigned 192.168.56.10 and gateway to 192.168.56.1 and DNS to 8.8.8.8 (Google Public DNS - but use any of your choice such as 208.67.222.222 from OpenDNS or your own).
Windows Network Details
Note: The Host system will have the IP address of 192.168.56.1 on the vboxnet0 interface
ubuntu:~$ ifconfig vboxnet0 Link encap:Ethernet HWaddr 0a:00:27:00:00:00 inet addr:192.168.56.1 Bcast:192.168.56.255 Mask:255.255.255.0 inet6 addr: fe80::800:27ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:184 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:28538 (28.5 KB)
(Note: In some cases, Python.org will not load in Internet Explorer 6 on Windows XP SP3. It is easier to download the files needed on the host system and use a Shared Folder to transfer it over to the guest or if you plan on installing Firefox or other browsers, do this and then download Python)
Turn off Windows Firewall and Automatic Update (Automatic Update will not work as Windows XP is not supported by Microsoft). Also disable the popup warning about these by going to Windows Security Center and selecting the option "Change the way Security Center alerts me". Untick all boxes as below:
Disable security alerts in XP
The last step is to install the Cuckoo Agent which runs a XMLRPC server, listening for connections. This file is found in the~bin/cuckoo/agent folder and is named agent.py. When this is run, it shows a Python window. Renaming the file to agent.pyw will not show the window.
Start agent.py on Windows and minimise the window (or if you renamed the file to agent.pyw check that it is running via Task Manager - the process name will be pythonw.exe)
Optional: Place the agent in the Startup folder of the guest system so that it will start automatically with Windows boot up.

Creating the snapshot


Snapshots must be created correctly for Cuckoo Sandbox so make sure you follow this exactly. Make sure that the virtual machine is running first! You will need to run these commands in Terminal:
vboxmanage snapshot "[VM Name]" take "[Snapshot Name]" --pause
vboxmanage controlvm "[VM Name]" poweroff
vboxmanage snapshot "[VM Name]" restorecurrent
In this case - our virtual machine is named WindowsXP_SP3. We also have a barebones installation of Windows so I will name the snapshot "barebones" so this is the commands used:
vboxmanage snapshot "WindowsXP_SP3" take "barebones" --pause
vboxmanage controlvm "WindowsXP_SP3" poweroff
vboxmanage snapshot "WindowsXP_SP3" restorecurrent
You can install various software such as Chrome, Firefox, Adobe PDF Reader and so on for testing malware with Cuckoo and create a new snapshot under a different name for Cuckoo to use.
Bear in mind that only one instance of a VM can run regardless of how many snapshots it have. If you wish to use multiple virtual machines, you can clone the virtual machine and name it differently and Cuckoo is able to use multiple virtual machines. IMPORTANT: Make sure you do change the static IP addresses in any clones you create
This image can be cloned if additional virtual machines are needed but make sure the static IP addresses are changed.
Be sure to make note of the following information:
  • Virtual machine name
    • In our case it's WindowsXP_SP3
  • Snapshot name
    • In our case it's barebones
  • Host's virtual IP address (vboxnet0)
    • In our case it's 192.168.56.1
  • Virtual machine's IP address
    • In our case it's 192.168.56.10

Configuring Cuckoo Snapshot

It's time to configure Cuckoo to work with your installed Virtual Machine.
3 configuration files need to be edited for Cuckoo. We'll start with the simple one first:

[cuckoo.conf]


We need the Host's virtual IP address (vboxnet0) - in our case it's192.168.56.1
Run the command:
gedit ~bin/cuckoo/conf/cuckoo.conf
Edit the following lines as followed:
machinery = virtualbox
memory_dump = on
[resultserver]
ip = 192.168.56.1
port = 2042
Memory dump is done using Volatility and by default this is set to off so if you don't want to capture a memory dump, leave this as off.
All other options to be left as default.

[auxiliary.conf]

First we need to know where tcpdump is located and using this command:
whereis tcpdump
Will tell you the location of tcpdump. On Ubuntu 14.04 - this should be /usr/sbin/tcpdump.
The command will open the auxiliary.conf file in the Text Editor:
gedit ~/bin/cuckoo/conf/auxiliary.conf
Edit the file as below - making sure the tcpdump line is the correct location:
[sniffer]
# Enable or disable the use of an external sniffer (tcpdump) [yes/no].
enabled = yes

# Specify the path to your local installation of tcpdump. Make sure this
# path is correct.
# You can check this using the command: whereis tcpdump
tcpdump = /usr/sbin/tcpdump

# Specify the network interface name on which tcpdump should monitor the
# traffic. Make sure the interface is active.
# The ifconfig command will show you the interface name.
interface = vboxnet0

virtualbox.conf


We are using VirtualBox so we need to edit the~/bin/cuckoo/conf/virtualbox.conf file.
If you are using a different virtualization software such as VMware Workstation - the file would be vmware.conf and the machineryline in cuckoo.conf file will read as machinery = vmware.
With the following information:
  • Virtual machine name
    • In our case it's WindowsXP_SP3
  • Snapshot name
    • In our case it's barebones
  • Host's virtual IP address (vboxnet0)
    • In our case it's 192.168.56.1
  • Virtual machine's IP address
    • In our case it's 192.168.56.10
Open the file for editing:
gedit ~/bin/cuckoo/conf/virtualbox.conf
Edit the file as followed:
machines = WindowsXP_SP3

[WindowsXP_SP3]
# Specify the label name of the current machine as specified in your
# VirtualBox configuration.
label = WindowsXP_SP3
platform = windows
ip = 192.168.56.10
snapshot = barebones
tags = windows_xp_sp3,32_bit,testing,university,minimal_installation

Starting Cuckoo Sandbox

Check if vboxnet0 is up and running:
ifconfig vboxnet0
You should get this output:
vboxnet0  Link encap:Ethernet  HWaddr 0a:00:27:00:00:00  
      inet addr:192.168.56.1  Bcast:192.168.56.255  Mask:255.255.255.0
      inet6 addr: fe80::800:27ff:fe00:0/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:547 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:0 (0.0 B)  TX bytes:93229 (93.2 KB)
If you get this:
vboxnet0: error fetching interface information: Device not found
Or it is not showing any IP address for vboxnet0 then run the commands to reset vboxnet0:
vboxmanage hostonlyif remove vboxnet0
vboxmanage hostonlyif create ipconfig vboxnet0 --ip 192.168.56.1
And check if vboxnet0 is showing up now.

Start Cuckoo


To start Cuckoo, you must be within Cuckoo's directory, so change the current directory to ~/bin/cuckoo first as below:
cd ~/bin/cuckoo
./cuckoo.py
You will see this output: NB: the command also shows an ASCII artwork which is not displayed
Cuckoo Sandbox 1.1
www.cuckoosandbox.org
Copyright (c) 2010-2014

   Checking for updates...
   Good! You have the latest version available.

  2014-11-05 23:07:42,599 [lib.cuckoo.core.scheduler] INFO: Using "virtualbox" machine manager
  2014-11-05 23:07:42,830 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s
  2014-11-05 23:07:42,831 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks...
Cuckoo is now running and ready for use!

Starting a Malware Analysis


There are three main ways to submit tasks to Cuckoo - via command line, simple web interface and Django web application. In all cases - you are able to add tasks to Cuckoo without starting Cuckoo Sandbox itself - it will simply be queued up and when Cuckoo is started, it will automatically start processing all queued tasks. This can be useful if you have many malware samples you want to test. You can then submit them all to Cuckoo and then start up Cuckoo Sandbox in the evening for processing all tasks overnight.

Command Line Method


In a separate terminal you can use the command:
~/bin/cuckoo/utils/submit.py --help
to see all the available options.
The most simple method to submit a malware is to simply issue the command:
~/bin/cuckoo/utils/submit.py <path to malware>
And Cuckoo will automatically start analysing the malware. The virtual machine will automatically start up using the selected snapshot and insert the malware onto the machine and then execute it. This will continue until Cuckoo reach its timeout limit at which point it will then generate a report.
You can also submit URL's via this to Cuckoo - it is able to run an analysis on a machine when it is visiting a website that is known to hijack a victim computer by attempting to inject malwares just by visiting a infected page.
All reports are then found in ~/bin/cuckoo/storage/analysis within directories named after their task ID (ie Task ID 1 would mean reports are in ~/bin/cuckoo/storage/analysis/1 and you will find the report in HTML and JSON format in the reports directory).

Simple Web Interface


Cuckoo Sandbox come with a python script able to show a simple web interface. This web interface can only be used to submit malware binaries but not URL's. To start it, in a separate terminal, run:
~/bin/cuckoo/utils/web.py -H localhost -p 8080
-p 8080 is port number so does not have to be 8080.
Open a web browser and navigate to http://localhost:8080
Simple web interface
Use the form to submit your malware.
If you want to submit more malware, go back to the form by clicking on Home at the top menu. Clicking on Browse will show you a list of all malware analysis tasks currently running or completed:
Simple web interface with task list
All completed jobs will have their MD5 hash (under Target) changed into a link to their report.html file. In the report.html file, you can save the page - images are embedded within the HTML source itself using base64 encoding so it is just a single file, making the report file highly portable and readable on any browsers too.

Django Web Interface


This is more useful for a large scale use of Cuckoo Sandbox - for example within an office network, allowing multiple users to use Cuckoo Sandbox within a network. In addition, you can also submit URL's for Cuckoo to run an analysis on.
You need the Django and the Mondodb packages to be installed so run this command:
sudo apt-get install python-django mongodb -y

The quick method


The Mongo reporting module need to be enabled in the filereporting.conf so open up Terminal and do:
gedit ~/bin/cuckoo/conf/reporting.conf
You want to edit the section [mongodb] to change the enabled = no toenabled = yes as below:
[mongodb]
enabled = yes
host = 127.0.0.1
post = 27017
Next, you need to edit the local_settings.py file under thecuckoo/web/web/ directory:
gedit ~/bin/cuckoo/web/web/local_settings.py
The lines you'll want to change are:
LANGUAGE_CODE = "en-us"  <-- change to en-gb

ADMINS = (
# ("Your Name", "your_email@example.com"),  <-- Add your name & email here
)

Starting the web interface


To start the Django web interface, you must run the manage.py from within the web directory, so change to that directory before running it as below:
cd ~/bin/cuckoo/web
./manage.py runserver
This will start the Django web interface on localhost, port 8000. Go tohttp://localhost:8000 in a browser to view. To run this as a webserver for any network interface (so a local network can access it ie for across an office), do this instead:
./manage.py runserver 0.0.0.0:PORT
(NOTE: If you want to use a port number lower than 1024 such as port 80 (default port for HTTP), you need to use the sudo command)

Using the Django web application


Dashboard: Going to the website, the first page you see is the Dashboard, showing an overall summary
Dashboard
Submitting tasks Clicking on Submit take you to a form to upload malware binaries to analyse. You can click on the URL link to submit an URL to analyse instead.

submit task

Pending tasks This show all queued tasks waiting to be processed by Cuckoo Sandbox.

Pending Tasks

File list This show all tasks that are pending/running/completed for malware binaries

completed files

URL list This show all tasks that are pending/running/completed for URL links WARNING: DO NOT VISIT THE URL SHOWN IN IMAGE ON YOUR PC - UNSAFE WEBSITE: very graphic content and may compromise and/or crash your machine

Completed URLs

Search You can search for past reports. The screenshot show what you can search for if you click on the "Help" link on the page - as you can see, it can search for a wide range of items to help you to narrow down what you are looking for.

search

Reset Cuckoo Sandbox


If you ever need to delete out all old reports and flush the Mongo database containing pending/completed tasks - there is a utility to do this. First make sure that cuckoo.py and any web interfaces are no longer running.
First make sure that you are in the cuckoo or cuckoo/utils directory - this will fail if you are not.
cd ~/bin/cuckoo/utils/
./clean.sh

Further information

Remember to consider this only a quick start guide at most. Cuckoo Sandbox is a powerful tool and it can do a lot more as it's very highly scalable. This guide is mainly aimed for a single user on a single computer running analyses for their own purpose. For any further assistance, please do check out the Cuckoo Sandbox documentation and the community website where people have asked questions about Cuckoo and there are some useful details in there too. You can also ask questions, in there after creating a free account.