This blog post is a continuation of a series of posts in which we try to educate you on the numerous services, analysis and consulting that Vestige has performed in the past and that we can perform for you and your client. In this post we’ll discuss host analysis of a very rare sort.
In order to understand this type of analysis, here’s a little background. I’m sure some of you are familiar with virtual machines. In short, a virtual machine is a computer that looks feels and acts like a normal computer but is not tied to any physical hardware. Of course, it has to run on a computer, but it runs in its own virtual universe on a computer. That universe can be tied to your network or it can be on its own network. A physical computer (usually a server) can run multiple virtual machines. But those virtual machines can be moved around from one computer to another (a strong benefit of virtual machines). The physical computer on which one or more virtual machines run is called a Host computer. The vast majority of host computers handle virtual machines that are compatible with VMWare or Hyper-V. One of the types of host machines for VMWare is called an ESXi server and it is that which we will discuss.
Our client was one that had multiple ESXi servers running various virtual machines. While they had quite a robust IT environment with controls, unfortunately an ESXi machine was placed on their network unprotected from most of their standard network protections such as firewalls. As a result, their ESXi server was identified in a Distributed Denial of Service (DDOS) attack. The client’s desire was analysis to determine whether the ESXi server was exposed to other issues, such as being hacked.
Typically in incident response cases and investigations of hacking, it is not the host server (such as ESXi) that is analyzed but instead it is the virtual machines which typically run. As a result, information on the preservation and forensic analysis of ESXi host servers was virtually non-existent.
Vestige therefore turned to its investigation, research and testing skills to come up with a process to analyze such a server. Research showed that an ESXi server, which not truly running on Linux, does share some properties with Linux machines. Digital forensic analysis of Linux machines is a known process in the forensic world, so that’s a great start! Vestige also set up a test environment with an ESXi server running the same version of software as the ESXi server in question. Testing provided information on how the operating system loaded which helped Vestige understand how a hacker could remain persistent on the server.
A majority of the testing, however, involved comparing files and activity on the server being investigated with a known good server. This methodology is one used often at Vestige. When we are able to classify and document how a known good program or device operates, any activity outside of that ‘normal’ realm that occurs during an investigation needs to be further reviewed to determine if it is malicious or not. The same goes for file comparisons.
Vestige – your Technical Experts including unusual analysis requests
In a nutshell, Vestige doesn’t back down from unusual analysis or analysis involving rare situations. In reality, this is an area that Vestige is often called upon, even by other forensic companies. As a customer of ours once said “I’m not sure if Vestige has ever dealt with this, but I know that they’ll have an answer.” I hope this blog shows that no matter what the situation, it pays to just give Vestige a call and see how our team of digital forensics experts may help with ESXi server security and data breach incident response.
by Greg Kelley, EnCE, DFCP, Chief Technology Officer at Vestige Digital Investigations
For more information CONTACT US.