When is network introspection not network introspection

If you’ve been digging into the guest introspection setup, and spotted the network introspection driver, this article will explain it’s purpose and how it differs to network introspection.

So what am I talking about? Well a colleague of mine was asking about an explanation of guest introspection (GI) vs network introspection (NI), and as they were digging into the intricacies they got stuck on the below image.

So let’s step back, the problem is simply a bit of naming and branding. Network introspection drivers, as installed by VMware Tools, has nothing to do with Network Introspection in the traditional sense. Time for another image:

Ok, so from the following image you can see that introspection services have been broken down into two distinct types, guest and network introspection. Now note the services that are provided by each, data security has been deprecated which is why it’s crossed out, let’s focus on GI for now.

So with GI we support these via either VMware Services or 3rd Party Services. Now here’s the thing, with Activity Monitoring and Identity Firewalling we ultimately need to know the IP address of the VM, as we perform actions based on this information. With Agentless AV/File Integrity we look at storage I/O.

So before we delve deeper, another graphic to look at:

GI is made up of three components –

Thin Agent:

Inside the Virtual Machine Guest OS we need to install Thin Agent (This is actually just an add-on to the existing VMTools drivers). This agent will communicate with the “MUX” via Virtual Machine Communication Interface (VMCI) Channel.

MUX Module:

When we enable Guest Introspection on the cluster level, NSX will install a new VIB (vSphere Installed Bundle) on each ESXi host in our cluster. The new VIB is called “epsec-mux” and MUX module acts as a Multiplexer accepting messages from the Thin Agent running in the VM via VMCI channel, then delivers the information to the SVM (Service Virtual Machine) via a TCP session.

As a part of Guest Introspection deployment, a new standard switch called “mservice-vswitch” is created at each host, and a new port group named “vmservice-vswitch” will be used for the communication.

Service Virtual Machine – SVM:

On each host in the cluster that we’ve enabled “Guest Introspection” on, NSX will deploy a Service Virtual Machine (SVM).  The SVM will offload the activity needs to be done inside the VM Guest OS and communicate with NSX Manager to send updates and to get new configurations. The SVM have two interfaces: one interface is connected to the standard vSwitch for the MUX traffic, and the Second interface to the management IP defined by the admin to enable the communication to the NSX manager. This communication is done via vDS.

Now to start bringing this all together. Have a look at the following overviews for two use cases with Guest Introspection.

General flow of a Guest introspection – Filescan

  • A file needs to have an action performed on it, read, write, create, etc.
  • The Thin Agent sees that there is an action and locks the file, scans the file, and send it up to the SVA for further investigation.
  • The Thin Agent communicates to the epsec-mux driver on the ESXi host through VMCI to pass this information onward.
  • The SVA communicates to the epsec-mux driver on the ESXi host through TCP/IP and scans the file, provides information on the contents, then sends back information.
  • Once information is gathered on the SVA, the SVA tells the Thin Agent to either delete or ignore the file.

General flow of a Guest introspection – Identity Firewall 

  1. User login to a Virtual Machine (The VM can be part of Virtual Desktop Infrastructure-  VDI) with AD credentials.
  2. On each VM the thin agent (This is VMTools with Guest Introspection installed) running inside the guest os, detects the login event and update the Service Virtual Machine (SVM) with a user id and IP address combination.
  3. The SVM update the NSX manager with these details user details plus AD Group Memberships.
  4. The NSX Manager will store the user id and the IP address inside its database.
  5. NSX manager will then translate the Security Group to VMs and understand where the NSX firewall policy needs to be pushed.
  6. NSX will push the firewall policy to the related VM in the relevant security group.
  7. The user will be able to access the application inside the datacenter from his VM.

Ok but still why the network introspection driver? See below.

The vsepflt.sys is the only driver required for normal Antivirus protection. The vnetflt.sys or vnetWFP.sys (for windows 10 and above) driver is called the NSX Network introspection driver. This driver captures networking events such as AD login/logout and all other normal networking traffic. This driver can be safely removed and does not effect AV if the AV is not configured to use network introspection.

So as you can see, what we are capturing is network events within the VM, this is outside of the visibility of the hypervisor, where the other type of Network Introspection is happening. So as a result we need this driver, who in turn needs to communicate this information to the NSX Manager, but ultimately via the GI-SVM.

In the next article, we’ll talk about proper Network Introspection, and how steering of traffic works.


Bal Birdy

Leave a Reply

%d bloggers like this: