Are you new to NSX-T and wondering why your TEP routing seems to be skewed and what to fix it? Read on or click the link in your Alexa App.
One of the big benefits of NSX-T is that it has multi-hypervisor support with ESXi and KVM. The two KVM derivatives supported are based on RHEL and Ubuntu, but there are some intricate differences with these, vs ESXi. If you are in your early stages of setup you’ll be setting up your tunnel endpoints, or TEPs. In an NSX-V world, these were what we fondly called the VTEP. Now where ESXi has advantages of KVM, at the networking level, are around the dedicated TCP/IP stacks that can be deployed for different VMkernel traffic types, e.g. VMotion and VXLAN. Unfortunately, this is not something that some KVM implementations have so you need to add static routing to make sure TEP communication is happening in the most optimal way possible.
Note: this is only important if you have remote TEPs on a different IP subnet e.g. if you have a mix of KVM and ESXi clusters, or your KVM TEPs are split across different subnets.
To achieve this goal we can add static routes to our KVM hypervisor to mitigate this issue. A big gotcha though is that this isn’t necessarily a persistent configuration. To make it consistent you have to update the /etc/network/interfaces on Ubuntu, or the equivalent in RHEL. Within this file, you can then add the static route, which on a reboot of your KVM host, will become active and persistent.
- Connect to your KVM host via your preferred SSH client, and run ifconfig to confirm your TEP inteface name
- Edit /etc/network/interfaces adding the static route to the remote TEP subnet, utilizing the KVM TEP interface, found previously, in the route command
- Reboot your KVM host and confirm your route is present in the routing table and validated by pinging another host on a different segment
I hope this helps in your base setup, feel free to comment and good luck learning NSX-T.
Bal has a proven record of delivering on enterprise network designs, leading data center and site migrations as a result of business mergers and acquisitions, and vendor migrations e.g. Cisco to Checkpoint/Juniper. As part of this he worked across several business sectors: Utilities, Banking, Retail and Government, and can base designs around sector specific standards e.g. PCI-DSS, DSD and ISM. He is proficient in several technology areas including Cisco, Juniper, F5, VMware, Citrix and Microsoft. These skills are supported by non-technical certifications: Prince2 Project Management Practitioner, ITILv3, TOGAF 9.1 Certified and Open Group Certified IT Architect – Open CA.
In addition to supporting the Livefire Team, Bal leads several innovation efforts within the VMware WRACE organization, including projects investigating the use of Virtual Reality/Augmented Reality, AI/ML and Interactive 360, to support customer and partner enablement.
Certifications:
BSc (Hons) Computer Science
CCNP/CCDP
VCDX-NV #269
Open Group Certificated Architect
Member of the Associated of Enterprise Architects