NSX-T VM Edge … The Geneve Gotcha

In this article we discuss the MTU requirements for the NSX-T Edge, read on or click the link in your Alexa App

This is quite a simple one, but unfortunately I was stung by it recently. If you are new to NSX-T one of the big differences with NSX-V is the centralized services model. With NSX-T we now have the concept of edge nodes, within which we run centralized components. These edge nodes may be deployed as bare metal or virtual machines on top of your hypervisor estate. Note: At the time of writing, ESXi is the only supported hypervisor for VM based edge nodes.

So the gotcha is this, if you are deploying Edge nodes as VMs, the hosts they run may not necessarily be prepared for NSX. As such you may not think you need to increase the MTU on these components. Unfortunately, this isn’t the case. The Edge VMs still participate in the overlay, and for Geneve tunnels. Which means, the VDS that the edges consume, must have the MTU increase to 1600 or more.

Learn from my mistake, do not forget this. In my situation I completely forgot this step, so I had a situation where basic IP connectivity checks were all ok. Ping worked using the standard packet sizes, I didn’t even think of increasing it. Yet when I tried to access my applications I saw timeouts. After two days of troubleshooting, yes two days, a good friend asked the question – Are you sure you increased the MTU on the VDS? You know that feeling, the minute the statement was made, even as it was being made, I suddenly realized what I forgot.

Bal Birdy on LinkedinBal Birdy on Twitter
Bal Birdy
Bal is an Open Group Certified IT Architect, and VCDX #269, specializing in the network and security arena, with over 15 years experience in enterprise level network/system technologies. His goal has always been to maintain a holistic view of the architecture allowing him to understand how various technology streams may impact the networking/infrastructure space.
Bal has a proven record of delivering on enterprise network designs, leading data center and site migrations as a result of business mergers and acquisitions, and vendor migrations e.g. Cisco to Checkpoint/Juniper. As part of this he worked across several business sectors: Utilities, Banking, Retail and Government, and can base designs around sector specific standards e.g. PCI-DSS, DSD and ISM. He is proficient in several technology areas including Cisco, Juniper, F5, VMware, Citrix and Microsoft. These skills are supported by non-technical certifications: Prince2 Project Management Practitioner, ITILv3, TOGAF 9.1 Certified and Open Group Certified IT Architect – Open CA.
In addition to supporting the Livefire Team, Bal leads several innovation efforts within the VMware WRACE organization, including projects investigating the use of Virtual Reality/Augmented Reality, AI/ML and Interactive 360, to support customer and partner enablement.

BSc (Hons) Computer Science
VCDX-NV #269
Open Group Certificated Architect
Member of the Associated of Enterprise Architects

Leave a Reply