NSX-T VM Edge … The Geneve Gotcha

In this article we discuss the MTU requirements for the NSX-T Edge, read on or click the link in your Alexa App

This is quite a simple one, but unfortunately I was stung by it recently. If you are new to NSX-T one of the big differences with NSX-V is the centralised services model. With NSX-T we now have the concept of edge nodes, within which we run centralised components. These edge nodes may be deployed as bare metal or virtual machines on top of your hypervisor estate. Note: At the time of writing, ESXi is the only supported hypervisor for VM based edge nodes.

So the gotcha is this, if you are deploying Edge nodes as VMs, the hosts they run may not necessarily be prepared for NSX. As such you may not think you need to increase the MTU on these components. Unfortunately, this isn’t the case. The Edge VMs still participate in the overlay, and for Geneve tunnels. Which means, the VDS that the edges consume, must have the MTU increase to 1600 or more.

Learn from my mistake, do not forget this. In my situation I completely forgot this step, so I had a situation where basic IP connectivity checks were all ok. Ping worked using the standard packet sizes, I didn’t even think of increasing it. Yet when I tried to access my applications I saw timeouts. After two days of troubleshooting, yes two days, a good friend asked the question – Are you sure you increased the MTU on the VDS? You know that feeling, the minute the statement was made, even as it was being made, I suddenly realised what I forgot.

Bal Birdy

Leave a Reply

%d bloggers like this: