VXLANs: Applying Virtualization to Networks.

VXLAN Network: a network between virtual machines, wherever they are

The development of VXLAN technology or extensible virtual networks is usually associated with the implementation of service provider company platforms.

However, the fundamental idea around VXLAN is applicable to any platform and arises to solve a network design problem with which the reader may feel identified.

Many of us have found ourselves in the situation where, having a distributed platform and dealing with a performance problem, we wanted to be able to somehow simplify the design, so that there would be a virtual network between teams located in different data centres.

In order to solve this situation we have the emergence of VXLAN, which proposes the creation of a level ¨virtual¨ of communication layer 2 or extended network. At the end of the day, the idea was to apply the virtualization paradigm to the networking and communications platform.

Today VXLANs, together with other network virtualization tools such as EVPN (Ethernet Virtual Private Network), has impacted on the development of network architecture suitable for service providers.

The resulting platform allows these companies, among others, to serve a large number of customers, offer each of them a good level of logical isolation and establish simple escalation procedures.

On the other hand, VXLAN network has been closely related to networks defined by SDN software. This is no surprise, since both concepts share the fundamental point of being the result of applying the paradigm of virtualization to the world of networks.

In fact, there are authors who consider that VXLAN is a fundamental piece for the effective development of networks defined by software, proposing architectures that include both ideas.

This combination is achieved by taking from SDN the idea of separating the logical part of the networks from the physical part and implementing this physical part considering the precepts of VXLAN.

For readers interested in this SDN – VXLAN relationship we recommend two readings; firstly our post on SDN to resume the basics, and secondly the article published in late 2017 by IEEE which proposes an architecture of SDN based on VXLAN.

The VXLAN solution

VXLAN proposes to generate a virtual network, also called overlay, from a LAN/WAN network infrastructure that will function as a base.

For the overlay network, VXLAN describes the encapsulation protocol and the tunnelling processes required for its operation.

As for the base network, called underlying or underlay, VXLAN presupposes the existence of a network where all active components are layer 3, ie are routers that implement some dynamic protocol such as OSPF, EIGRP or IS-IS.

Both overlay and underlay networks are shown in the following diagram:

About the underlay network

Let’s talk a little more about the base network or underlay.

This network, as we have already mentioned, consists of layer 3 devices, i.e. routers that make optimal use of the available roads or routes.

The choice of the optimal route will be made on the basis of an equal cost multi-path protocol (ECMP).

It is interesting to note that underlay networks rule out the use of STP (Spanning Tree Protocol). Let’s remember that STP’s main objective is to avoid the creation of loops in the transmission of packets.

In a scenario of two possible routes of equal cost, the application of Spanning Tree would result in the annulment of one of the routes with all the cost that this decision supposes. It is precisely this cost that underlay networks are trying to avoid.

Of course, there is technology specially designed to implement underlay networks, such as EVPN (Ethernet VPN), for example, which initially emerged as the solution for any encapsulation scheme and which currently use VXLAN and MPLS.

The reader interested in reviewing the underlay layer and EVPN should read this interesting article on the subject.

About VXLAN overlay networks

Now let’s concentrate on VXLAN.

The segmentation capability brought about by the traditional VLAN concept under the 802.1Q standard has a limitation on the number of VLANs that can be created.

This number is restricted by the size of the VLAN identifier, which is 12-bit, so the VLAN limit is just over 4000 VLANs.

So if you manage the infrastructure of a service provider and decide to deliver 4 VLANs per customer you will have a not very encouraging limit of 1000 potential customers.

To overcome this limit, VXLAN uses a 24-bit word, known as VNI (VXLAN Network Identifier), as a segment identifier, providing just over 16 million possible VXLANs.

On the other hand, VXLAN Network, as we have already said, proposes the creation of a virtual network layer 2, which gives a new level of abstraction.

At this level, the independence between created virtual networks or VNs is maintained. That is, there will be no communication between the elements of different NVs unless a security rule explicitly allows it and a router implements it.

In addition, the possibility of defining a VN between devices located in different data centres is considered.

So we could have two virtual machines located in two different data centres that are integrated into the same virtual VXLAN Network.

In the following diagram look at the two highlighted virtual machines that belong to the same virtual network.

Let’s talk about the components: VXLAN uses elements called VTEP (VXLAN Tunnelling End Point) for its task.

VTEPs provide the connection between the overlay network and the underlay network and effect the traffic encapsulation process.

For this, each VTEP will have an IP address corresponding to the underlay network and one or more VNI addresses corresponding to the extended virtual networks it can handle.

As for the encapsulation, when the packet to be transmitted reaches the VTEP it is encapsulated in the so-called jumbo frames, which contain the original packet from the virtual machine plus the following headers:

• VXLAN header, which contains the identifier of the associated VXLAN
• UDP header, with information on UDP ports of destination and origin. Destination: 4789. Origin calculated within range 49152 and 65535.
• IP header, with the address of the destination VTEP or a Multicast address.
• Ethernet header, which will contain the MAC address of the next physical device on the underlay network. This header will be the only one that will change as the jumbo frame passes through the different routers on the underlay network.

The following image shows the type encapsulation performed by a VTEP:

To complete the transport of VXLAN packets associated with a specific VN, VTEPs must create a virtual tunnel, which will remain active long enough to transfer these VXLAN packets.

To identify the VTEP destination with which it is required to establish the tunnel follow the same operations of any Ethernet network, ie the entire sequence associated with the ARP protocol, including the sending of Broadcast packets.

The difference is that at the moment of receiving the Broadcast ARP packet the source VTEP will encapsulate this packet in a multicast packet associated to the corresponding VNI, so that the destination VTEP responds and the tunnel can be established.

As far as the implementation of VTEPs is concerned, there are two possibilities:

1. Physical devices: You can use devices such as switches or routers that have the VTEP interface.
2. Software elements: Hypervisors can be used to create and maintain virtual machines that run the VTEP software at the same time.

In many implementations you will find a mixture of both possibilities, offering a lot of flexibility considering that not all switches and routers on the market support VTEP functionality.

The reader who wishes to delve into the technical details associated with VXLAN insurance will find the RFC 7348 very interesting.

Challenges associated with VXLAN

The implementation of VXLAN brings with it a new set of elements that can affect the performance of applications; therefore, the adequacy of monitoring processes is vital.

However, the challenges in VXLAN monitoring start with the loss of visibility that involves having two overlapping networks and an encapsulation process prior to the use of the entire communications structure.

Considering this, we can foresee that it will be difficult to isolate the traffic being monitored, as well as to correlate correctly the traffic generated in the overlay network with the corresponding traffic in the underlay network.

On the other hand there is the work with the jumbo frames, the fragmentation processes and the processes of encryption and decryption. We have to consider all the negative effect that these elements could have on the performance of the platform.

Another interesting point to evaluate in detail is the cost in time and resources involved in the creation of tunnels, since as provided by the definition of VXLAN this is a dynamic and recurrent process.

So in a VXLAN platform it would be interesting to monitor the multicast packets generated by the VTEPs, ARP tables and the time of setting up the tunnels.

We believe it would be opportune to resume the effect that Broadcast and multicast packages could have on the performance of the platforms, for which we recommend that you review the following post.

Finally, monitoring the VXLAN platform would not make sense in isolation; it is essential to consider the platform as a whole and adapt the monitoring processes to this structure.

As the natural target of VXLAN platforms is the service provider company, which usually have complex platforms where automation, scalability and flexibility are fundamental objectives, it is crucial to have adequate monitoring tools.

For these cases Pandora FMS has developed a solution designed just for the service providers, and of course we invite you to investigate more about all their capabilities and advantages.

To do so, we recommend that you visit this link, request all the information you require and tell us about your project.