VLAN External Network in Openshift using NMState Operators and Multus

NMstate operators and Multus CNI provide us with plugins and APIs, to set up external network in openshift container ecosystem. Let us see how we can implement a external VLAN networking for certain pods/VMs apart from, pods/VMs using internal cluster networking.

When we install NMState operator, it avails us 3 different APIs, namely:

• Node Network Configuration Policy : Node level network settings implementation.

         # oc get nncp

• Node Network State : The current state of node network is stored in this.

        # oc get nns/<node-name>

• Node Network Configuration Enactment : The status of NNCP implementation across cluster nodes, is checked in this object.

        # oc get nnce

• Network Attachment Definition (NAD) : Apart from above APIs, we will also make use of NAD from multus CNI plugin framework. This implements namespaced object that exposes existing layer 2 network devices such as bridges and switches to VMs and Pods.

And then, Multus Operator installation, comes with various plugins that will help us implement networking options for cluster node's network interface as NNCP for configuring 

• VLAN
• IPVLAN
• McVLAN
• Bridge Networking, on these cluster nodes.

Below image shows the architecture components involved in running an external network besides default pod networking in OCP platform. Such cluster nodes or servers, with more than one network connections are called multihomed. Below architecture applies when Openshift Virtualization Operator, Multus CNI and NMState operators are installed.

Now, let us see, how these come together to create a VLAN external network in our case.

VLAN works in 2 different ways: 

A> With Bridge Filtering the VLAN traffic, which is default mode for bridge as shown below.

 

 

I> Create a NNCP yaml manifest for implementing a Linux Bridge, where in VLAN filtering are enabled by default. Lets call below yaml file as linuxBridge.yaml.

 

# oc apply -f  linuxBridge.yaml    # <<< Create the bridge component

Note: NNCP are cluster wide resource. And affect all the nodes in cluster. To limit the effect of NNCP to limited nodes in cluster, we can make use of  annotations and nodeSelectors keys and values field.

 II> Implement a Network Attachment Definition (NAD), which are namespace specific objects, with required VLAN id in it and mentioning the bridge it will bind to , that is br1 as in above definition of NNCP. This definition also mentions VLAN id.

 

# oc create -f  <above_nad.yaml> -n <namespace-name>

Define as many NAD in namespace , for adding as many VLAN traffic, to given pods/VMs in that particular namespace.

B> Bridge not filtering the VLAN Traffic.

Next we see, how to implement VLAN network without Bridge performing the filtering. This essentially means that between the interface of VM/Pod and bridge, there exist a tap device for each VLAN traffic, identified by it's ID as shown above.

I> First we define VLAN tagged interface that gets connected as port to Bridge with VLAN filtering diabled with curly braces as in above figure. Use command, oc create -f to create the bridge.

II> Next create NAD without VLAN id parameter, as  below,

Use oc apply/create command to create NAD in given namespace.

For further reference, recommend reading redhat docs . For VLAN filter understanding, read here.