Repository navigation
F #-: VRouter: Management NIC in separate MGMT VRF - #419
netaviator wants to merge 8 commits into
Conversation
The NIC that OpenNebula marks as VROUTER_MANAGEMENT moves into a VRF, so its default route no longer collides with the one in the main table. Opt out with ONEAPP_VNF_MGMT_VRF_ENABLED=NO. Signed-off-by: Jan Gilla <jgilla@level66.network>
Moving a NIC into a VRF flushes its IPv6 addresses and routes. Put the static address, the ULA and the default route back, inside the VRF table. Signed-off-by: Jan Gilla <jgilla@level66.network>
The appliance talks to OneGate from the default VRF, which cannot reach hosts behind the management NIC once it is enslaved. Add a policy rule that sends the traffic for the OneGate address (IP literals only) to the VRF table, and remove it again at cleanup. Signed-off-by: Jan Gilla <jgilla@level66.network>
one-context adds the default route of every NIC with 'ip route add', so the gateway of the first NIC (the management one) wins and the others are never installed. With the management gateway in the VRF table, apply the gateways (IPv4 with ETHx_METRIC, IPv6) of the NICs that are not marked as management in the main table. Signed-off-by: Jan Gilla <jgilla@level66.network>
Describe the ONEAPP_VNF_MGMT_VRF_ENABLED attribute (on by default), what the module configures and in which order, the accepted values and what it does not cover. Signed-off-by: Jan Gilla <jgilla@level66.network>
|
@netaviator Hi. |
|
looks like I do not see your inline comments. Any chance you can provide them? Best |
Moving a NIC into the VRF drops its static routes from the main table, and deleting the VRF drops the routes of the VRF table. Put the ETH<n>_ROUTES of the management NIC into the VRF table. On stop, give the default routes of all NICs, the IPv6 addresses and the ETH<n>_ROUTES of the management NIC back to the main table, since one-context does not run again. A default route is only added when the NIC has none, so one that one-context installed (for example with a metric) is kept. Signed-off-by: Jan Gilla <jgilla@level66.network>
|
Thanks for testing this and for the detailed steps. I pushed 3ece29c, which addresses the three points:
I checked the first two cases and the existing default route on a VM with a management NIC and a second NIC. The unit specs cover the rest. The README describes the new behavior. |
roamingengineer311
left a comment
There was a problem hiding this comment.
I retested the updated image and confirmed that the static route and re-contextualization issues are fixed. I left one follow-up comment about duplicate default routes still appearing after a service restart. Could you please take a look?
…MgmtVRF The bash helper runs scripts with pipefail. The check for an existing default route ended in grep -q, which exits at the first match while ip route show still writes. ip then dies of SIGPIPE, the pipeline fails with a default route in place, and a second default without metric was added next to the one of one-context. It happened in about half of the restarts in a test VM, depending on timing. The check now lets grep read the whole output. Signed-off-by: Jan Gilla <jgilla@level66.network>
roamingengineer311
left a comment
There was a problem hiding this comment.
Thanks for the fix. I tested the latest change. No further concerns from my side.
… host name A host name in ONEGATE_ENDPOINT got no policy rule, so the default VRF could not reach OneGate behind the management NIC. The name is now resolved through the VRF (falling back to the default VRF) once the NIC and its routes are in place, and a rule is added for every IPv4 and IPv6 address. Stop and start delete every rule of preference 100, since the addresses behind a name can change between runs. Signed-off-by: Jan Gilla <jgilla@level66.network>
OpenNebula marks one NIC of a Virtual Router as the management NIC. In OpenNebula it is set with VROUTER_MANAGEMENT = "YES" on the NIC of the Virtual Router template, and arrives in the VM context as ETH_VROUTER_MANAGEMENT=YES. one-context adds the default route of every NIC with ip route add, so the gateway of the first NIC wins and the default route of the management NIC collides with the one of the NIC that carries the traffic.
Move the management NIC into its own VRF (mgmt, routing table 1000), so its default route lives in a separate table and the main table stays free for the routed NICs. It is on by default. Opt out with ONEAPP_VNF_MGMT_VRF_ENABLED=NO. Without a NIC marked as management the module does nothing.
Values that end up in a shell command (gateways, metrics, IPv6 addresses) are checked; an invalid one is logged and skipped.