Repository navigation
F #-: VRouter: FRR: Dynamic Routing with BGP and OSPF - #416
netaviator wants to merge 5 commits into
Conversation
…tributes The VRouter cannot talk to upstream routers dynamically. Add an FRR module that runs bgpd, ospfd and bfdd next to the existing services and builds /etc/frr/frr.conf from attributes. Nothing changes unless it is turned on. The attributes come from the VM context at boot (ONEAPP_VNF_BGP_*, ONEAPP_VNF_OSPF_*, ONEAPP_VNF_FRR_ROUTER_ID, ONEAPP_VNF_STATIC_ROUTES) and, while the VM runs, from the VM user template through OneGate. A change is checked with frr-reload.py --test and then loaded with --reload, so other neighbors keep their sessions. If the reload fails, the last good config is loaded again. A failed apply is retried with a growing delay. BGP has IPv4 and IPv6 neighbors with password, BFD, prefix filters, local pref, MED and prepend. OSPF is OSPFv2 with area, cost, passive, timers, MD5 key and BFD per interface, an optional default route and redistribution of connected and static routes. On the VRRP backup, BGP gets a longer AS path and a higher MED, and OSPF a higher cost and metric, so peers prefer the master. This needs the failover hooks (OpenNebula#405). The backup settings are switched by a hook in /etc/one-appliance/ha-hooks.d, which Failover has to call. Without OpenNebula#405 the hook is installed but never run, and both nodes stay on the master settings. The service template is not part of this. The marketplace template needs an input for each attribute it should offer; the README lists them. There is no limit on the number of neighbors or OSPF interfaces in the module. Checked with 483 unit specs, 28 of which compare the generated frr.conf with stored files, and with real FRR 10.2.1 in Docker (sessions, filters, live changes, rollback, the backup settings, OSPF, install and configure on Alpine). It was also tested live on VMs with several instances and external peering. Signed-off-by: Jan Gilla <jgilla@level66.network>
Graceful restart (helper only): ONEAPP_VNF_BGP_GRACEFUL_RESTART=NO|HELPER, GR_RESTART_TIME and GR_STALEPATH_TIME. The setting is read at boot only. ECMP: ONEAPP_VNF_BGP_MAXIMUM_PATHS, MAXIMUM_PATHS_IBGP and MULTIPATH_RELAX, plus static routes to the same prefix with different gateways. Identical gateway pairs are rejected as duplicates. Signed-off-by: Jan Gilla <jgilla@level66.network>
The hooks are in PR OpenNebula#415. Also name the test image as a Docker image. Signed-off-by: Jan Gilla <jgilla@level66.network>
zebra turns IPv4 and IPv6 forwarding on when it starts, so the standby forwards while FRR runs. The README said the standby has forwarding stopped. Signed-off-by: Jan Gilla <jgilla@level66.network>
zebra turns IPv4 and IPv6 forwarding on whenever it starts, and the stock Failover does not switch it off on a standby, because it skips the stop of a service that is not running. When the VIP moved, the new master forwarded client traffic before its NAT4 rules were in place, and the server reset the connections it knew under the old address. The 10-frr hook now writes 0 to the forwarding sysctls after its apply on down, and the poller does the same after each apply while the node is BACKUP. The README describes it. Signed-off-by: Jan Gilla <jgilla@level66.network>
roamingengineer311
left a comment
There was a problem hiding this comment.
FRR configuration is not updated after re-contextualization or reboot
I changed ONEAPP_VNF_BGP_NEIGHBOR1_EXPORT_PREFIXES from 198.51.100.0/24 to 192.168.100.0/24. The updated value is present in /run/one-context/one_env, but /etc/frr/frr.conf still contains:
ip prefix-list BGP-N1-OUT seq 10 permit 198.51.100.0/24
Rebooting the active Virtual Router did not update the generated FRR configuration. The BGP session is established and the router receives one prefix from the upstream (PfxRcd=1), but advertises none (PfxSnt=0). The expected export is 192.168.100.0/24.
I suspect this affects other FRR settings provided through contextualization as well, not just the export prefix. Could you check whether the FRR configuration is regenerated and applied after re-contextualization and at boot?
Add dynamic routing to the VRouter, with BGP and OSPF as options. The router can then exchange routes with upstream and internal routers instead of relying on static routes.
Tuning options: BFD for quicker failover, ECMP to use multiple paths in parallel, and graceful restart (helper mode) so routes are kept while a neighbor restarts.
VRRP integration: the active router is preferred. On the standby, BGP gets a longer AS path and a higher MED, and OSPF a higher cost, so peers send traffic to the active router. The standby settings are switched by a hook in /etc/one-appliance/ha-hooks.d, so this needs the failover hooks (#415).
The module runs bgpd, ospfd and bfdd next to the existing services and builds /etc/frr/frr.conf from ONEAPP_VNF_BGP_, ONEAPP_VNF_OSPF_, ONEAPP_VNF_FRR_ROUTER_ID and ONEAPP_VNF_STATIC_ROUTES. The attributes come from the VM context at boot and, while the VM runs, from the VM user template through OneGate. A change is checked with frr-reload.py --test and then loaded with --reload, so other neighbors keep their sessions. If the reload fails, the last good config is loaded again. Nothing changes unless it is turned on. The README lists all attributes.
The service template is not part of this. The marketplace template needs an input for each attribute it should offer.