This is a just a very short article about limitations and options with route leaking on the ex4600 switch. This switch is base on the Trident II chip manufactured by Broadcom. It is not very new but it is still a solid piece of silicon used by many platforms across the vendors (Arista, Cisco Juniper). In case of Juniper it is used in the QFX5100 and the EX4600. The EX4600 is classified as an aggregation switch for offices or small DCs so there are rather serious limitations on what you could expect from the device as from all the members of EX switch family.
The obvious intent of the designers of this hip was to have very high throughput and in exchange losing some higher routing functions. This has one rather serious implication – there is only one route lookup operation available. Unfortunately for some operations you need to perform the route lookup twice specifically if you:
-
- leak routes
- double-encapsulate or decapsulate (VTEP/IRB situation)
There are possibly other examples but I have personally only encountered the two above. The good news is that the double route lookup has been actually added through some tweaks in Junos so you can actually do route leaking. The issue is that there are multiple KB articles that have not been updated (as of writing this article) KB19860 and the more funny one KB23027.
The first one (KB19860) is mostly correct but it is missing missing the crucial information that this is not available on all EX switches – actually it is not available on quite a few of them and also the KB also misses a release information. The second one (KB23027) is just plain wrong on many accounts and the proposed solution is to use “tromboning” with physical cables between the VRF ports. Fortunately this is note really needed and route leaking actually does work.
The main issue I’ve encountered was that when I tried to leak interface lo0.0 I had was getting rather odd results.
The situation was that the system services were to be sourced from lo0.0 but as there were limitations and design restrictions in place – this loopback was not part of the routing instance that was actually providing the transport connectivity and on top there was to be another loopback in the transport routing VRF (called test in the example below).
So as for the egress traffic the logic was simple just use a 0/0 static route with a next-table stanza as an egress. There was no other connectivity to the inet.0 (default) instance so this was fine. on the return path I’ve created a RIB group importing all of the inet.0 into the ri_test.inet.0 which in this case should made the lo0.0 available in the new VRF for the return traffic.
set routing-options static route 0.0.0.0/0 next-table ri__test.inet.0 set routing-options interface-routes rib-group inet rg_test set routing-options rib-groups rg_test import-rib inet.0 set routing-options rib-groups rg_test import-rib ri_test.inet.0
From the test VRF the routes were happily propagated via BGP. But that is where the oddity started. The lo0 with an ip of 1.1.1.1 was unreachable using ping with the TTL exceeded ICMP message. But traceroute worked just fine.
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. From 1.0.0.2 icmp_seq=1 Time to live exceeded From 1.0.0.2 icmp_seq=1 Time to live exceeded From 1.0.0.2 icmp_seq=1 Time to live exceeded
After long investigation all looked normal – routing table was populated as expected, forwarding table was showing next-hope as routable (which is expected in route leaking scenario).
Routing table: ri_test.inet Internet: Enabled protocols: Bridging, Destination Type RtRef Next hop Type Index NhRef Netif 1.1.1.1/32 user 0 rtbl 1 3
I also got some interesting troubleshooting commands that are not exposed directly in JUNOS. Below you can see those command. They are very helpful as they are showing the routing and forwarding tables as understood by the underlying system rather than the interpreted version you can pull through JUNOS.
> start shell % vty fpc0 <<<( in stack VC this needs to be done separately for each fpc) show route ip table show route ip table index <value> show route ip prefix 1.1.1.1 show shim vrf-table routes show shim bridge interface set dc bc "l3 egress show" set dc bc "l3 l3table show" set dc bc "l3 defip show"
It turned out that there was no next hop for the lo0.0 to respond to the ICMP query as the configuration was hitting the Trident II limitation mentioned at the beginning of this article. On top of this the implementation of the double route lookup was suffering from a bug (PR1449410) that affects local interfaces and especially loopbacks.
In summary you can do route leaking on ex4600 as the issue has been resolved in JUNOS 18.3 and later (i.e. not in the recommended release at the time of writing). Also the recommended way of actually doing this is described here in an updated guide.
In conclusion – yes route leaking is possible but the double route lookup requires JUNOS 18.3 or never. I would also suspect that this could be rather intense for the CPU on the EXes,but I haven’t had the opportunity to stress test this so it is just a guess.