05-27-2014 06:26 PM
I have a problem with a network setup I've been working on for several days and need some input. I'm not Brocade-literate as our sites have been exclusively Cisco before this one.
I have a pair of 6650's running latest Router code (8.0.10c) in an MCT cluster arrangement. My access layer closets consist of either a single or stack of 6450's (11 closets total) - code on these is also 8.0.10c (booting Switch code).
Each closet connects to the cores with redundant fibre links (MM, OM3, 10GB). Lags are dynamic, 2 ports and match the core deployed lags (one per core).
Each floor of closets (4 floors total) is on a separate data and separate voice vlan. So, 4 vlans for user data, 4 vlans for voice, 1 vlan for management, and one vlan for the servers.
I'm using VRFs on the cores to segregate networks coming from upstream that cannot ever route between them.
I'm also using VRRP-E for all the VE interfaces on the cores except the VE for the vlan pointing upstream.
Right now I'm working on a single VRF for the network described above, using OSPF to route between vlans and out to the upstream router.
Problem: I can ping from the console of the L2 6450 (using serial cable) to any VE VRRP on the cores (both the virtual addresses and real ones) and get a reply, but I cannot plug a PC into the access ports and ping anything at all.
Ports connecting core to access switches are tagged with management vlan, user data vlan, and voice vlan appropriate to their floor - both ends. Dual-mode is set with data vlan and voice-vlan is specified on each user port. A show vlan displays tagged voice vlan and dual mode data vlan correctly. The 6450's have been configured with management-vlan for the vlan we chose for managing the switches. Cores have had management-vrf set to this VRF.
This should be elementary, but is proving otherwise.
So, back to basics. I unplugged all fibre ports in each closet. I've now connected one 6450 (residing in the same closet as the cores - to make this troubleshooting easier).
I read this Limitation about 10 minutes before posting this question and I'm not feeling good about what I think it means for this network design - but hope I am wrong:
MCT is better than standard LAG technology to scale HyperEdge Architecture with Layer 2 traffic on the uplinks from the access to distribution layers. However if the distribution layer involves Layer 3 switching and routing then the classical hierarchical network design is a better solution.
MCT is currently limited to two switches.
MCT imposes certain limitations on the network design that should be considered including:
MCT on FastIron SX or ICX 6650 doesn’t support Layer 3 multicast traffic. For designs involving Layer 3 multicast traffic, the Brocade MLXe Router with the NetIron 5.4 or later software release can be used, that supports Layer 3 multicast traffic.
Running Layer 3 dynamic routing protocols is not supported on the ICL and CCEP links. As shown in the Typical Deployment section VRRP/VRRP-E is recommended for Layer 3 gateway redundancy when using MCT.
Since I can ping from the console to any VE in that OSPF area, I think it is routing properly on the core, but can't for the life of me understand why I cannot ping from a user port in a properly tagged vlan (yes, the laptop has a correct IP and subnet mask for the vlan the port is in).
Troubleshooting so far:
show fdp neigh - shows all connected cores from 6450 and shows this switch from core.
show mac - from 6450 can see 3 upstream macs for each vlan configured on it, as well as the laptop NIC on a local port.
- from cores can see downstream macs as well as laptop mac.
sho vlan - on 6450 shows correct dual mode ports and voice tagged ports as well as connection to cores properly tagged with all 3 vlans. From core shows proper tagged vlans on the link ports pointed downstream.
ospf routes are contained within the vrf correctly and show all connected VEs.
MCT is up and functional. Keep-alive vlan and link is up and functional.
VRRP-E shows correct Master and Backup cores with proper VIPs listed.
Client lags are all Deployed and up.
No blocking ports noted.
I do show Queues for the ethernet port connected to the 6450 on the core - which may help lead to an answer.
Any advice will be actioned when I'm onsite tomorrow, so please give me time to respond. I cannot disclose any configs to anyone as this is a classified network, but I can try to fill in any non-protected information where appropriate.
Thank you all in advance - sorry for the long story, but I want to make sure there is enough detail for you to work with.