Ethernet Switches & Routers

Reply
Occasional Contributor
Posts: 12
Registered: ‎02-21-2013

Need some help setting up a management VRF

I'm coming to some MLXe's from the Cisco world and am having a bit of trouble with setting up a management VRF.  I need a management VRF because I need some static routes applied only to the management port.  On the Cisco side, this is about all I do (with some of the routes removed for brevity):

vrf definition Mgmt-intf

!

address-family ipv4

exit-address-family

!

address-family ipv6

exit-address-family

!

interface GigabitEthernet0

description MGMT

vrf forwarding Mgmt-intf

ip address 10.0.0.10 255.255.255.0

!

ip route vrf Mgmt-intf 10.0.1.0 255.255.255.0 10.0.0.1

And there you have it.  I've created the VRF, put the management port into it and added a static route that only applies to the management interface.

On the Brocade side, the first hurdle is the route distinguisher.  I don't use MPLS or VRF's in normal practice so I'm not familiar with that.  These routers will be acting as internet BGP routers so the docs telling me to define a route distinguisher with an AS number makes me nervous.  It says I can use an Ip address too, which of course makes me equally nervous.  So my first question is does the route distinguisher have any significant meaning or is it simply a unique ID that each VRF needs that plays no role in routing on the box?

Since these routers are still in test, I decided to just try 10.0.0.0:0 as my RD.  I set that up, defined it as the management VRF and then added a static route:

interface management 1

ip address 10.0.0.10/24

enable

!

vrf Mgmt-intf

rd 10.0.0.0:0

address-family ipv4

exit-address-family

exit-vrf                                                         

!                                                                

management-vrf Mgmt-intf                                         

ip route vrf Mgmt-intf 10.0.1.0/24 10.0.0.1

That resulted in the router moving my route statement up into the vrf definition, which I guess is fine, just different than Cisco, but it doesn't seem to have done anything and the route table still has no routes:

#sh ip route vrf Mgmt-intf

Total number of IP routes: 0

and for whatever reason, the normal route table still contains a route for the management interface which I would not expect it to (with a src-vrf showing as none):

sh ip route

Total number of IP routes: 4

Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric

BGP  Codes - i:iBGP e:eBGP

ISIS Codes - L1:Level-1 L2:Level-2

OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link

STATIC Codes - d:DHCPv6

        Destination        Gateway         Port          Cost          Type Uptime src-vrf

2       10.0.0.0/24        DIRECT          mgmt 1        0/0           D    15d22h -

Additionally, if I ping the management interface's gateway, it works, but if I ping it from the vrf, it doesn't:

#ping vrf Mgmt-intf 10.0.0.1

Sending 1, 16-byte ICMP Echo to 10.0.0.1, timeout 5000 msec, TTL 64

Type Control-c to abort

Request timed out.

No reply from remote host.

#ping 10.0.0.1

Sending 1, 16-byte ICMP Echo to 10.0.0.1, timeout 5000 msec, TTL 64

Type Control-c to abort

Reply from 10.0.0.1      : bytes=16 time=1ms TTL=64

Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms.

So my second question is can someone tell me what needs to be done to have the management ethernet work properly in its own VRF with its own static routes that do not affect the regular routing table?

Thanks!

New Contributor
Posts: 4
Registered: ‎08-12-2009

Re: Need some help setting up a management VRF

OK.

You are close, very close.  I am not a VRF guru but have used it some in a couple of instances one of which is for management.

1) Your interface definition is missing its VRF "binding" so it is bound to the default VRF.

interface management 1

ip address 10.0.0.10/24

vrf forwarding mgmt-intf

enable

(I would suggest naming it just "Mgmt" instead of "Mgmt-intf" as it is not an interface but a virtual routing instance but that is style rather than substance.)

2) Try putting your static route definition in the "vrf Mgmt-intf" context (which is what I have done).

!

vrf Mgmt-intf

rd 10.0.0.0:0

address-family ipv4

ip route 10.0.1.0/24 10.0.0.1

exit-address-family

exit-vrf                                                        

The systax that you use seems to be correct and intuitive so I am not sure why it doesn't work.


3) As far as I know, and the documentation is not particularly helpful on this, the RD value is not significant as long as it is unique.                                                               



I hope this is accurate and helps.


-Dave Cooley




Occasional Contributor
Posts: 12
Registered: ‎02-21-2013

Re: Need some help setting up a management VRF

Hmm, I don't have an option to define 'vrf' at the management interface level, with or without the vrf itself being defined as the management vrf:

SSH@bdr5#conf t

SSH@bdr5(config)#int mana 1

SSH@bdr5(config-if-mgmt-1)#?

  clear                         Clear table/statistics/keys

  cls                           Clear screen

  disable                       Disable the interface

  enable                        Enable the interface

  end                           End Configuration level and go to Privileged

                                level

  exit                          Exit current level

  ip                            IP interface

  ipv6                          IPv6 parameters

  no                            Undo/disable commands

  quit                          Exit to User level

  show                          Display system information

  write                         Write running configuration to flash or terminal

  <cr>

SSH@bdr5(config-if-mgmt-1)# exit

SSH@bdr5(config)#no management-vrf Mgmt-intf

VRF Mgmt-intf has been un-configured as management-vrf

SSH@bdr5(config)#int mana 1

SSH@bdr5(config-if-mgmt-1)#?

  clear                         Clear table/statistics/keys

  cls                           Clear screen

  disable                       Disable the interface

  enable                        Enable the interface

  end                           End Configuration level and go to Privileged

                                level

  exit                          Exit current level

  ip                            IP interface

  ipv6                          IPv6 parameters

  no                            Undo/disable commands

  quit                          Exit to User level

  show                          Display system information

  write                         Write running configuration to flash or terminal

  <cr>

I do see the vrf command available on the regular line card interfaces.

When I ran that ip route statement, the router actually moved the route up into the vrf config for me:

!

vrf Mgmt-intf

rd 10.0.0.0:0

address-family ipv4

ip route 10.0.1.0/24 10.0.0.1

exit-address-family

exit-vrf     



I'm running 5.4.0b on an MLXe 400 with the MR2 management and -X line cards.  I also tried taking it out of route-only mode and still no change.  Maybe this is a bug?

Occasional Contributor
Posts: 12
Registered: ‎02-21-2013

Re: Need some help setting up a management VRF

I contacted support on this.  Not want I wanted to hear but apparently, for reasons unknown, the management port is the one port on the router that cannot be put in a different VRF.  So you can create a management VRF, but you can't put the management port in it?!?!

I guess this means the only way to have true out of band management through the management module would be to create a VRF that encompasses all other routing on the router so you can use the default VRF strictly for management; obviously a pretty ugly solution.  Otherwise, you're stuck using a line card port for management, when the line card might be what you're wanting to manage or troubleshoot a problem on, or using a VE (which fortunately can be added to a VRF), but that doesn't work if you're running in route only mode since none of your ports would be switched/tagged.  I'm running route-only so I'm probably going to be stuck using a line card port, but I only have one line card with spare copper (cheap) ports, the rest is all 10gig, so I'm going to just have to hope I never have issues with the line card.

New Contributor
Posts: 3
Registered: ‎05-12-2011

Re: Need some help setting up a management VRF

Hi,

I'm not sure if this applies to your requirements, but there is no switching or routing between the linecards and the mgmt int. So even if you have a default or specific route pointing to the mgmt int, that won't affect the linecard traffic. So from your example, you could just use the 10.0.0.10/24 to the mgmt int plus an static route to the 10.0.1.0/24 with a next-hop of 10.0.0.10. That should do exactly what you described in your cisco example but please correct me if I'm missing in other requirements.

Thanks

Frequent Contributor
Posts: 131
Registered: ‎07-02-2012

Re: Need some help setting up a management VRF

Hello,

Been there, there is absolutely nothing useful you can do with the management interface. The only thing you can do is enable / disable and change the IP address.

I, too, wanted to use it and forward it in a VRF that could be used for management (Cisco style). But hey, I had to change my plans.

HTH.

Y.

Occasional Contributor
Posts: 12
Registered: ‎02-21-2013

Re: Need some help setting up a management VRF

The static routes I need are outbound from the management interface and need to be completely independent from all other routing on the device.  There does not appear to be any way to do that, like ybzahr also confirms.  Since I also want the security of a management vrf, it does seem the management port is completely useless.

New Contributor
Posts: 3
Registered: ‎05-12-2011

Re: Need some help setting up a management VRF

I fully agree with you on the MLX mgmt int limitations. Sorry for not being clear about this, but the default MLX mgmt int is already completely independent from all other routing on the device and the static routes i suggested will be only for the outbound traffic from this mgmt int. I know that it's not clear from the show ip route command since you see the routes in the global routng table, but believe that there won't be any traffic moving from the linecards to the mgmt int despite any static routes. This is the way we setup telnet/ssh access to our MLX in our labs. I've been asking the documentation team to clarify this in our documentation, so hopefully they can take care of this soon.

Anyhow hope this helps as well but please let me know if there's any other specific security features that you would need in in this mgmt int if you decide to use it.

Occasional Contributor
Posts: 12
Registered: ‎02-21-2013

Re: Need some help setting up a management VRF

Hmm, how do I make an outbound route that's specific to the management interface and won't affect the normal routing table or get injected into OSPF/BGP as a static redistribution?

I likely won't go that route anyway since I need the protection of the VRF on all the management protocols but just in case, I'd like to know how to do it.

New Contributor
Posts: 3
Registered: ‎05-12-2011

Re: Need some help setting up a management VRF

An static route that uses the mgmt prefix/int as the next hop will still show up in the routing table and could be injected into OSPF if it's not filtered. It won't affect the forwarding though because MLX architecture doesn't have a way to forward any traffic from the linecards to the mgmt interface.

This configuration could look just like this

interface management 1

ip address 10.0.0.10/24


ip route 10.0.0.0/8 10.0.0.10


but if you get a telnet session from outside the 10.0.0.0/8 net then you would need an specific or default route. Redistributing this static route into OSPF/BGP so you would probably need to filter.


Anyhow, like i said, we are going to improve this in both documentation and with a clearer CLI so it's clearer in the future.


Thanks


Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

vADC is now Pulse Secure
Download FREE NVMe eBook