03-02-2016 11:49 PM
We have a 3d rendering business where the amount of files and transfer of said files is becoming a bandwidth issue.
That being said, we decided to dip our toes into fiber channel for video and file transfer in the render farm.
I want to point out up front that I am completely new to this arena, so I apologize up front for what is sure to be a post full of nonsense.
Here is what I am used to... Configure my IP addresses.. (subnet, gateway, etc) plug in a cable and Viola! access to the network.
I was expecting something like this using fiber... (insert joke here) since we did not know if it would be a huge gain or not, we bought a few test pieces. (some 10gb hp NICs and an 8gb hp fiber switch (AM866-B I think). I was hoping the NICs would be backwards compatible the way copper NICs usually are, but all I ever got from the web interface was orange blinking lights and no connectivity.
My troublshooting process went like this... Maybe the transceivers were not compatible, or maybe mixing 8gb and 10gb was the problem. So I bought a few more of each. I plugged in the second set of 8gb cards and was quickly getting all the connections working in the web interface. So I go to the network admin area and no NIC... I go to device manager and I see the new cards are showing as storage controllers and not NICs. ohhhh boy... so I go through a week of testing troubleshooting and finally getting an iscsi VHD working with targets and initiators... success! But...
I had two issues. One: We have a bazillion external file references that get called in with textures and references of all kinds, so the VHD path UNC wasn't working anymore. Which I could live with, but the artists were whining like you wouldn't believe. So I thought maybe the best route would be to use 8gb NICs and scrap the 10gb ones... I couldn't find any 8gb NICs. Only 8gb storage controllers.. Do 8gb NICs exist?
So I bought a 6720 Brocade 10gb switch... I thought that would be the best mix of what I needed... at least 32 ports and 10gb connectivity across the board. Here is where I hit my biggest hurdle. No web interface.... NOOOOOO!!! CLI only. I have been bashing my brains out on this setup for almost a week. Trying to get the switch configured. I'm at my wits end...
I figured out how to name the switch from the default sw0 to something more intuitive, assigned a virtual IP, converted the switch to VCS since it sounded easier to handle for a low security risk network environment over logical chassis. I have green blinking lights on both ends of the chain (NIC and router), but I still cannot get files to transfer.
I am not sure if it is a switchport issue... the documentation assumes SO much knowledge.. I feel like I am fumbling around in the dark. So I am here looking for some advice brfore bailing on the switch and going to a 24 port IBM one they recommended. I love the robust nature of this switch, but I have done all the troubleshooting I can think of.
Here is one problem I keep running into... every time I try to go to interface Management 1/0 it tells me there is a syntax error and never lets me deeper into the tree. I tried long notation, short notation, changing the backslash forward slash, mixing the 1 and zero around... nothing.. my closest error says invalid argument on the 1/0.. What are the 1/0 flags doing? It says to add it, but never says what it does!
Here is another noob question I have... does each port get a unique IP? Or are the Ip addresses only for the switch as a whole? I thought maybe that was my issue... that communication needed to be setup on each port.
If you read all of this, thank you. I grately appreciate your time. I know it's long winded, but I felt it was my duty to do as much as I could on my own without bothering the expert fiber gurus on this forum.
I would love to get any wisdom the community has to offer, and my apologies again for sounding like a complete noob. Please let me know where I may have not given proper information to answer the questions.
Thank you for your time.
03-03-2016 02:08 AM
The 1/0 attributes that are referenced in the "interface management 1/0" command are the rbridge id and the local interface. It sounds like the rbridge id may not be set to 1. If you enter "show vcs" at the # prompt, or "do show vcs" at the (config) prompt then it will list all of the rbridges in your fabric. It looks like you only have one so this will be the one you are connected to. If you use the rbridge id/0 in place of the 1/0 then this should get you on your way.
In answer to your other question about the IP addresses. By default the 6720 is a layer 2 switch so each port does not have a unique IP address.
Also, although this looks like a 32 port switch, ports 1 to 24 are 1/10GbE Ethernt and ports 25 to 32 are fibre channel (but only enabled with an appropriate licence.
03-03-2016 07:21 AM
Thank you so much for putting that into human language.
I am already so much further just in an hour or so. I am still getting stuck when trying to assign a default gateway.
I have tried several different methods and it keeps saying syntax error.
One site said to go into device(config-rbridge-id-2)# ip route (ipaddress)
but I don't know hot to get into that mode I tried typing config-rbridge-id-2 (2 is my id) and I keep getting the same error.
But I enabled switchport and set access.. more testing to follow.
Thanks so much already! What a huge difference...
03-03-2016 07:35 AM
It's far from ideal right now and I have no default gateway, but at least I'm able to transfer files.
03-03-2016 07:49 AM
In answer to your earlier post about getting into the correct context to set the default route, you just need to type "rbridge 2" once you are in config mode. You can then set up the default route.
conf t prompt changes to (config)
rb 2 prompt changes to (config-rbridge-id-2)
ip route 0.0.0.0/0 a.b.c.d
where a.b.c.d is your next hop gateway. If you only want to run this as a layer 2 switch but manage it from a remote subnet then a.b.c.d will be reachable via the management interface, and this should work just fine.
03-04-2016 04:07 AM
Thanks again for that...
I have another question... big surprise...
Is there a chart or legend that shows which types of ports are in the switch. You mentioned having only so many of one type, etc. I see no documentation about that.
Also, there is what seems to be random color blocks on the switch. Some labels are Black and some are silver. (The little numbers that show the port numbers and have the led light on the ports)
Is there a system to that, do the different color designations tell me anything about the port, or is it arbitrary?
So stoked to have this up and running. Did some tests yesterday and I was getting close to 2gb a second across the raid. That's insane.. it's varied from a ten to twenty fold increase in traffic speed. Our 4k video people thank you.
03-04-2016 06:43 AM
I just had a thought that you said you had a 6720 with 32 ports. This is actually known as a 6730
There is a hardware manual at http://www1.brocade.com/downloads/documents/product_manuals/B_VDX/VDX6730_HardwareManual.pdf
Page 23 shows you the various ports that are on the switch
If it is a 6720 then this is the link to the hardware guide
Page 18 shows you a front view of the switch
You will see that the 6730 has an extra set of ports on the far right - these are Fibre Channel ports, but you will need a software licence in order to make use of them.
Both the 6720 and 6730 are no longer sold by Brocade, so the admin guide may not be easy to find unless you have a support contract. Having a support contract will give you access to the latest firmware and easy access to documentation, as well as full support from Brocade's TAC.
03-04-2016 02:58 PM
My apologies for not being more specific. I didn't have the model number when I posted the topic.
It's a 60 port model (BDR VDX-6720-40-R) I maybe confused it when I said we needed at least 32 ports.
It has 60 ports.. I thought that would leave us room to grow...
03-05-2016 03:16 AM
Sorry, if my misunderstanding has confused you.
The 6720 has only 10GbE ports so don't worry about my references to fibre channel in previous posts. All ports are the same but the only thing you need to watch out for is that the part number you quoted has only forty ports available for use (as shipped from the factory). The other twenty ports can be enabled with software licence.
It may be that your switch already has a licence installed to enable some of the remaining ports
"show licence" will indicate whether any such licences have been added to allow you to use the other ports. In terms of which ports can be used, you cna use any ports up to the forty port maximum (assuming no other licences).
Normally the ports are reserved automatically as you configure them, but if you find that you no longer want to use a previously reserved port then you can manually adjust this. At the start of the running configuration you will see the reservations, something like
To release the port, you will need to shut it down first
int te 3/0/1
Hope that helps and the 6720 works for you.
03-05-2016 06:43 AM
Thank you again for all the wisdom...
I will know in a couple days.
Now that the switch is working, I ordered the rest of the components to put it into a working environment.
Once it's in place, I will know for sure if the things you helped me do have exercised the demons from the system.
I will leave this topic open until I test everything out.
Looking forward to it.. The speed tests so far have been outstanding.. Our new server has a 16 drive raid.. each drive has 64 megs of cache and a battery caching module for the ram and system. Files that would normally take 30 to 60 seconds to save are only taking a few seconds. Yea Fiber optics!