Wingspan

Bringing "IT" All Together

by Tim.Graumann on ‎08-03-2010 01:14 PM (686 Views)

For the past several months, I’ve used this forum to highlight the various technology components that make up Brocade’s new campus infrastructure: a state-of-the-art data center, a world-class converged IPv6-ready network that relies exclusively on Brocade products, expanded unified communications capabilities, and a sustainable/green design. So what do we end up with when we bring it all together? In this case, the whole really is more than the sum of its parts.

When we designed the new campus, we didn’t just want to consolidate our existing technologies and services at a new site. We wanted to create a showcase of Brocade’s capabilities by leveraging our own products wherever possible. And thanks to the hard work of the hundreds of people from IT, Engineering, Facilities and Real Estate, we have. Their dedication combined with Brocade technology has proven to be very powerful. They’ve helped make a real difference for our staff, customers and our business.

Today, the new IP network foundation is delivering 1Gbps to the desktops of the 2,500 Brocadians who work onsite, 10Gbps in the data center core, and 1Gbps to the WAN and Internet. That capacity is helping to support bandwidth-hungry applications, including voice communications based on VoIP, wireless service, video conferencing using traditional H.320 and the newer H.264, and a multitude of web services. Here are the rest of the highlights:

Technology

Key Points

Metrics

Benefits

San Jose Campus

  • One campus
  • LEED gold
  • Highest efficiency equipment in power and chiller plants
    • Three buildings (with the option to build a fourth)
    • 18 floors in total
    • 556,000 sq ft
    • 1,850 racks (Engineering and IT)
    • 2,500 employees moved
    • A state-of-the-art facility with an extraordinary network in support of all Brocade systems and services

    Data Center

    • Three San Jose-area data centers consolidated into one 5,000 square-foot, state-of-the-art facility (pictured)
    • Business continuity supported by a high-efficiency UPS system with a stand-by generator, second B Bus rail for all critical equipment, redundant in-row cooling units, a back-up water reservoir system, and the latest fire suppression technology
    • Provisioned for a future Tier upgrade that will enable installation of a second UPS system and redundant chiller
      • 12 rows
      • 24 IRCs
      • 119 racks
      • 200 tons of cooling with redundancy
      • 1500 kVA of distributed electrical power in A / B configuration
      • Designed to optimize energy efficiency with hot row - cold row configuration and hot row containment; custom-designed in-row cooling units; fully automated controls at the POD level with energy monitoring capabilities; energy efficient chillers and cooling towers; and high-efficiency motors with variable frequency drives; occupancy sensing lighting; state of the art VESDA system;  fire suppression using dry pipe and dry agent to protect equipment;  and a calculated PUE of 1.3

      LAN

      • All-new IP network relies exclusively on Brocade products
      • Brocade’s NetIron MLX technology eliminates the need for the aggregation layer in a   traditional network configuration and connects the network edge directly to the core
      • FCX switches at the edge
      • 1,440 strands of fibre interconnecting all the buildings on campus
          • 4 MLX-32 routers (2 data center, 2 campus core)
          • 1,344 1000 Mbps connections per MLX-32
          • 210 FCX switches provide 10,080 1 Gbps ports distributed throughout    the campus
          • Connecting the network edge directly to the core has greatly reduced capital outlay and simplified operations
          • It also lessens requirements for space, power and cooling for lower total cost of ownership
          • High-density, high-efficiency modular switches reduce need for space, power and cooling in the IDFs

          MAN

          • Metro IP and FC backbone to temporary expansion buildings, deployed on Brocade hardware
            • 1 dark fibre, 4 X 8 Gbps FC connections multiplexed into one 32 Gbps connection for SAN interconnect
            • 2 dark fibres connecting IP traffic at 10 Gbps
              • Allowed for transparent user and Data Center storage migrations from existing Brocade buildings to the new campus
              • Provides extremely low network latency between buildings in San Jose

              WAN

              • The network transport connecting Brocade’s campus   and vendor-neutral co-location facilities travels over a dense wave division-multiplexing (DWDM) network with optical protection for uninterrupted, high-bandwidth connectivity to the outside world, using the   ServerIron ADX platform
              • All-new WAN relies on Brocade products
              • Fully redundant 1 Gbps WAN links, upgradable to dark fibre if needed
              • WAN connectivity between Brocade and partner networks leverages inexpensive Ethernet that allows us to connect directly to service providers and business partners with cost-effective, scalable   bandwidth via co-location facilities
              • Foundation to move to MPLS in the next 6 months
              • Two physically diverse paths
              • 2 X 1 Gbps   connections, all deployed on Brocade hardware
              • DWDM
                • § 4 nodes: 110,   130
                • § PAIX, Equinix   colos
                • § 2 Gbps to   Internet
                • § 2 Gbps to WAN
              • Partnered with AboveNet and Ciena for connectivity
                • Gives Brocade the flexibility to match capacity to business demand, and pay as we go
                • Provides a scalable bandwidth platform for future growth and business continuity needs

                WLAN

                • Entirely new wireless infrastructure based on   N-capable equipment
                • Design includes ~ 50% overlap in WAP coverage such   that a single WAP failure will cause the surrounding WAPS to automatically increase strength in order to maintain coverage
                • High-security AirDefense system which detects and   counters rogue WAPs
                  • 180 Brocade 7131 WAPs
                  • Support a/b/g/n protocols
                  • Dual 7000 WLAN Controllers
                  • Secure and robust WLAN provides network connectivity similar to wired

                  Telephony & DAS

                  • Replaced multiple, disparate phone systems with a single unified Avaya IP telephony system
                  • All voice mail provided by Exchange Unified Messaging
                  • New low-cost Long Distance over SIP trunks
                  • SIP interface between the IP-PBX and Exchange UM
                  • Integrated Distributed Antenna System (DAS) to repeat wireless carrier signals
                    • Avaya Aura 5.2 with Session Managers and System Manager
                    • ~2500 handsets
                    • 1 Attendant console
                    • All local calls via 9 PRI PSTN circuits, from two carriers
                    • All long distance calls, ~ 1200/day, via SIP trunk to a 3rd carrier
                    • 3 Avaya G450 gateways (one per building)
                    • Commscope/Andrew DAS solution
                      • Fully IP-based telephony system reduces cabling infrastructure costs and administrative overhead
                      • Low-cost long distance calls
                      • Multi-site failover capabilities
                      • Advanced user functionality from the Exchange UM system
                      • Foundation laid for future Unified Communications capabilities

                      SAN / Storage

                      • Upgraded storage-area network (SAN) using Brocade’s latest product suite to address the demands of ballooning data growth and server virtualization
                      • Dual DCX SAN cores along with Brocade 5100 edge devices and embedded SAN switch devices in every blade-server chassis
                      • All SAN equipment managed and monitored through   Brocade’s Data Center Fabric Manager (DCFM) Enterprise product
                      • SAN virtualization is being considered for general   deployment within the campus and the Broomfield, Colorado data centers
                      • Existing SAN storage from current San Jose data   centers migrated and integrated into the campus infrastructure to form   several aggregated storage pools
                      • Non-blade server builds for the campus use Brocade’s 815 HBAs to provide redundant 8Gbps links into the dual-fabric SAN
                      • 2 fully redundant SAN fabrics/cores
                      • 2 DCXs provide 192 ports per fabric
                      • 11 edge switches per fabric
                      • 1,144 ports @ 8 Gbps per fabric
                      • 14 ISLs per fabric:
                        • § 8 @ 64 Gbps
                        • § 2 @ 24 Gbps
                        • § 1 @ 16 Gbps
                        • § 3 @ 8 GBPs
                      • 18 access gateways per fabric
                      • 10 SAN arrays
                      • 650 TB raw SAN storage
                      • 8 NAS arrays
                      • 185 TB raw NAS storage
                      • High-availability   storage fabric for maximum uptime
                      • All blade   chassis are connected to the SAN using Brocade Access Gateway devices,   virtualizing SAN ports and greatly simplifying SAN management
                      • Increasing ISL   throughput to 8Gbps reduces our cabling and patch requirements and lowers the   number of potential failure points
                      • Purpose-specific   storage arrays allow for high-performance application tuning

                      Servers

                      • Large part of the campus migration has been from   physical to virtualized servers
                      • Standardized on blade platforms for all X86 processing needs
                      • Using Sun M Series platform for monolithic workloads
                      • 140 Windows servers: 40 physical, 100 virtual
                      • 820 UNIX servers: 320 physical, 500 virtual
                      • 63% of servers in the new campus are virtualized
                      • Efficiencies gained through standardized management include lower cooling costs
                      • Better power and space management that will deliver significant OpEx reduction and long-term cost savings
                      • Flexibility to support internal and external clouds through aggressive virtualization

                      So, where do we go next?  As Brocade continues to work its tactical and strategic plans, IT will have a central and evolving role. The infrastructure we now have in place gives us the tools to meet and quickly support business needs, and it enables us to deliver even greater value to our customers.

                      For example, our core and edge network infrastructure provides built-in capacity and scalability to support storage virtualization and a service-oriented architecture. It also provides bandwidth to support future applications like Fixed Mobile Convergence and Unified Communications, including video to the desktop using Microsoft Office Communicator and advanced web cams such as Microsoft’s LifeCam Show.

                      Although the new campus project officially wraps up next month, it’s certainly not the end of the road for us. My next blog will coincide with the official grand opening celebrations at the new campus, which gives me a perfect opportunity to share our project successes, lessons learned and notable project metrics. I appreciate your continued interest in my blogs and I invite you to check back.

                      Pictured below is Brocade’s state-of-the-art data center located at the new San Jose campus

                      http://www.brocade.com/community/tim_blog_data_center.jpg

                      An outside view of Brocade’s new San Jose campus

                      http://www.brocade.com/community/tim_blog_new_campus.jpg