Storage Networks

Coming soon to your All Flash Array: iNVMe!

by Curt.Beckmann on ‎08-15-2017 03:48 PM (5,764 Views)

With the NetApp/Broadcom/Brocade FC-NVMe demo at FMS and T11’s FC-NVMe face-to-face meeting both going down, last week was a busy week for NVMe and NVMe over Fibre Channel. We’ve all heard how NVMe is the fast, next-gen alternative to SCSI, whether direct- or fabric-attached, and last week was no different.  I also heard (again) more than one speaker suggest that RDMA is part of all NVMe fabrics, though it’s not. Yes, RDMA has (like Fibre Channel) zero-copy efficiency, but NVMe-oF hype often hints that RDMA has special magic pixie dust for NVMe. Nope. In defining NVMe over Fibre Channel, the T11 chose to put NVMe directly on top of Fibre Channel (without RDMA) because 1) Fibre Channel already did zero-copy, and 2) RDMA would impede dual protocol (FCP / FC-NVMe) HBAs and SANs.  As I wrote in NVMe over Fibre Channel for dummies, dual protocol SANs offer the performance benefits of NVMe while radically reducing the risks of adopting NVMe over Fabrics. And soon, as we also heard last week, there will be one more non-RDMA fabric for NVMe…

iSCSI the next gen.png

You see, during the time that we at Brocade were explaining that FC-NVMe didn’t need RDMA, folks at Facebook were puzzling over claims that adopting NVMe over Ethernet (either RoCEv2 or iWARP) would require upgrading every existing (and future) NIC in their datacenters to an RDMA-enabled NIC. The logic didn’t add up (but costs sure would), so they jointly proposed (with Intel and Dell EMC) NVMe over TCP (with zero copies of RDMA). My co-panelist Dave Minturn of Intel described it during the NVMe-over-Fabrics panel preso at FMS last week. You see, Facebook will forgo a bit of latency to access their storage with commodity infrastructure.  Sound familiar? Back in my Rhapsody Networks days, the iSCSI buzz led us to make SAN ASICs with both Fibre Channel and Gig-E.  Now, after 15 quick years, 80% of storage arrays continue to use Fibre Channel for performance and reliability; the other 20% use iSCSI for, yep, commodity infrastructure. As I described in the dummies book (linked above), Fibre Channel is the reliable, high performance, datacenter-optimized protocol that grew up at the time that Ethernet was crushing other protocols to become the leading best-effort commodity network protocol. That's why I expect that future arrays will use FC-NVMe for performance and reliability, and commodity servers will use NVMe over TCP (or, as we Brocadians like to call it, “iNVMe”).  It’s kinda like, déjà vu all over again, or maybe Back to the Future, storage style.  But definitely not Groundhog Day… yet.

( Next gen iSCSI font credit: https://fontmeme.com/fonts/starnext-font/ )

Labels