Announcing NVMesh 2

Introduction

This week we announced the availability of NVMesh 2. This release and its underlying technologies have been in the works in our labs for quite some time, so it’s a very exciting moment for both myself and the development team to see it coming out to help customers accelerate and optimize their most demanding workloads.

In this blog series, I will be covering the NVMesh 2 release and unfold its underlying technologies and their benefits to customers.

But first, a little bit of history. NVMesh was first released to the world back in early 2017 as the world’s first software-defined block storage truly designed from scratch to provide shared access to NVMe drives. Since then, NVMesh was adopted by customers looking to optimize the efficiency of their infrastructure, as well as accelerating their key applications. These requirements are nowadays not only catering to high-performance computing customers. NVMesh is now deployed in banks, cloud providers and media and entertainment studios.

NVMesh is now deployed in production in customers such as CMA, which have replaced legacy SAN to accelerate database transactions and batch jobs for one of the largest state agencies in the U.S. Or Technicolor, where NVMesh is at the core of the 4K/8K post-production of Hollywood’s top movies at 4K and 8K resolutions.

This market adoption has enabled us to improve the product’s overall robustness, but have also helped us to gather a lot of customer feedback and requirements.

What have we learned from customers?

First, we identified two major trends we see in the market:

• Enterprises are adopting software-defined architectures at scale, where application latency is key – Software-defined which used to be deployed in the corners of the data center is now deployed in masses. This challenges existing infrastructure, since customers are looking to provide the best possible applications response time from the hardware they already have. Nevertheless, for the most demanding workloads, they would like to have the option to upgrade hardware to get the lowest possible latency.
• New storage and networking technologies (100/200 GbE, RDMA, SmartNICs) enable scale-out in-server flash architectures – As vendors are coming out with new classes of media (NVMe, Optane, upcoming storage class memory and more) and networking (100/200GbE, RDMA, SmartNICs), it is evident that the hardware-focused approach the storage industry is no longer the right approach. Only a software-based approach can provide the best hardware efficiency by using the latest available technologies. In addition, it can also provide the best scalability – both down, up, out – any direction you want. Build using the hardware you already have, the lowest cost or the best performing – only you dictate.

What have we done with this feedback?

We decided to focus on a few key elements that came out from our discussions with customers:

• Optimize existing infrastructure – We need to provide low-latency access to a shared pool of NVMe drives to customers with standard Gigabit Ethernet networks. This will enable customers to reduce the latency of their existing racks simply by installing our software.
• Optimize the use of storage resources – With the decline in NVMe pricing comes the expansion of use cases for NVMe. As customers are implementing NVMe into additional use cases such as video post-production, they expect to benefit from high capacity efficiency. The expectation is two-fold – give me more space for the buck, but still provide me with very high performance.
• Simplify my administration – As IT gets more complex, customers are constantly looking for ways to simplify it. Simpler installation, less moving parts, less steps, more visualization.

We took all of the above and NVMesh 2 came to life.

So what is NVMesh 2 all about?

• Connectivity – By adding support for standard gigabit Ethernet and TCP/IP to the previously supported RoCEv2 and Infiniband, NVMesh 2 enables customers to gain from universal access to shared NVMe drives using their existing network infrastructure. Got an existing 10GbE network? We can let you deploy a software-based NVMe virtual SAN using this network. Got RoCEv2? We support that. Got Infiniband? We support it too. We call this flexible connectivity MeshConnect.

• Efficiency – With the addition of erasure coding N+M type of data redundancy, NVMesh 2 is now truly the only software-defined solution that offers customers with a parity-based data redundancy scheme that runs as a distributed service, meaning that the parity calculations are running on the clients, in a totally decentralized fashion. Why is that important? Because like any other component in our architecture, it was designed for large scale. Adding more clients will increase the overall CPU power available for parity calculations. No more choking of dual controller all-flash arrays. Now, this all comes with a choice – striping, mirroring and striping and mirroring are also available, providing customers with all the tools they need to build the infrastructure that best meets their fault domain designs. We call this overall choice of data redundancy schemes MeshProtect. 

• Analytics – What good is a high-performance solution if one cannot visually analyze the workloads running on it? NVMesh 2 provides a built-in, highly customized performance analytics platform. It provides detailed cluster-wide and per-object statistics on throughput, IOPS, and latency. We call this new capability MeshInspect.
• Simplicity – Sometimes an oversight by vendors, I am a big believer in user experience. Installation, first-time use, UI, documentation – these are all constantly challenged to provide a simpler overall experience. NVMesh 2 comes with less installation steps, a simpler user interface and expanded documentation than our previous release.

Summary

With NVMesh 2, Excelero delivers the lowest-latency distributed block storage for shared NVMe. It now provides customers the ability to benefit from this latency using their existing Ethernet networks. NVMesh now provides a distributed, parity-based data redundancy that provides both high efficiency and performance that can limitlessly scale. NVMesh now visualizes performance analytics which lets users the ability to better understand their workloads. All in all, NVMesh 2 sets a new standard for software-defined storage, with its performance, latency, scalability, flexible hardware support, and analytics.

Contact us to see a live demonstration of our capabilities.