ISC 2019 is the highlight event of the year for Europe’s high-performance computing (HPC) community as it gathers over 3,500 researchers. Excelero and its partners are using ISC to spotlight new solutions and innovative scale-out HPC storage deployments at leading research and scientific institutions in Europe.
Excelero powers Arcastream’s PixStor
As HPC-deploying organizations struggle to scale storage architectures to store and analyze massive volumes of data throughout its lifecycle, the example set by Excelero and ArcaStream will give attendees a wealth of new ideas on how they can leverage NVMe storage so it can be shared across the network at local performance throughout the lifecycle today – and scale to meet future demands with confidence.
Databases, AI and ML and other HPC use cases often require the use of a parallel file system to retain application workflows, yet need a software-defined block storage solution underlying it to meet demands for ultra low latency and superior performance. IBM Spectrum Scale is a popular file system choice that is proven in numerous HPC workflows.
Excelero and ArcaStream recently partnered at Imperial College London, a major science and technology research center, to deploy Excelero’s NVMesh® software-defined block storage and the PixStor high performance scalable storage platform based on IBM Spectrum Scale. PixStor combines flash, disk, tape, and cloud storage into a single global name space. With a software-defined architecture, it uses open standard commodity hardware to avoid vendor lock-in coupled with powerful data management tools – including tiering, cloud integration, monitoring, search and analytics – to drive workflow efficiencies and reduce costs.
Shared NVMe at Local Performance for high-performance metadata tier
Excelero’s NVMesh storage software integrates seamlessly with the PixStor platform to enable a shared NVMe Flash metadata tier for Imperial College and others. Excelero NVMesh takes NVMe drives in individual servers, or dedicated storage servers, and virtualizes them into aggregated volumes that can be rapidly accessed as a shared resource by multiple clients over simple Ethernet. Excelero NVMesh with PixStor uniquely delivers centralized management, redundancy and high utilization rates, while maintaining local NVMe performance.
With thousands of HPC-deploying institutions around the world facing the same challenge – surging storage volumes yet flat or barely growing budgets – the example set by Excelero and ArcaStream provides a powerful example of what’s possible with next-generation software-define storage options.
Speed, Ease and Scalability: a Future-Proof Architecture
Customer ROI of the solution is immediate and impressive: system usability compared to traditional proprietary storage deployment improves dramatically, with end-user performance rock-solid and consistent. With the Excelero-Arcastream solution researchers can store and access data with ease and speed they have never had – performance that also helps labs intelligently manage growing storage demands. More cost-effective management of storage capacity allows customers to avoid costly storage additions. Ready scalability enables a future-proof architecture that can grow as its needs grow.
NVMesh Supercharges BeeGFS: Excelero and ThinkParQ Go to Market with joint solution
Announced last month, Excelero and thinkParQ’s joint go-to-market relationship is bearing fruit, and presents another choice of scale-out file system for HPC deployments. The joint solution features NVMesh® software-defined block storage solution for shared NVMe at local latency along with ThinkParQ’s leading parallel cluster file system BeeGFS. Benchmark results from early deployments spotlight the maximum resource utilization and flexibility that are possible from combining the two solutions – performance that is perfectly suited for data-intensive workloads such as high performance computing (HPC), AI, ML, and analytics.
In response to customer demand, Excelero and ThinkParQ already have deployed NVMe storage clusters with BeeGFS and NVMesh to increase productivity of advanced GPU compute clusters. These organizations experienced the storage bottleneck issue endemic to AI – that I/O intensive applications cannot be processed fast enough even on GPU-based systems. As a result, IT teams often copy datasets in and out locally, which is an inefficient and time-consuming process.
To demonstrate the possibilities of this new scale-out infrastructure, the companies used the industry standard mdtest and IOR benchmarks. The test system was a compact 2U 4-server chassis with a total of 24 NVMe drives, connected via a 100Gbit RDMA network to 8 BeeGFS client compute nodes. Tests were run on the exact same hardware configuration with BeeGFS utilizing the direct-attached NVMe vs. BeeGFS utilizing NVMesh logical volumes. Taking advantage of NVMesh to offload mirroring operations, BeeGFS file create operations were boosted 3x, while metadata read operations were boosted 2.5x. For small random file access, which is often considered to be especially critical for application efficiency, NVMesh’s low latency technology boosted BeeGFS 4K write IOPS to 1.25 million per second, a 2.5x improvement. By leveraging NVMesh distributed erasure coding for BeeGFS, customers can get up to 90% usable capacity while still tolerating drive failures. All the while, achieving 75GB/s streaming throughput from this entry-level system.
These and other groundbreaking innovations for the HPC market by Excelero and its partners will be demonstrated live at ISC. Visit Excelero in Hall 3, Booth E-1039 to learn more.
