POC and Test Plans for AFAs
With a “virtualize everything” approach to performance-oriented applications in the datacenter, identifying the right storage solution is a major concern. When a user wants to augment or migrate from traditional HDD storage to flash-based storage, it can be a daunting task to evaluate which storage achieves top application performance.
What’s an end user to do?
To identify which flash-based storage array will provide the highest benefit for the applications and workloads that will run on it, testing your applications in a Proof of Concept (POC) helps an end user gain real-world insight into whether the array will, in fact, hit all its required marks.
Applications that reside on flash-based storage react differently and have different requirements compared with apps that run on HDD-based storage. For example, flash-based storage provides IOPS that are an order of magnitude higher than traditional HDD; and randomized I/O is preferred given the capabilities of SSD within an all-flash array.
IDC, a market research firm that specializes in Information Technology, recently released the “All-Flash Array Performance Testing Framework.” This framework focuses on the specific items that end users should be aware of when testing an all-flash array (AFA). This framework includes insight into how AFAs differ in both technology and testing methodologies compared with HDD-based storage.
Tegile and IDC inherently agree with the different ways that an end user can set up a successful POC for an AFA and the different ways that the actual tests can be run. Below, see our recommendations for how to test Tegile arrays and note how closely our recommendations mirror those of IDC.
From a Tegile perspective, there are four general approaches that are effective to evaluate flash-based storage. Approach 1 & 2 focus on the end-to-end infrastructure. Approaches 3 & 4 focus on achieving the max capability of the storage array. These are the approaches that we see our customers take when testing our products. Each approach has its own advantages. At Tegile we work collaboratively with customers to identify the best method for them to test their unique mix of applications and workloads. When customers use the right POC method, they can make the right decision as to which storage solution supports their applications and workloads best.
Top 4 Approaches to a POC
The advantage of this approach is that it gives one a direct comparison of how much the array can accelerate one’s applications compared with existing storage.
This approach is very similar to IDC’s Class A testing: “Class A testing, if you can do it, would provide the most accurate way to forecast how an array would perform in production with your actual workload.”1
The challenge is that sometimes because of the limitation of the infrastructure and the existing application limitation, the results of this type of POC may not represent the full capability of the flash-based array, although the customer would still see significant improvement in terms of overall system performance.
This approach can be highly effective to measure overall system performance and can mimic the results very closely, based on the application type that is tested. The challenges are having the adequate server and networking resources to run benchmarking tools (such as HammerDB for DB and LoginVSI for VDI). Other challenges are the time to set up different environments for each application-type, plus configuring the end-to-end setup to achieve the highest performance at an overall system level.
This approach resembles IDC’s Class B test type, which states: “Class B testing can produce very accurate results as well but can be very time consuming if you model each application separately and can add additional cost.”2
This approach usually entails using application-specific benchmarking tools, such as vdbench or SQLIO. This approach can be highly effective when testing needs to be run for a set of specific applications, such as VDI, SQL, etc. The challenge of this approach lies in designing the suitable workloads and test configurations that can closely simulate one’s application datasets (R/W ratios, variable block sizes, etc.)
IDC’s Class-C test type also states: “Class C testing is more of a generic testing methodology and is generally more time efficient than Class B…”3
In all engagements, Tegile has always offered a complimentary performance analysis of an end user’s current compute state. This performance analysis is of the user’s current application environment in regards to R/W ratio, IOPS, block sizes, etc. Our analysis tool is lightweight and leverages Perfmon, Iostat and ESXTOP. Based on the results, synthetic workloads can be defined to simulate the real application workload. Performance can also be measured by using the synthetic benchmarking tools, as mentioned above in POC Approach #1. This methodology works extremely well in an environment where I/O is highly dynamic and randomized, and where the user wants the design of the application workloads to mirror, as closely as possible, their production workload. This approach is also one of the most time and resource-efficient testing methodologies.
IDC’s Class D test type uses “generic data sets, data streams, and workflows designed to closely model 3rd Platform computing workloads.”4 This “Class D testing will produce results that are also meaningful for comparisons and may be easier to do than the other three higher classes.”5
How do you know the appropriate test and testing tools are leveraging the correct data sets and data streams for those specific tests? After all, unless you apply the appropriate R/W ratios, block sizes and I/O attributes to your testing tools, your testing results will not effectively achieve the results needed for your specific workloads. In comes the Tegile POC Playbook.
POC Playbook for Enterprise Storage
To achieve results for users in a timely fashion with the most benefit, Tegile created the Performance POC Playbook. Tegile customers use our POC Playbook to make the most educated decision when reviewing solutions for storage for their specific use cases. This POC Playbook is designed for end users who want to test Tegile IntelliFlash all-flash arrays and IntelliFlash Hybrid arrays because Tegile provides both.
To take the guess work out of setting up the most appropriate test plan, Tegile Sales Engineers provide potential users with the Performance POC Playbook. This playbook, in conjunction with the performance analysis we complete, includes proven testing tools, tests optimized based on the most common application workloads (and their respective data sets and data streams), and best practice recommendations to help streamline POCs with predictable and relevant test results.
To learn more about the playbook or engage in a POC, reach out to us at firstname.lastname@example.org. We’re eager to share how our Performance POC Playbook can help you identify which storage solution can do the most for your business.
1 – 5 Eric Burgener, IDC – AFA Performance Testing Framework, page 8.