With the creation of the new Tegile IntelliFlash App for Splunk, today we announced an ability to run Splunk on Tegile.
Capitalize on Big Data & the Internet of Things (IoT)
IT pros are aware that machine data is one of the fastest growing and most complex areas of Big Data; with the outlook of the Internet of Things (IoT), it’s also one of the most valuable.
It’s one thing to store data created by all of these machines, which Tegile has done well for years. It’s another thing, with Splunk on Tegile, to use the data we store to provide users with actionable events to simplify their storage management with Splunk.
The Tegile IntelliFlash App for Splunk helps our customers monetize their data in new ways. With this app, Tegile allows organizations to store, access and analyze all of their machine data. As storage pros know, real-time analytics have a 24/7 appetite for storage. See below top use cases.
|How Splunk on Tegile
Optimizes Storage Management
Top 3 Use Cases
1) Predictive analytics for IT operations
2) Security information and event management
3) IoT and real-time business analytics
Splunk on Tegile Solution Offering
First off, Splunk is a data processing platform engine used to classify and query machine-generated data (MGD). Splunk places data into a repository that can be queried by administrators and uses a “pooling” architecture to place data in tiered buckets: hot data, warm data, and cold data. Tegile uses a very complementary architecture.
|Tegile is Purpose Built for Splunk|
|Tegile automatically places
hot, warm & cold data from Splunk buckets
into the appropriate storage layer on the array
Splunk on Tegile
During hot to warm/cold data bucket movement, Tegile reduces the overall workload for Splunk, array and network. In contrast, a number of different issues can occur with other solutions that don’t have this specific functionality.
Other vendors have offerings for Splunk, but they are not as streamlined as Tegile storage. For some vendors, Splunk hot buckets are segregated into small, fast volumes and warm/cold buckets are segregated into large slow volumes (see diagram at right).
Second, during hot to warm/cold data movement, Splunk must read the entire bucket from the fast volume, then write all that data to the warm/cold bucket. This becomes a big issue for enterprise environments where an admin would have to manage hundreds of volumes separately.
Or, as some storage vendors require silos of storage to achieve functionality similar to that of Tegile, separate storage array appliances are required to provide both performance and capacity while traversing a network, which can induce additional latency during queries.
What’s the downside of storage alternatives for Splunk?
The above alternate solutions have these downsides:
- Increased overhead for Splunk
- Increased overhead for the arrays themselves
- Increased network latency
- Increased complexity of deployment and management
The below diagram is another illustration of the increased complexity of storage alternatives for Splunk.
Note how data is segregated across multiple arrays,
which increases complexity and latency.
Benefits of “Real-Time Caching” with Tegile
Tegile reduces query times on Splunk by providing flash-optimized storage and metadata. Because Tegile arrays automatically place hot, warm and cold data in the same array, filesystem and volume, the movement of data becomes a simple metadata update within the array.
|Benefits of “Real-Time Caching” with Tegile|
Tegile’s patented metadata acceleration process isolates and aggregates metadata on flash, which reduces query time significantly. Also, upon data ingest, Tegile compresses the already compressed Splunk data, which results in improved data reduction ratios and cost savings.
Even though Splunk will compress raw data, Tegile is able to achieve additional data reduction savings (via inline compression and deduplication) on Splunk’s index files. When Splunk compresses raw data and stores it, it also creates time series index files (tsidx) that associate terms in the raw data with the offset (in the raw file) of the occurrence of that term.
Tegile facilitates further reduction of those tsidx files beyond that which Splunk achieves on its own. This is especially true in a indexer cluster, where data can repeat itself. In our own testing, an additional 40% data reduction was achieved.
Existing Splunk Users
If you’re an existing Splunk user, you may wish to take next steps. You might ask yourself these questions:
- How do I integrate storage into my Splunk UI for embedded insights?
- How do I upgrade my existing slow legacy, disk-based storage for Splunk?
- How do I simplify and reduce the cost and risk of my existing Splunk architecture?
If you are ready for next steps, why not download the reference architecture and see how the Splunk on Tegile solution can help.
Existing Tegile Customers
If you’re an existing Tegile customer and you want to use Splunk or use it already, we recommend trying our new Tegile IntelliFlash App for Splunk. As mentioned above, this app provides a single-pane-glass plugin that collects pertinent information from Tegile arrays and displays it directly into the Splunk user interface.
The type of data retrieved includes array controllers, pools, volumes and mappings to assist administrators in monitoring and management. This data allows more granular management and monitoring capabilities when storing Splunk data on Tegile. The installation is very easy and can be completed in about 5 minutes.