Aerospike embraces new, JSON-ready document model for its database

This week, Aerospike, a company that provides real-time data and hybrid memory for its storage, announced the sixth edition of their database designed for supporting large data flows at extremely high speeds. The move brings modern protocols and formats to its core engine that’s designed to support contemporaneous data processing and storage.

Aerospike’s database gained traction with companies confronting large, continuous data streams from relentless and indefatigable sources like online games, connected devices or online ads. Sony, for example, uses the database to support personalizing experiences for PlayStation consoles. Dream11 manages more than 100 million fantasy sports players in India who track the performance of their teams. 

In the past, some engineers have raved about the database and one called it an “an incredible piece of engineering of a database, all optimized towards a single use case: guaranteed single-millisecond reads and writes on a key-value store that’s too expensive to fit in RAM.”

The new version maintains that same focus, but expands the storage model to offer more complexity while also improving other performance by adding better indexing and more capability for backups. 

“Our focus is to take that goodness that we have with the database and expand it to a much broader set of use cases.” explained Subbu Iyer, the CEO of Aerospike. “That’s why we focused on really going after building a native document model.”

The document model differentiator

One of the most prominent features of the new sixth edition is support for JSON format and what database administrators often call the “document model”. Developers enjoy using the format because JSON is the native data structure for JavaScript, the common language for browser-based applications. They can often send information directly between the database and the users’ browsers with little or no intervention. 

Using the same JSON-centric model throughout the stack can remove the need for elaborate rewriting or reformatting, a step that can slow performance and require more developer time. The data flows easily between the user’s browser and the database, a feature that can decrease latency and speed up throughput, both important features for the product. 

The JSON focus is not just for databases relying on document models for storage. Oracle, PostgreSQL and MySQL are just some of the traditional databases that have added the option over the years. Some newer databases like Couchbase were designed around the standard. 

Not all of these options may scale as easily nor support the same low latency. In its marketing material, the company says, “first and only real-time data platform to support JSON document data models that can deliver sub-millisecond performance at the gigabyte-to-petabyte scale.” 

The company regularly compares their product favorably to other lightweight data storage solutions like Redis or Cassandra. The most common competition may be clusters of MySQL NDB installations. 

“We don’t see ourselves playing in the simple session-store application, which have to stitch and store some JSON documents that come and go. That’s not the place we play.” said Iyer. “But as you scale your document-based workloads and you start kind of getting some level of scale half a terabyte to a terabyte plus, that’s where we see a really good sweet spot from there all the way up to petabytes.”

Complicated data made easy

The new model also allows developers to create more complicated data structures on the fly, adding new fields wherever they may be useful. This flexibility allows developers to deploy different structures for each object in the database. While this flexibility prompts debate, some companies like MongoDB continue to do very well with it. 

“A lot of our customers today are using us for only a couple of different use cases and they obviously have a lot more applications and developers who are building applications with documents as their primary storage model.” explained Iyer. “We’re already talking to customers who are saying, ‘Oh, now I can actually use you in a much broader set of applications’.”

The new version also includes several other features to expand support for different roles. The latest version also improves regulatory compliance for Federal users. The new version now supports FIPS 140-2 which opens up the opportunity for the company to support some of the biggest data.

“We built in the ability to back up and restore to S3.” added Iyer. “A lot of people want to use S3 as an archival storage, so they want to move data away from the active database into S3. We allow for that both ways, which is backup and restore.”

The company has also improved the speed of the secondary indices so they can perform as quickly as the primary indices. In the past, fast responses were only available to queries that relied upon the primary index. 

The new announcement also starts to chart some paths ahead for the company. They want to expand the database to tackle more event-driven data streams for time-series data, often from log files. They also intend to evaluate adding graph functionality to help make smarter decisions about networked data sets. 

At the end of the briefing, Bill Odell, Aerospike’s chief marketing officer, pointed out that the new version is already ready to run and is currently available.