Vehicle intelligence firm Spireon is working with Snowflake to leverage Amazon Web Services’ data warehouse capabilities.
More than a few IT organizations have naturally gravitated to the Redshift data warehouse service provided by Amazon Web Services (AWS) when they first start to make extensive use of a public cloud. But as they begin to better understand the requirements of emerging Internet of Things (IoT) applications, alternative data warehouses running on AWS that enable IT organizations to analyze data in near real-time start to make a lot more economic sense.
Case in point is Spireon, a vehicle intelligence company that makes use of the AWS cloud to track nearly four million vehicles generating billions of events. Dave Withers, director of information architecture for Spireon, says initially the company relied on AWS Redshift to analyze, for example, global positioning system (GPS) data.
See also: Data warehouse wars to heat up in 2018
But it quickly became apparent that being charged on an hourly basis to process that data was becoming costly. Spireon opted to replace AWS Redshift with a Snowflake data warehouse service provided by Snowflake Computing that also runs on AWS mainly because billing for usage of that service occurs on a per second basis.
Withers says Spireon ingests massive amounts, so if every time the company imports data into AWS Redshift it requires a one-hour commitment even thought Spireon is only using the service for a few seconds at a time.
“Those costs can really add up,” says Withers.
Spireon is also making use of Snowpipe, a tool that Snowflake Computing developed to automatically move data from the AWS S3 service into its data warehouse as soon as data arrives in the AWS storage service. Currently available in a preview mode only, Snowpipe is an extension of the AWS Simple Queue Service (SQS). Withers says it’s a lot less expensive to store data using Snowflake Computing that it is to pay to store data directly on the AWS S3 service for an extended amount of time.
Data Warehouse Time & Cost Savings Sought
To further reduce costs, Withers says Spireon moves data into the AWS Glacier data archiving service as quickly as possible, which allows Spireon to store data at costs below pennies per gigabyte.
The best part, says Withers, is that Spireon has been able to reduce the amount of time it takes to process and analyze GPS data to under an hour.
Arguably, the most interesting aspect of how Spireon is employing the Snowflake Computing service is that it highlights how vigorous competition between AWS and third-party data warehouse service providers can be on the same public cloud. AWS views AWS Redshift as an opportunity to generate higher margin revenues on top of a base storage service. Conversely. rivals such as Snowflake Computing are providing an alternative approach based on their own relational database technology at much lower cost.
The underlying database Snowflake Computing developed is based on a relational database that is designed from the ground up to run on clusters made up of unified instances of compute and storage that dynamically scales out in a public cloud.
It’s too early to say how “co-opetition” between AWS and rivals such as Snowflake Computing will evolve over time. Snowflake Computing recently raised another $105 million in funding and is led by CEO Bob Muglia, who previously ran several different divisions of Microsoft. At the close of 2017, the company now has over 900 customers. That may not be enough to threaten AWS hegemony. But it is enough to make Snowflake Computing an alternative approach to data warehousing at a fraction of the cost.
