In the past 2 years, the rise of the time series database (TSDB) has been meteoric – growing faster than any other database model (as defined by db-engines.com):
LONDON, United Kingdom, October 17th 2019 – Over the last 2 years, DataOps has been developing Gallium (IoT Smart Data Compression Algorithm) technology, now launched as part of a new era of data services on Snowflake.
To deliver this, the DataOps team had to find new ways of triggering external data services from within Snowflake based on table updates without building external polling solutions.
DataOps has solved the challenge of triggering actions and processes in external systems from within Snowflake and has made the solution available to all Snowflake customers for free.
By implementing webhooks, Snowflake users can now take advantage of powerful bidirectional integration from Snowflake to external systems like Talend and Databricks.
Benoit Dageville and Thierry Cruanes of Snowflake expressed their enthusiasm for this solution.
“Thierry and I have been working closely with the DataOps team for the last year. DataOps are one of the most experienced teams around the deep insides of Snowflake and we have been enjoying working with them. DataOps work closely with Snowflake as part of their data platform, but are also specialists in IoT timeseries data. They have produced Gallium which is a unique time series compression algorithm with limited loss of fidelity. Whether its 5:1 or 500:1 compression, the business value to companies using Snowflake and Gallium for IoT timeseries data compression will be significant.
As part of the process to integrate Gallium directly into Snowflake, DataOps have developed an excellent opensource approach to launching external data services, like Gallium, from within Snowflake using only the Snowflake capabilities available today.
This is an example of great solutions coming from the Snowflake ecosystem to help Snowflake users. It’s great to see innovative solutions from leading partners like DataOps. This Gallium integration is the first of a new age of data services called natively from within Snowflake, and with the generous opensourcing of this code, we look forward to seeing many more.”
The use cases for webhooks in Snowflake vary from triggering simple Email or SMS alerts based on events in Snowflake to complex integration and orchestration with external systems.
For example, a process in Talend can trigger
Databricks users can also benefit from using webhooks in Snowflake. Many Databricks customers hold data inside Snowflake for training models. When a critical mass of new data has been added into Snowflake,
As a partner and service provider of Snowflake, Talend, Databricks, Azure and AWS, DataOps saw the need for a universally supported and
In the coming weeks, DataOps will be publishing more blogs detailing how they used these webhooks with Table Streams, how to use them for Data Services and more.