This series of blog posts builds on our previous set on The Challenges of Repeatable and Idempotent Schema Management:
Blog - DataOps
Over the previous 2 blog posts, we have seen that managing the lifecycle of database objects in an idempotent manner is impacted by the imperative nature of most SQL statements, which require a known initial state for changes to be applied repeatably.
It may appear that most of this should be possible with native SQL statements and indeed some DDL operations in Snowflake are naturally idempotent, whereas others have impacts on data and object state, and some are not idempotent at all. Let’s look at some of the ways SQL tries to help us with this and the problems that remain.
This is the first in a series of blog posts discussing automating the lifecycle events (e.g. creation, alteration, deletion) of database objects which is critical in achieving repeatable DataOps processes. Rather than the database itself being the source of truth for the data structure and models, instead the code within the central repository should define these.
To paraphrase Datalytyx CTO Guy Adams’ words during the recent ‘Say hello to DataOps for Snowflake’ webinar, DataOps is as much a philosophy as it is a set of technologies and tools. And, because it’s a new area, a lot of people are still struggling to get to grips with it.
Unless it’s on Snowflake with Instantaneous Demand
One of the standard promises of Cloud Data Warehouses when compared to their on-premises alternatives is elasticity. As your requirements go up and down, the system can stretch or shrink with them. All such systems are capable of change to some extent. The question is all about timing.
And 5 Reasons Snowflake’s Data Sharehouse Does
For the past decade the technology world has been obsessed with APIs. While the nature of the APIs themselves has changed over the years (e.g. from SOAP to REST), the premise has always been the same – APIs are the way to allow different systems to share information. This doctrine is so well accepted, no one questions it. Certainly, for transactional processes, such as creating a user account, APIs work fine.
LONDON, United Kingdom, October 17th 2019 – DataOps has launched Gallium as the first of a new era of Snowflake Data Services.
LONDON, United Kingdom, October 17th 2019 – Over the last 2 years, DataOps has been developing Gallium (IoT Smart Data Compression Algorithm) technology, now launched as part of a new era of data services on Snowflake.
In the past 2 years, the rise of the time series database (TSDB) has been meteoric – growing faster than any other database model (as defined by db-engines.com):