Blog

Blog - DataOps

DataOps Launches Support for Snowflake Snowpark

DataOps Launches Support for Snowflake Snowpark

Introduction

Recently DataOps.live announced our support for Snowflake Snowpark.

Snowflake is known for its performance, scalability, and concurrency. Before Snowpark, users interacted with Snowflake predominately through SQL. Now, customers will be able to execute more workflows entirely within Snowflake’s Data Cloud, without the need to manage additional processing systems.

FUTURE PERMISSION GRANTs: why they are very useful and why you shouldn't use them in a DataOps world

FUTURE PERMISSION GRANTs: why they are very useful and why you shouldn't use them in a DataOps world

FUTURE GRANTS, ALL TABLES in the Snowflake Data Cloud (and similar constructs in other databases) are necessary and powerful tools for manually administered databases.  However, they have significant downsides in terms of flexibility, auditability, potential information bleeds, Principle of Least Privilege etc. In a DataOps approach all the same convenience is possible, but all of these limitations are addressed.

PART 2: The Challenges of Repeatable and Idempotent Schema Management: Idempotence in Snowflake

PART 2: The Challenges of Repeatable and Idempotent Schema Management: Idempotence in Snowflake

It may appear that most of this should be possible with native SQL statements and indeed some DDL operations in Snowflake are naturally idempotent, whereas others have impacts on data and object state, and some are not idempotent at all. Let’s look at some of the ways SQL tries to help us with this and the problems that remain.

PART 1: The Challenges of Repeatable and Idempotent Schema Management: Introduction

PART 1: The Challenges of Repeatable and Idempotent Schema Management: Introduction

This is the first in a series of blog posts discussing automating the lifecycle events (e.g. creation, alteration, deletion) of database objects which is critical in achieving repeatable DataOps processes. Rather than the database itself being the source of truth for the data structure and models, instead the code within the central repository should define these.