Why enter Snowflake Data Clean Room?

In today’s market there is tremendous pressure to make data assets trustworthy and ready for business use. C-Suite wants to monetize data by making it available to the desired customers real-time or near-real time. Data enrichment from diverse data sets brought together by fostering partnerships and collaborations with stakeholders brings in real data insights. This can further lead to innovation and competitive advantage in the marketplace.

The traditional approach of bringing data together from different business partners has been particularly challenging because it involved physically copying the data. We do not need a reminder of the cumbersome process of having to configure security settings such as user authentication, SSL/TLS encryption, security system rules and access controls. In this scenario data was only secure to the extent of how secure the data landing area was. Data Ingestion into consumer specific data landscape always required to build ETL/ELT data pipelines.

In the above context, the Secure Data Sharing feature of Snowflake is a great bonus where the need for copying and moving data has been eliminated. The process is amazingly simple and direct if both partners involved have their own snowflake environment. Secure data sharing does not move or copy data, thus removing the cumbersome process of ETL. With secure data sharing, data owners can monetize their data assets. The data provider gives secure and revocable access to consumers.

One caveat in the secure data sharing feature is that, when shared, data is available to the consumer as is. Any restricted data that is not shareable needs to be filtered out before sharing. Parties providing and consuming data sets must agree to share the attributes that are shared across.

Data Clean Room (DCR) framework offered by Snowflake however makes the sharing option further controlled and regulated using additional layers of data protection and security. DCR framework is a true marvel in the technology marketplace to share data securely between two or more organizations or even internal customers of an organization. Unlike secure data sharing, this framework adds more layers of security and restricts accessible rows and columns by rules and policies defined and implemented by the provider. This is important for businesses having sensitive, confidential and regulated data attributes that must not be shared with other parties. Personally Identifiable Information (PII) must not leave the organization where it is generated. In such cases, generally data is aggregated before sharing, which limits the scope of analysis that can be done on this data set. Data Clean Room is a neat solution in such cases. The data set from provider can be shared with multiple consumers. A provider and consumer can share data with each other (bi-directional partnership) as well. Data is made available to the consumer in near-real time. The shared data using DCR framework is not moved, copied, or leaves the Snowflake environment of the data owner.

Below is a sneak-peek into the Data Clean Room wherein a 3-step process makes data available to the consumer for a joint analysis and produces customer targeted cost aggregates.

Figure 1

Room does not mean a special physical space but indicates extra layers of security added by the framework to protect sensitive and regulated data, making it available for combined analysis with the data set in consumer data landscape. The provider defines rules about the type of queries that can be run on the data. The rules are tagged as query templates and shared with consumer. This establishes transparency and trust between the parties regarding the shareable attributes based on both parties’ consent. The rules are defined using secure views, row access policies, secure data sharing and stored procedures in such a way that the consumer is prevented from running queries that will expose any underlying raw, sensitive data.

A Data Clean Room (DCR) in the context of a hospital (Wellness Hospital) and Health insurance company is depicted in the Figure 1 image. In this scenario, the health insurance company is consuming patient data from hospitals to process claims more efficiently and accurately. DCR framework allows Hospital to share patient data ensuring that privacy and security regulations such as GDPR (General Data Protection Regulation) and HIPAA (Health Insurance Portability and Accountability Act) are followed. Some other extended use cases for DCR implementation could be that insurance companies can leverage a patient’s medical information, such as diagnosis, treatment, and medication data, to perform multiple functions, including validating claims, calculating appropriate reimbursement, identifying and preventing fraudulent claims, and improving the overall quality of care.

References:

Related posts:

Iniciamos a coluna de hoje com uma grande dúvida pairando sobre nós: ditadura merece comemoração? A morte de diversas pessoas merece comemoração? A censura e as agressões que foram acometidas a…