The data subject request (DSR) is one of the most challenging components of GDPR. It provides individuals in the EEA (European Economic Area) the right to request what personal data has been collected, how that data is being used and to have that data exported, restricted, changed, and erased. For many organizations that rely on data lakes to store their big data, sifting through millions of files to find and modify individual records for a DSR within prescribed GDPR timelines (typically 30 days) is a time consuming, and quite possibly, impossible task.
Fortunately there’s a path forward. Delta Lake, a powerful open source storage layer that brings reliability to data lakes, is natively integrated within the Databricks Unified Data Analytics Platform, and the two combine make it easy to quickly find and surgically remove individual records.