Wind Turbine Predictive Maintenance on Azure Databricks – Part 4: Delta Lake Enhancements and Synapse Analytics Storage
In part 4 of 6 of implementing a real-time Wind Turbine Predictive Maintenance application, we continue Delta Lake processing by bringing data to Silver and then Gold level of enhancements.
For demo purposes, we also expand the data in Delta Lake to provide a richer historical dataset to be used later for predictive modeling by generating data for a year for multiple wind turbines. Following that, we store some of the data in a Synapse Analytics database. We also optimize the physical data in Delta Lake and create a view that combines all the data we will use for training our model.
