Data Pipeline for Marketing KPIs from multiple sources
Problem
One of the most common challenges when creating and planning a marketing campaign is how to consolidate all of the data from all of the platforms where the campaign is promoted in order to see the KPIs and also make comparisons between the platforms (such as how many clicks did the campaign had on Facebook, Google, etc.).
Besides clicks and views, for the end-user, it’s important to see how the money is spent (which of the platforms is more efficient for a particular campaign).
Solution
We used Azure SQL and VM technologies to implement a Python/SQL solution. The different marketing platforms, each with its APIs, offer the data in their unique forms, such as tables, reports, etc. Having that in mind, we developed a real-time gathering of data from these APIs, formated and transformed the data in a compatible way, and finally stored the transformed data in a Data Warehouse.
Results
We developed a system that consolidates data in real-time. This system collects, transforms, and stores marketing data from different sources into a single Data Warehouse. The system makes it easy to analyze all data through simple visualization and storytelling that is easy to navigate and understand for the end-user (such as marketing agencies or individual companies).
