Job Description
Responsibilities
Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements using Azure data factory and Azure databricks
Develop and maintain innovative Azure Azure data factory and Azure databricks
Solution design using Microsoft Azure services function app, logic app, Azure functions ,key vault and other tools
Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications
Build data pipelines to collectively bring together data
Other responsibilities include extracting data, troubleshooting and maintaining the data lakehouse
Tech Stack
Python,SQL and NoSQL databases, Spark-SQL, ADF , Databricks
Experience with Azure: ADLS, Databricks, SQL DW, Azure Functions, Serverless Architecture, ARM Templates, Azure DevOps
