ROLE PURPOSE
n
The purpose of the Data Engineer is to provide skills for working on Projects and Programmes on Azure Cloud Data and Analytics Platform
n
KEY ACCOUNTABILITIES
n
- Provide design thinking develop technical designs, hands on involvement with code deliverables
n
- Review of deliverables from team members, provide day-to-day support and mentoring to team members
n
- Perform role of individual contributor. Build and deploy data mappings, sessions, and workflows in Azure Data Factory, Databricks.
n
- Utilize Azure Cloud Technology and Python tool sets to develop complex data pipelines.
n
- Conduct data pipeline performance tuning, troubleshooting, support, and capacity estimation.
n
- Implement and refine required business transformation rules and logic.
n
- Profile and understand large amounts of source data available, including structured and semi-structured/web activity data.
n
- Conduct thorough testing of ETL code changes to ensure quality deliverables
n
- Provide release notes for deployments, review release notes
n
Keep up to date with new skills - Develop technology skills in all areas of Platform
n
CUSTOMER CONDUCT FRAMEWORK
n
FUNCTIONAL / TECHNICAL SKILLS
n
n
Skills and Experience:
n
- Strong technical background, proven experience in delivery of high-volume Data applications
n
- Strong Analytical and troubleshooting skills
n
- Highly experienced with Microsoft technologies SQL, Azure native data products
n
o Strong knowledge of - Azure Data Factory (Preferred) OR Azure Data Bricks (Python)
n
- Experienced in ETL tools, Data Projects
n
n
- Experience with Python programming would be highly desirable
n
- Experience with Azure Functions would be a plus
n
- Experience working with Real time Integrations would be a plus
n
n
n
DECISION MAKING AUTHORITY/ IMPACT
n
- Technical decisions are expected to be taken which impact Data Projects delivered on Azure Cloud
