Azure Data Engineer

Position :Azure Data Engineer

Experience : 6 Years 12 years

Location : Bangalore/ Remote

Notice Period: Immediate to 30 Days

Job Description :

We are seeking an experienced Azure Data Engineer to join our team. The ideal candidate will have a strong background in designing, developing, and managing ETL processes and data pipelines on the Microsoft Azure platform. Proficiency in Azure Data Factory (ADF), PySpark, and CI/CD pipelines is essential. The role requires a hands-on approach to design, modeling, development, and delivery, with a focus on scalable and reliable enterprise solutions.

Key Responsibilities:

Development:

• Hands-on design, modeling, development, and delivery of data solutions.
• Strong working knowledge of Azure services and components, with the ability to integrate various components into a scalable, reliable, and sustainable enterprise solution.
• Develop and maintain ETL processes and data pipelines using Azure Data Factory (ADF).
• Optimize data transformation processes using PySpark.
• Production experience delivering CI/CD pipelines across Azure and vendor products.
• Contribute to the design and development of enterprise standards.
• Key knowledge of architectural patterns across code and infrastructure development.

Requirements:

Technical Skills and Experience:

• Bachelor's or master's degree in computer science, Engineering, Data Science, or equivalent experience, with a preference for experience and a proven track record in advanced, innovative environments.
• 7-8 years of professional experience in data engineering.
• Strong expertise in Microsoft Azure data services, particularly Azure Data Factory (ADF) and PySpark.
• Experience with data pipeline design, development, and maintenance.
• Proficiency in SQL and experience with NoSQL databases.
• Knowledge of ETL/ELT processes and tools.
• Familiarity with big data technologies such as Hadoop and Spark.
• Experience with Azure Databricks, Data Lake, SQL Database, and other Azure data services.
• Production experience delivering CI/CD pipelines across Azure and vendor products.
• Knowledge of architectural patterns across code and infrastructure development.
• Fully conversant with big-data processing approaches and schema-on-read methodologies, with a preference for a deep understanding of Spark, Databricks, and Delta Lake.
• Experience with Agile delivery frameworks and tools (SAFe, Jira, Confluence, Azure DevOps).
• Experience with visualization tools (Power BI, developing reports, dashboards, KPI scorecards).

Preferred Qualifications:

• Azure Data Engineer certification.
• Experience with data warehousing concepts and technologies.
• Understanding of data security and compliance requirements.