We are seeking skilledData Engineers with prior experience on Azure data engineering servicesand SQL serverto join our team. As a Data Engineer, you will be responsible for designing and implementing robust data infrastructure, building scalable data pipelines, and ensuring efficientdata integration and storage.
Responsibilities - Design, develop, and maintain scalable data pipelines and use Azure Data Factory and Azure Stream Analytics
- Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learninginitiatives.
- Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness.
- Implement data governance and security best practices to ensure compliance and data integrity.
- Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
- Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.
Qualifications - Proven experience as a Data Engineer or in a similar role.
- Experience in designingand hands-on development in cloud-based (AWS/Azure) analytics solutions.
- Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, Azure App Service, Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.
- Experience in SQL server and procedures.
- Thorough understanding of Azure Infrastructure offerings.
- Strong experience in common data warehouse modelling principles including Kimball, Inmon.
- Experience in additional Modern Database terminologies.
- Working knowledge of Python or Java or Scala is desirable
- Strong knowledge of data modelling, ETL processes, and database technologies
- Experience with big data processing frameworks (Hadoop, Spark) and data pipeline orchestration tools (Airflow).
- Solid understanding of data governance, data security, and data quality best practices.
- Strong analytical and problem-solving skills, with attention to detail.
