Overview
The Azure Data Engineer plays a crucial role in designing, implementing, and managing data solutions on the Azure platform. This contractual position requires an individual with a robust understanding of cloud-based data architecture and the ability to process large datasets. By leveraging Azure services, the Data Engineer ensures that the organization's data management strategies align with business needs. The role is pivotal in facilitating effective data-driven decision-making by creating efficient data pipelines, maintaining data quality, and ensuring optimal data accessibility. Furthermore, the ideal candidate will collaborate closely with data scientists, software developers, and business stakeholders to support various data analytics initiatives. This position not only demands technical expertise but also requires a keen analytical mindset to transform complex data into insightful information.
Key Responsibilities
- Design and develop effective data pipelines using Azure Data Factory.
- Implement ETL processes to extract, transform, and load data from various sources.
- Manage and optimize SQL Server databases for maximum performance.
- Collaborate with data architects to build data models that meet business requirements.
- Utilize Azure Synapse to analyze large datasets for business intelligence.
- Integrate Python scripts to automate data processing tasks.
- Ensure data quality and accuracy throughout the data lifecycle.
- Document data workflows and architecture for future reference.
- Conduct data profiling to discover data inconsistencies and trends.
- Develop dashboards and reports using Power BI for data visualization.
- Work closely with cross-functional teams to gather business requirements.
- Monitor and troubleshoot data pipelines and infrastructure issues.
- Stay updated on emerging Azure technologies and data engineering best practices.
- Provide recommendations for improving data management processes.
- Assist in the migration of data to the Azure platform from on-premises solutions.
Required Qualifications
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- A minimum of 5 years of experience in data engineering or data analysis.
- Proficiency in Azure services, especially Azure Data Factory and Azure Synapse.
- Strong knowledge of SQL and experience with SQL Server.
- Experience in developing and optimizing ETL processes.
- Familiarity with data modeling techniques and tools.
- Experience with Python and its libraries for data manipulation.
- Knowledge of data warehousing concepts and implementations.
- Experience in creating visualizations using Power BI.
- Strong analytical and problem-solving skills.
- Ability to work collaboratively in a team-oriented environment.
- Excellent verbal and written communication skills.
- Experience with version control systems (e.g., Git) is a plus.
- Certification in Azure Data Engineering is preferred.
- Ability to manage multiple projects and meet deadlines.
- Strong attention to detail and commitment to quality.
Skills: sql server,data modeling,python scripting,power bi,data warehousing,azure,etl
