Senior Data Engineer

Experience: 6+ Years

Location: Hyderabad

Primary skill set: Azure Data factory, Databricks, Datalake,SQL, SPROC'S

Notice period: Immediate to 20 days

6+ years of experience in data engineering or related roles. Proficiency in Azure Data Factory (ADF) for building and managing data pipelines.

Lead the design, development, and implementation of data pipelines using Azure Data Factory (ADF) and Databricks Notebooks.

Architect and optimize data warehouses and data lakes for performance, scalability, and reliability.

Develop and optimize stored procedures (SPROCs) for efficient data processing and manipulation.

Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.

Implement best practices for data governance, data quality, and data security across the data infrastructure.

Manage and monitor data pipelines to ensure reliability, availability, and performance.

Stay current with emerging technologies and industry trends in data engineering and recommend innovations to enhance our data ecosystem.

Provide technical leadership and mentorship to junior members of the data engineering team.

Participate in code reviews, performance optimization, and troubleshooting activities.

Required Skills and Qualifications: Bachelor's degree in Computer Science, Engineering, or related field.

Strong expertise in SQL Server and T-SQL programming.

Experience with Databricks Notebooks for data exploration, analysis, and processing.

Solid understanding of data warehouse and data lake architectures and best practices.

Proven track record of designing and optimizing data systems for performance and scalability.

Experience working with large-scale datasets and distributed computing frameworks.

Strong problem-solving and analytical skills. Excellent communication and collaboration skills.

Ability to work independently and effectively in a fast-paced environment.