Data Engineer--PD

Job Title: Data Engineer

Experience: 6+ Years (Relevant)

Work Mode: Remote

Notice Period: Immediate to 15 days

Job Description:

We are seeking an experienced Data Engineer with a strong background in data engineering technologies to join our remote team. The ideal candidate will have 6+ years of relevant experience and will be responsible for designing, developing, and maintaining robust data processing solutions.

Responsibilities:

Design, develop, and maintain data pipelines using PySpark.

Work with Azure Data Lake and Azure Databricks to manage and process large datasets.

Collaborate with data scientists and analysts to ensure data accuracy and availability.

Implement data integration and ETL processes to support business needs.

Optimize and improve data workflows for performance and scalability.

Ensure data quality and integrity throughout the data lifecycle.

Monitor and troubleshoot data issues and system performance.

Stay updated with the latest trends and technologies in data engineering.

Required Skills:

Strong experience with PySpark for data processing.

Proficient in working with Azure Data Lake.

Hands-on experience with Azure Databricks.

Solid understanding of ETL processes and data integration.

Strong problem-solving skills and attention to detail.

Excellent communication and collaboration abilities.

Technical Skills:

#PySpark, #DataLake, #AzureDatabricks