Job Title:Data Engineer- Architect
Location:Chennai
Position Overview:We are looking for an experienced Senior Data Engineer with 5+ years in the field to join our talented team. The ideal candidate will have a strong background in Python, AWS, Databricks, and Spark, and will be instrumental in designing and implementing robust data pipelines and architectures that drive business insights.
Key Responsibilities:
- Design, develop, and optimize scalable data pipelines using Python and Apache Spark.
- Leverage AWS services (e.g., S3, Redshift, Glue) to build and manage data solutions in the cloud.
- Implement and maintain data integration workflows on Databricks.
- Collaborate with data scientists and analysts to understand data needs and translate them into technical solutions.
- Ensure data quality and reliability through rigorous testing and monitoring.
- Optimize performance and cost-effectiveness of data processes and architectures.
- Document data architectures, processes, and workflows for future reference.
- Mentor and guide junior engineers to enhance team capabilities and knowledge.
Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field; Master's preferred.
- 5+ years of experience in data engineering or a related field.
- Strong proficiency in Python, with experience in data manipulation and ETL processes.
- Extensive experience with AWS services, particularly in data processing and storage.
- Hands-on experience with Databricks and Spark for big data processing.
- Solid understanding of data warehousing concepts and technologies.
- Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL).
- Strong analytical and problem-solving skills with attention to detail.
- Excellent communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.
