Senior Data Engineer

Saras Analytics

Early Applicant

3 months ago
Be among the first 50 applicants

Exp: 0-5 Years

Hyderabad / Secunderabad, Telangana, India

Job Description

We areseeking a seasoned and proficient Senior Python Data Engineer with substantialexperience in cloud technologies.
As a pivotal member of our data engineeringteam, you will play a crucial role in designing, implementing, and optimizingdata pipelines, ensuring seamless integration with cloud platforms.
The idealcandidate will possess a strong command of Python, data engineering principles,and a proven track record of successful implementation of scalable solutions incloud environments.

Responsibilities:

1. Data Pipeline Development:
Design, develop, and maintain scalable and efficient data pipelinesusing Python and cloud-based technologies.
Implement Extract, Transform, Load (ETL) processes to seamlessly movedata from diverse sources into our cloud-based data warehouse.
2. Cloud Integration:
Utilize cloud platforms (e.g., Google Cloud, AWS, Azure) to deploy,manage, and optimize data engineering solutions.
Leverage cloud-native services for storage, processing, and analysis oflarge datasets.
3. Data Modelling and Architecture:
Collaborate with data scientists, analysts, and other stakeholders todesign effective data models that align with business requirements.
Ensure the scalability, reliability, and performance of the overall datainfrastructure on cloud platforms.
4. Optimization and Performance:
Continuously optimize data processes for improved performance,scalability, and cost-effectiveness in a cloud environment.
Monitor and troubleshoot issues, ensuring timely resolution and minimalimpact on data availability.
5. Quality Assurance:
Implement data quality checks and validation processes to ensure theaccuracy and completeness of data in the cloud-based data warehouse.
Collaborate with cross-functional teams to identify and address dataquality issues.
6. Collaboration and Communication:
Work closely with data scientists, analysts, and other teams tounderstand data requirements and provide technical support.
Collaborate with other engineering teams to seamlessly integrate dataengineering solutions into larger cloud-based systems.
7. Documentation:
Create and maintain comprehensive documentation for data engineeringprocesses, cloud architecture, and pipelines.
Technical Skills:
1. Programming Languages: Proficiencyin Python for data engineering tasks, scripting, and automation.
2. Data Engineering Technologies:
Extensive experience with data engineering frameworks like distributeddata processing.
Understanding and hands-on experience with workflow management toolslike Apache Airflow.
3. Cloud Platforms:
In-depth knowledge and hands-on experience with at least one major cloudplatform: AWS, Azure, or Google Cloud.
Familiarity with cloud-native services for data processing, storage, and analytics.
4. ETL Processes: Provenexpertise in designing and implementing Extract, Transform, Load (ETL)processes.
5. SQL and Databases: Proficientin SQL with experience in working with relational databases (e.g., PostgreSQL,MySQL) and cloud-based database services.
6. Data Modeling: Strongunderstanding of data modeling principles and experience in designing effectivedata models.
7. Version Control: Familiaritywith version control systems, such as Git, for tracking changes in code andconfigurations.
8. Collaboration Tools: Experienceusing collaboration and project management tools for effective communicationand project tracking.
9. Containerization and Orchestration: Familiaritywith containerization technologies (e.g., Docker) and orchestration tools(e.g., Kubernetes).
10. Monitoring and Troubleshooting: Abilityto implement monitoring solutions and troubleshoot issues in data pipelines.
11. Data Quality Assurance: Experiencein implementing data quality checks and validation processes.
12. Agile Methodologies: Familiaritywith agile development methodologies and practices.
Soft Skills:
Strong problem-solving and critical-thinking abilities.
Excellent communication skills, both written and verbal.
Ability to work collaboratively in a cross-functional team environment.
Attention to detail and commitment to delivering high-quality solutions.