Data Engineer

Job Description

Job Overview:

. To work on development, monitoring and maintenance of Data pipelines across AE clusters.

Primary responsibilities:

. Develop, Monitor and Maintain data pipeline for various AE plants.

. Create and maintain optimal data pipeline architecture.

. Assemble large, complex data sets that meet functional / non-functional business requirements.

. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designinginfrastructure for greater scalability.

. Work with stakeholders including the Data officers and stewards to assist with data-related technical issues and support their data infrastructure needs.

. Work on incidents highlighted by the data officers.

. Incident diagnosis, routing, evaluation & resolution.

. Analyze the root cause of incidents.

. Create incident closure report.

Qualifications

Qualifications

• Bachelor's degree in Computer Science, Electronics & Communication Engineering, a related technical field, or equivalent practical experience.
• 3-4 years of experience in Java or Scala software development.
• Experience in large-scale software development.
• Excellent software engineering skills (i.e., data structures, algorithms, software design).
• Excellent problem-solving, investigative, and troubleshooting skills.

Additional Information

Skills

• Self-starter and empowered professional with strong execution and project management capabilities
• Ability to collaborate effectively, well developed inter personal relationships with all levels in the organization and outside contacts.
• Outstanding written and verbal communication skills.
• High Collaboration & a perseverance to drive performance & change

Additional information

Key Competencies-

• Distributed computing systems
• Experience with CI/CD tools such as Jenkins or Github Actions
• Experience with Python programming
• Good working Knowledge of SDLC.
• Hands on experience of shell scripting.
• Hands on experience with SCM tools (git/bitbucket)
• Working knowledge of Docker & Kubernetes
• Experience in developing data pipelines using spark & scala.
• Knowledge of kafka/flume.
• Problem solving skill.
• Experience of customer experience.
• Good debugging and troubleshooting skill.
• Experience of code review.
• Experience in writing python and shell scripts.
• In-Depth Knowledge of SQL and Other Database Solutions
• Having a strong understanding of Apache Hadoop-based analytics
• Hands on experience on InteliJ, Github /Bitbucket, HUE.

Soft Skills:

. Good Communication Skills

. Ability to use extensively the latest IT communication tools

( Skype /MS TEAMS etc.)

. Decent Level in English as Business Languagew