Hadoop Data Engineer

Role: Senior Software Engineer/Manager ( Specific to Data Engineering, software development roles)

Experience Range : 7 to 10 Years

Location - Bangalore

· 6-7+ years of Experience in Hadoop or any Cloud Bigdata components (specific to the Data Engineering role), Hadoop hands on (airflow, oozie, hive, hdfs, sqoop, pig, map, reduce )

· 4+ years of Exp in Spark (Spark Batch, Streaming, Mlib etc). Candidates should possess proficiency in utilizing the Apache Spark framework.

· 6-7+ years of experience in Python programming language.

· 4+ years of experience with pyspark data transformation (json,csv,rdbms,stream) pipeline design , development and deployment with kubernate/onprem platform (not cloud based).

· 2+ years of experience in designing and implementing data workflows with Apache Airflow.

· Kafka or equivalent Cloud Bigdata components (specific to the Data Engineering role)

· Exposure in Oracle, MySql, SQL Server, DB2, Teradata, SPARK SQL, POSTGRES SQL, Spark SQL

· Unix/Shell Scripting experience.

· Cloud technologies GCP preferable.

Additional Requirements:

· Exposure to Large enterprise data

· Experience in application support and maintenance of spark applications

· Experience in optimize and tune the performance to handle large and medium scale data volume with spark.

· Experience in performance tuning techniques for large-scale data processing.

· Experience working with Continuous Integration/Continuous Deployment tools

· Experience working on project(s) involving the implementation of solutions applying development life cycles (SDLC)

· Adherence to clean coding principles: Candidates should be capable of producing code that is devoid of bugs and can be easily understood and replicated by other developers.

· Strong teamwork abilities: developers typically collaborate closely with data scientists and other backend developers.

Therefore, candidates should exhibit excellent communication and collaboration skills.