Data Engineer

Key Responsibilities:

• Design and implement data processing pipelines using Hadoop and related technologies (HDFS, MapReduce, Hive, Pig, HBase, etc.).
• Develop and maintain data ingestion processes from various data sources into the Hadoop environment.
• Optimize and tune Hadoop clusters for performance, reliability, and scalability.
• Collaborate with data scientists and analysts to understand data requirements and provide solutions for data processing and analysis.
• Ensure data quality, integrity, and security within the Hadoop ecosystem.
• Monitor and troubleshoot Hadoop cluster performance and issues.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in Hadoop ecosystem technologies.
• Proficiency in Hadoop components such as HDFS, MapReduce, Hive, Pig, HBase, and Spark.
• Strong programming skills in Java, Python, or Scala.
• Experience with data warehousing and ETL processes.
• Knowledge of data modeling, data architecture, and data integration.
• Familiarity with cloud platforms (AWS, Azure, Google Cloud) is a plus.