We are seeking a talented Data Engineer to join our team
The ideal candidate will have a strong background in designing, building, and optimizing data pipelines and infrastructure to support our data analytics and machine learning initiatives
Responsibilities
Design, develop, and maintain scalable and efficient data pipelines to process large volumes of structured and unstructured data from various sources
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical solutions
Implement data integration and ETL processes to extract, transform, and load data into data warehouses, data lakes, and other storage systems
Optimize data pipeline performance for speed, reliability, and cost-effectiveness, leveraging technologies such as Apache Spark, Apache Flink, and AWS Glue
Ensure data quality and integrity throughout the data lifecycle, implementing data validation and monitoring processes
Design and maintain data models, schemas, and metadata to support data analysis and reporting requirements
Stay up to date with emerging technologies and best practices in data engineering, recommending, and implementing innovative solutions
Collaborate with cross-functional teams to drive data-driven decision-making and deliver actionable insights
Qualifications
BE, B-Tech, BCA, MCA, M-Tech, or Bachelors degree in Computer Science, Information Technology, or related field
Minimum 5 years experience as a Data Engineer or similar role, with a focus on building and optimizing data pipelines and infrastructure
Strong programming skills in languages such as Python, Scala, or Java
Experience with big data technologies such as Hadoop, Spark, Kafka, and HDFS
Proficiency in SQL and relational databases (eg, PostgreSQL, MySQL) as well as NoSQL databases (eg, MongoDB, Cassandra)
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform
Excellent problem-solving skills and attention to detail
Strong communication and collaboration skills, with the ability to work effectively in a team environment
Experience with data visualization tools (eg, Tableau, Power BI) is a plus
Knowledge of machine learning concepts and techniques is a plus
