Software Engineer - Data Engineering

Early Applicant

Exp: 1-3 Years

Job Description

You will design and implement robust, scalable, and high-performance data pipelines using Spark, Scala, and Airflow with familiarity on Google Cloud.
You develop, construct, test, and maintain architectures such as databases and large-scale processing systems.
You will assemble large, complex data sets that meet functional and non-functional business requirements.
You build the infrastructure required for optimal extraction, transformation, and loading (ETL) of data from various data sources.
You will collaborate with data scientists and other stakeholders to improve data models that drive business processes.
You implement data flow and tracking using Apache Airflow.
You ensure data quality and integrity across various data processing stages.
You monitor and optimize the performance of the data processing pipeline.
You will tune and optimize Spark jobs for performance and efficiency, ensuring they run effectively on large-scale data sets.
You troubleshoot and resolve issues related to data pipelines and infrastructure.
You stay up-to-date with the latest technologies and best practices in Big Data and data engineering.
You adhere to Zeotap s company, privacy and information security policies and procedures
You complete all the awareness trainings assigned on time

Requirements:

1+ years of experience in building and deploying high scale solutions
Must have very good problem-solving skills and clear fundamentals of DS and algorithms
Expert coding skills in Java or Scala
Expert coding skills in Go or Python is a huge plus
Apache Spark or other Bigdata stack experience is a mandatory
High level and low-level design skills.
Deep understanding of any OLTP, OLAP, NoSQL or Graph databases is a huge plus.
Deep knowledge of distributed systems and design is a huge plus
Hands-on with Streaming technologies like Kafka, Flink, Samza etc is a huge plus
Good knowledge of scalable technologies involving Bigdata technologies
Bachelor or Master s degree in information systems, computer science or other related fields is preferred