Data Engineer

Data Pipeline Development:

Design, develop, and maintain scalable and robust data pipelines to ingest, process, and store large volumes of data from various sources.

Implement ETL (Extract, Transform, Load) processes to ensure data integrity and quality.

Data Architecture:

Collaborate with data architects and data scientists to create and optimize data models and databases that support analytical needs.

Ensure data is organized, accessible, and well-documented for both technical and non-technical stakeholders.

Collaboration:

Work closely with cross-functional teams, including analytics, product, and IT, to understand data requirements and deliver effective solutions.

Act as a liaison between data users and technical teams to facilitate data accessibility and usability.

Performance Optimization:

Automate, monitor and optimize data pipelines for performance, scalability, and reliability.

Troubleshoot and resolve data quality issues and system failures promptly.

Tool Implementation:

Stay updated on the latest data engineering tools, technologies, and best practices.

Evaluate and implement new tools that enhance data processing and analysis capabilities.

Documentation & Reporting:

Create and maintain comprehensive documentation for data workflows, processes, and systems.

Provide regular updates and reports on data engineering projects and performance metrics.

Qualifications:

Education:

Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.

Experience:

3+ years of experience in data engineering, data integration, or related roles.

Experience with cloud platforms (e.g., AWS, Azure, GCP) and data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).

Technical Skills:

Proficiency in programming languages such as Python, Java, or Scala.

Strong SQL skills and experience with databases (e.g., MySQL, PostgreSQL, MongoDB).

Familiarity with data pipeline frameworks (e.g., Apache Spark, Apache Kafka, Apache Airflow).