Technical Data Engineering Manager

Early Applicant

Exp: 5-7 Years

Job Description

Team Leadership

Lead, mentor, and manage a team of data engineers.
Foster a collaborative and innovative team culture.
Provide guidance on data engineering best practices, methodologies, and technologies.

Data Architecture

Design and implement scalable, efficient, and robust data architectures.
Collaborate with other teams to understand data requirements and ensure alignment with business goals.
Evaluate and recommend data storage solutions, both on-premises and cloud-based.

ETL (Extract, Transform, Load) Processes

Oversee the development and maintenance of ETL processes for efficient data movement.
Ensure data quality and integrity during the ETL processes.
Optimize ETL workflows for performance and scalability.

Data Migration

Plan and execute data migration projects, ensuring minimal downtime and data integrity.
Develop strategies for seamless transition between different data storage systems.
Collaborate with relevant teams to address data mapping, transformation, and validation.

Data Warehousing

Manage the design and implementation of data warehouses.
Ensure the integration of various data sources into the data warehouse.
Collaborate with business intelligence teams to support reporting and analytics needs.

Data Integration

Implement data integration solutions to facilitate seamless data flow between different systems.
Work with API integration and real-time data processing technologies.
Ensure data consistency and accuracy across integrated systems.

Performance Monitoring And Optimization

Monitor and optimize data processing and storage systems for performance.
Implement best practices for data partitioning, indexing, and query optimization.
Troubleshoot and resolve performance issues in collaboration with IT teams.

Data Governance And Security

Implement and enforce data governance policies and standards.
Collaborate with security teams to ensure data privacy and compliance with regulations.
Establish and maintain access controls for data repositories.

Technical Skills

Proficient in SQL for database management and querying.
Expertise in using PySpark/Python for data engineering tasks and automation.
Have strong experience in Data Engineering and Data Migration with Large data volumes.
Familiarity with version control systems and collaborative coding practices.

Qualifications

Bachelor's or Master's degree in a related field (e.g., Computer Science, Information Systems).
Proven experience in data engineering, with a focus on technical leadership roles.
Strong proficiency in data engineering tools and technologies (atleast two in: Snowflake, Databricks, Apache Spark, Hadoop, Kafka, Flink).
Knowledge of cloud-based data solutions (AWS).
Familiarity with data governance, privacy, and security principles.
Excellent communication and interpersonal skills.