Partner with Data Engineers, Data architects, domain experts, data analysts and other teams to build foundational data sets that are trusted, well understood and aligned with business strategy
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Lead a team of Data Engineers with work load management and mentor the team on technical challenges
Problem solver with excellent interpersonal skills with ability to make sound complex decisions in a fast-paced, technical environment
Support to build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and GCP big data technologies
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Create data engineering tools for faster data transformation and data lake loading
Identify the challenges faced in DLL and propose & Implement solution
Work with stakeholders including the Product Owner, Product Leader, Chapter Leader, Scrum Master, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
Skills/Tools/Techniques
Hands on proficiency with big data tools: Hadoop, Hive, Spark & Scala
Experience in GCP cloud services: Google cloud storage, Big query, Spanner, Cloud Pub/Sub, stackdriver & Composer
Experience in CI/CD using GitlabEE
Experience with data pipeline and workflow management tools: Airflow, etc.
