SUMMARY:
The Data Engineer will be a part of our Data team, performing Data Architecture Designs and ETL developments in On Prem and Cloud Data Hub environments. Looking for highly self-motivated person who has customer service orientated, team oriented and collaborative attitude. Candidate should be logical and efficient, with keen attention to detail. Should be able to effectively prioritize and execute tasks while under pressure as well as adapt to learn new technologies as required.
THIS IS FULLY ONSITE IN OUR KONDAPUR, HYDERABAD OFFICE
Responsibilities:
- Build and maintain real-time and batch data pipelines across the advanced analytics platform.
- Design, develop and orchestrate highly robust and scalable ETL pipelines.
- Design and implement Dimensional and NoSQL data modelling as per the business requirements.
- Develop highly optimal codebase and perform Spark optimizations for Big Data use cases.
- Design, develop and deploy optimal monitoring and testing strategy for the data products.
- Collaborate with stakeholders and advanced analytics business partners to understand business needs and translate requirements into scalable data engineering solutions.
- Collaborate with data scientists to prepare data for model development and production.
- Collaborate with data visualization and reporting application developers to ensure the sustainability of production applications and reports.
- Collaborate with data architects on the enhancement of Chamberlain's enterprise data architecture and platforms.
- Comply with health and safety guidelines and rules.
- Maintain professional and technical knowledge by attending educational workshops, professional publications, establishing personal networks, and participating in professional societies.
Qualifications:
- Relevant degree preferred.
- 4-5 years or more years of relevant experience as a Data Engineer or Data Architect experience with distributed data systems, data mapping, and building ETL (Extract, Transform, and Load) and ELT (Extract, Load and Transform) pipelines.
- 2 or more years of relevant experience with Databricks is a MUST
- Certification in Databricks will be an advantage
- Experience with Azure Public Cloud, ADF, AWS technologies will be very helpful
- Programming experience with Scala or Python is a plus.
- Hands-on architecture and design of all data solutions including modeling and mapping required.
- Experience in data modeling and building and modifying in a relational database environment required.
