We are looking for an experienced and creative Data Engineer to join our dynamic data and analytics team.
In this position, you will require hands-on expertise in data engineering and the AWS Tech Stack. You should also be able to provide direction and guidance to developers, oversee the development and unit testing, as well as document the developed solution. Building strong customer relationships for ongoing business is also a key aspect of this role. To succeed in this position, you should have experience with Cloud-based Data Solution Architectures, the Software Development Life Cycle (including both Agile and waterfall methodologies), Data Engineering and ETL tools/platforms, and data modeling practices.
KEY RESPONSIBILITIES
Build scalable end-to-end data pipelines to integrate and model datasets from different sources that meet functional and non-functional requirements.
Manage the technical scope and architecture of the project before, during, and after delivery.
You will be responsible for working with data engineering teams to deliver cutting-edge data products on the cloud for our customers.
Work with business and functional stakeholders to understand data requirements and downstream analytics needs.
Partner with multiple areas of business to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements.
Responsible for ratifying technology solutions, producing concise Design documents, contributing to work estimates
Translate business requirements & E2E designs into technical implementations based on system capabilities.
Define, and promote re-usable, extendible, scalable, and maintainable solutions considering trade-off for cost vs benefit
Communicate at all levels clearly and credibly about the importance of solution design.
Foster a data-driven culture throughout the team and lead data engineering projects that will have an impact throughout the organization.
Perform technical walk-throughs to ensure effective communication of system architecture.
Work with data and analytics experts to strive for greater functionality in our data systems and products; and help to grow our data team with exceptional engineers.
REQUIRED EXPERIENCE, SKILLS & QUALIFICATIONS
Around 8-15 years of relevant experience working with High-Performance Data Products or Data Systems as a Data Engineer.
Advanced level proficiency in Designing and Developing Data Products (using i.e. PySpark, Spark SQL, Scala, etc.), orchestration tools/services (i.e. Airflow, etc.)
Proficient with Software Engineering best practices, such as unit testing and integration testing, and software development tools, such as IntelliJ, Maven, Git, and Docker among others.
Extensive experience in at least one cloud platform (AWS preferred) with Big Data and AI/ML services (ECS, EMR, Bedrock, Sagemaker, Quicksight, Lake Formation, etc.)
Advanced knowledge of Apache Spark, Kafka, or equivalent streaming/batch processing and event-based messaging.
Strong Data Analysis skills and ability to slice and dice the data as needed for stakeholders reporting.
Relevant experience in databases (columnar, NoSQL, and MPP databases: Redshift, Dynamodb, Aurora, Postgres, and/or Snowflake).
Should be aware of Security compliances and design practices.
Exceptional interpersonal, analytical, and communication skills including the ability to explain and discuss DevOps concepts with colleagues and teams.
Expertise in test management and defect tracking tools like HP Quality Center, and JIRA.
Fully adhere to and evangelize an entire CI/CD pipeline.
Idea on API development and use of JSON/XML as data formats.
Proficiency in Designing and Developing Data Products and leading a team of Data Engineers to drive end-to-end execution
DESIRED EXPERIENCE, SKILLS & QUALIFICATIONS
Experience in the Healthcare Laboratory (IVD) domain is a plus.
Experience with security and privacy regulations (GDPR, HIPAA, etc.)
Demonstrated ability to collaborate effectively with cross-functional teams in a fast-paced and dynamic environment.
Proven track record of conducting root cause analyses on both internal and external data and processes to address specific business inquiries and identify areas for enhancement.
EDUCATION
Master s degree/Bachelor s degree in Computer Science or related.
