Azure Data Engineer

Role: Azure Data Engineer

Job Type: Long Term Contract (Permanent with Purview Services)

Location: Pune (Onsite)

Experience: 6+ Years

NOTE: Client is looking for Immediate joiners

Job Description:

Overview of the Demand:

Data Refinery is a software platform or eco-system that provides a mechanism to transforms large amounts of raw data into consumable, quality information that's ready for analytics. The Data Refinery is the link between gathering data and extracting value from it.To make the data consumable we provide a mechanism to transform the raw data into high quality Data Assets. The Data Assets are designed by domains to be re-usable across many use cases. For instance, we have Data Assets for, party, settlement, trade and payments. The quality of each Data Asset is assured by coordinating changes through a business and IT governance process. This provides the business with confidence that each Data Asset can be trusted for analytics. The Data Refinery also provides a mechanism to enrich information about relationships between Data Assets - including a Single View of a Customer. We use advanced matching technology to get a complete view of the customer's interactions and influences on their behavior.

As the Data Refinery is a software platform aiming for the as a service model - as such, the service/platform does not own the assets produced, any business transformations, any SLAs/OLAs on that data, or any of the data itself.

The Data Refinery also works on core capabilities and services for the upkeep and governance of said assets along with development of dashboards and other ideas.

Purpose of the job:

Full Stack Data Engineer role is expected to own and manage end to end activities within the team. Person would be entrusted to developed solutions/design ideas related to Data Engineering, service upkeep, developing new capabilities and identify design ideas to enable the software to meet the acceptance and success criteria. Work with architects/BA to build data components on the Big data environment.

Responsible for:

As a key member of the technical team, you will be expected to define and contribute at a high-level to many aspects of our collaborative agile development process:

Must Haves:

• Hands on experience or understanding on DevOps related to Jenkins CI/CD.
• Hands on experience in building microservices (Java/SpringBoot stack)
• Expertise in software development
• Python and applications/services built with python
• Promoting development standards, code reviews, mentoring, knowledge sharing

Good to have skills

• Data Engineering knowldege(spark, scala, Hadoop and scheduling services like apache airflow).
• Experience working with cloud applications/infrastructure

Major Challenges:

• Integrating with an established, complex Hadoop based project adhering to tight deadlines
• Working with globally dispersed and diversified teams.
• Supporting specific use-cases and onboarding new use-cases