Data Engineer Specialist

Data Engineer Specialist:

Location - Vikhroli, Mumbai

Relevant Exp - 3-4 years

Job Description:

Knowledge and skills (general, technical and soft skills)

• Proven track record of working in the designing of Data Engineering or analytical solutions, preferably for the Manufacturing / Energy / Oil & Gas Industry.
• Preferable experience in this technology stack & languages: Google BigQuery & Pub/Sub, DataBricks, Microsoft Data Factory, Microsoft SQL, Microsoft Data Lake, Microsoft PowerBI, RapidMiner, SAS, VS Code, Docker, Kubernetes, Apache Airflow, Java, Python, SQL.
• Knowledge and experience in applying technology service management, agile, lean, and DevOps principles and practices for solution design and implementation. Strong capacity for planning.
• Strong communication, presentation, and interpersonal skills, with the ability to communicate effectively with culturally diverse audiences and stakeholders.
• Attitude: mental outlook, perspective, and disposition towards chasing the goals and helping company grow in the data layer, self-directed, business value-oriented, with a passion for ideation, innovation, and customer-centricity.
• Team player.
• Critical thinking and aptitude to challenge status quo.

Responsibilities:

Design, develop & implement the company's Data Layer: Design & build components of the data ecosystem both on OT-Edge & Cloud, generating subsequently the Operational documentation. Those main components of the ecosystem are:

ETL/ELT processes & data pipelines.

Data Storage Solutions (for diverse technologies/architectures as Databases, Data Lake, Data Lake House and modern Data Warehouse).

Data Processing Engines, those running all the pre-processing layer before the analytical layer.

Data processes Orchestration & Workflow Management

Modern messaging systems/architectures to support more efficient data exchange where event driven architectures are required.

Orchestration and monitoring systems to measure and control data processes.

Enable analytical & visualization/consumption layer through data transportation, data processing, data storing.

Ensure Data Quality and Reliability: Implement data quality checks, monitoring systems, and error handling mechanisms to ensure the accuracy, consistency, and reliability of data. Follow Industrial Resiliency and Effective Monitoring and visibility principles to ensure pipeline reliability.

Support Data Performance optimization: Support RUN team to optimize data storage, retrieval, and processing performance to meet performance and scalability requirements while reducing cost if possible.

Enable Data Accessibility: Facilitate data access for analytical layer by designing and implementing data APIs, interfaces, data catalogs ( including OT UNS-unified name space ), data microservices, and data access controls.

Collaborate with Cross-functional Teams: Work closely to support other domains in Implementation ( 1.Manufacturing, 2. Supply Chain & Sustainability, 3.Sales & Marketing 4. Finance & HR) being Data Engineering a support domain together with 1. Infrastructure, 2. Program & Project Management and 3. Technology Integration & Platforms )