Data Engineer

Ford

Early Applicant

3 months ago
Be among the first 50 applicants

Exp: 2-5 Years

Full time

Chennai, India

Job Description

5+ years of complex SQL development experience
5+ years of Mainframe (TSO, JCL, DB2, Cobol, VSAM, IBM Debugger, File Manager, IMS, CA7, Endevor) Experience
2+ years of Cloud experience (GCP preferred) with solutions designed and implemented at production scale
Strong understanding of key GCP services, esspecially those related to data processing [Batch/Real Time] leveraging Terraform, BigQuery, BQSQL, Dataflow, DataFusion, DataProc, Cloud Build, Airflow & Pub/Sub
Experience in analysis, design, development, implementation using Data Warehousing applications e.g IBM Datastage/DataFlow/DataProc
Experience in integrating various data sources like Oracle, Teradata, DB2, Big Query & Flat files
Experience in designing jobs by using various Stages like Look-up, Filter, Sort, copy, remove duplicates, Join, Funnel and Aggregator.
Experience in debugging ETL jobs to check in for the errors and warnings associated with each Job run
Hands on experience in Tuning Jobs process identifying and resolve performance in Parallel Jobs
Involving in review meetings and coordinating with the team in job designing and fine-tuning the job performance.
Good experience in Autosys or Astronomer, Accurev & Github
Good experience in production support activities
BE/BTech in CS/IT or equivalent degree in relevant domain
Proven experience in SQL development and Data engineering [IBM Datastage preferred]
Strong knowledge of Mainframe JCLs, ETL processes, data integration and data management
Proficiency in programming languages such as SQL, Python or Java
Familiarity with cloud computing platforms and data storage solutions
Excellent problem solving skills and ability to work independently as well as team player
Professional Certification in GCP (e.g., Professional Data Engineer)
Experience in Mainframe to GCP migration considered plus
Agile Methodology Practitioner
Responsible for gathering business requirements and design Mainframe/ETL systems to meet application needs
Work closely with stakeholders to gather requirements and provide technical guidance on the migration process
Collaborate with the data engineering teams to understand the existing JCL/ETL jobs and their dependencies.
Design and implement migration strategies for moving ETL to GCP native services.
Develop, test, and deploy data pipelines and workflows, ensuring scalability, reliability, and performance
Troubleshoot and resolve any production issues or challenges that arise during the migration
Deliver product features as part of roadmap and business needs
Lead in Effort estimation for the new developments and enhancements.
Co-create test plan and lead execution and code deploymentLead performance tuning to optimize the job execution.
Responsible for validating, run, scheduling and monitoring jobs using data stage director
Responsible for debugging Ticketing issues in the production phase