- The requirement is for a fast-paced and innovative cloud enablement team to support on-prem Cloudera migration to Google Cloud Platform using serverless orchestrations.
- The successful candidate will have a minimum of 6 years experience with the last 2 - 3 years of experience as a Data Engineer on the GCP platforms.
Mandatory Skills & Experience:
- Experience with BigQuery, ML Platform (Vertex AI or others), Dataflow and Dataproc.
- Expertise in architecting solutions for modern big data applications on-prem & cloud platforms.
- Experience with migrating on-prem Hadoop workload to cloud platforms.
- Proven experience in designing and building scalable infrastructure and platform to collect, process and analyse very large amounts of data (structured, un-structured and streaming real-time data).
- Knowledge in handling in-memory processing systems such as Apache Spark, Apache Beam aka Dataflow in GCP and BigQuery
- Expert level of skills with Streaming pipelines using PubSub & Dataflow or similar technologies.
Desired Skills & Experience:
- Experience with Dataflow, Apache Spark , BigQuery, PubSub and Kafka
- Experience with Document store, key-value pair and relational stores.
- Experience with SAFe/Spotify based Agile methodologies
