Egen.ai - Senior GCP Data Engineer
Egen.ai, formed through the merger of SpringML and Egen Solutions, leverages cloud, data, AI, and platform engineering to empower organizations on their digital transformation journeys. The company specializes in combining platform engineering expertise with data, AI, analytics, and cloud capabilities to deliver impactful solutions for clients.
Role Description: Senior GCP Data Engineer
- Location: Hyderabad, India
- Employment Type: Full-time, On-site (Hybrid)
- Experience Required: 4 - 6 years
- Notice Period: 30 days or less than 30 days
The Senior GCP Data Engineer will play a key role in developing and managing data engineering processes, including data modeling, ETL workflows, data warehousing, and data analytics, to support the organization's technology initiatives.
Key Responsibilities:
- Design, develop, and maintain data pipelines using Python.
- Build and optimize data pipelines using Apache Beam SDK and PySpark.
- Develop CI/CD pipelines tailored for data engineering processes.
- Leverage Google Cloud Platform services, including GCS, BigQuery, Dataflow, Cloud Functions, IAM, and Cloud Composer DAG.
- Work with SQL databases (e.g., Oracle, PostgreSQL, SQL Server) and NoSQL databases (e.g., MongoDB, Elasticsearch, Firestore, DynamoDB).
- Execute data migration and data processing technologies to ensure seamless data integration and management.
Preferred Skills:
- Experience with containerization and orchestration tools such as Docker and Kubernetes.
Qualifications:
- Strong experience in data engineering, data modeling, and ETL processes.
- Proficiency in data warehousing and data analytics.
- In-depth knowledge of Google Cloud Platform (GCP) technologies.
- Solid understanding of data processing and transformation tools.
- Experience working with both relational and non-relational databases.
- Excellent problem-solving and analytical skills.
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
