Key Responsibilities:
- Create, construct, and manage data pipelines with AWS services including S3, Glue, EMR, and Redshift.
- Use Kinesis, Stream, and Firehose to implement batch and stream data loading.
- For infrastructure as code, use AWS CDK to ensure scalability and dependability.
- Improve the effectiveness and performance of SQL queries.
- Write Python and other programming language code and maintain it.
- Use Code Pipeline to implement CI/CD pipelines and GitLab to manage code repositories.
- Create and put into practice data models for data warehousing systems.
- For workflow management, make use of orchestration solutions such as Step Functions and Airflow.
- Manage the CI-CD, code, performance testing, test coverage, and scanning aspects of the DevOps activity.
Minimum Qualifications:
- Bachelor's degree in Information Systems, Computer Science or related field
- 7+ years of relevant overall experience
- Practical knowledge of AWS data services, including S3, Glue, EMR, and Redshift.
- Proficiency with AWS Kinesis Stream and Firehose for batch and data stream loading.
- Strong familiarity with AWS CDK.
- Proficiency with various programming languages, including Python.
- Expertise in query tweaking and SQL optimization
- Expertise in working with JSON, SML, CSV , YAML files.
- Familiarity with CI/CD workflows and GitLab.
- Knowledge of scrum and agile techniques.
- Hands-On experience with data warehousing and data modeling.
- Familiarity with orchestration tools such as Step Functions and Airflow.
- Practical knowledge of data visualization tools such as PowerBI and Tableau.
Preferred Qualifications:
- Familiarity with big data technologies such as Spark, Hadoop, etc.
