Job brief
We are looking for a Lead Data Analyst with experience in building high-performing, scalable, enterprise-grade applications.
You will be part of a talented software team that works on mission-critical applications. You will be responsible for application development while providing expertise in the full software development lifecycle, from concept and design to testing.
Job Description:
- As a key contributor to the data engineering team, the candidate is expected to:
- Evaluate innovative technologies and tools while establishing standard design patterns and best practices for the team.
- Build and deploy AWS cluster and tuning the environment.
- Build and deploy modular data pipeline components such as Apache Airflow DAGs, AWS Glue jobs, AWS Glue crawlers.
- Translate Business or Functional Requirements to actionable technical build specifications.
- Collaborate with other technology teams to extract, transform, and load data from a wide variety of data sources.
- Work closely with product teams to deliver data products in a collaborative and agile environment.
- Perform data analysis and onboarding activities as new data sources are added to the platform.
- Proficient in data modeling techniques and concepts to support data consumers in designing the most efficient method of storage and retrieval of data.
- Work in multiple project for AWS requirement building.
- Able to contribute in Change management process.
Skill Set :
- Experience in AWS Data processing, Analytics, and storage Services such as Simple Storage Service (s3), Glue, Athena, step Function, Lambda etc
- Experience in extracting and delivering data from various databases such as SnowFlake, Redshift, Postgres, RDS
- Coding experience with Python, SQL, yaml, spark programming (pyspark)
- Hands on experience with Apache Airflow as a pipeline orchestration tool or similar
- Experience in AWS Serverless services such as Fargate, SNS, SQS, Lambda etc
- Experience in Containerized Workloads and using cloud services such as AWS ECS, ECR and Fargate to scale and organize these workloads.
- Experience in data modeling and working with analytics teams to design efficient data structures, Clusters.
- Applied knowledge of working in agile, scrum, or DevOps environments and teams
- Applied knowledge of Versioning software like GitLab, SVN etc.
- Applied knowledge of modern software delivery methods like TDD, BDD, CI/CD
- Experience with development lifecycle (development, testing, documentation, and versioning)
