Senior Cloud Data Engineer – AWS

Senior Cloud Data Engineer AWS Key Result Areas and Activities: Share and Build Expertise- Develop and share expertise in AWS and Python and actively mine the experience and expertise in the organization for sharing across teams and clients in the firm. Support the Cloud and Data Engineering COE initiatives Data Pipeline Development and Optimization: Design, develop, and optimize scalable ETL/ELT pipelines using AWS services such as Glue, Lambda, and Step Functions to manage large-scale data processing. Data Storage and Management: Implement and manage data storage solutions like S3, Redshift, and RDS, ensuring optimal performance, cost efficiency, and compliance with data governance standards. Cloud Infrastructure and Security Management: Architect and manage cloud infrastructure using AWS services such as CloudFormation, EC2, and IAM, ensuring security, reliability, and scalability of data systems. Monitoring, Maintenance, and Troubleshooting: Set up monitoring and alerting systems using CloudWatch, AWS Config, and other tools to ensure system health, troubleshoot issues, and maintain high availability. Collaboration and Stakeholder Communication: Collaborate with cross-functional teams, including data analysts and business stakeholders, to understand requirements, provide technical solutions, and ensure data architecture aligns with business goals. Must have: Expertise in AWS services like S3, EC2, EMR, Lambda, Glue, and Redshift. Strong knowledge of cloud databases such as Aurora, Snowflake, and Singlestore. Proficient in big data tools like Hadoop, Hive, Spark, and programming languages such as Scala, Python, and PySpark. Advanced SQL skills with experience in developing complex ETL workflows, including SCD, deduplication, and aggregation. Hands-on experience with orchestration tools like Apache Airflow. Thorough understanding of AWS VPC, subnets, security groups, and route tables. Qualifications: Minimum 7+ years of IT experience with 3+ years of AWS related projects. Bachelor's degree in computer science, engineering, or related field (master's degree is a plus