This role is for one of the Weekday's clients
Key Responsibilities
- Design and construct robust, scalable data pipelines using Python to support ongoing analytics and data processing needs.
- Manage and optimize data retrieval and storage operations with PostgreSQL.
- Develop, maintain, and scale our data processing workflows using Apache Airflow.
- Collaborate with data scientists and analysts to gather requirements and translate them into data engineering solutions.
- Implement complex big data projects with a focus on collecting, parsing, managing, analyzing, and visualizing large data sets.
- Ensure data integrity and system reliability by implementing automated tests and monitoring data quality.
- Optimize data flow and collection to improve data accuracy and utility.
- Provide leadership and guidance to junior data engineers, including code reviews and mentorship on best practices in data engineering.
- Stay abreast of industry trends and innovations in data engineering, continuously improving our systems and processes.
Required Skills And Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Minimum of 8 years of experience in a Data Engineering role.
- Expert proficiency in Python programming for data applications.
- Extensive experience with PostgreSQL for complex database management and optimizations.
- Proficient in designing and implementing automated workflows and data processes using Apache Airflow.
- Strong foundation in SQL, data modeling, and ETL processes.
- Experience with cloud services (AWS, Azure, Google Cloud) is preferred.
- Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
Skills: python,airflow,postgresql
