Big Data Engineer

You will be responsible for developing Big Data Analytics workflows driving Diagnostic, Prescriptive and Predictive analytics. The automated workflows must be tuned for large-scale data processing in a performant and scalable manner.

The ideal candidate must be an independent problem solver with excellent communication skills. Your ability to stay up to date with the changes in the Big Data Analytics and Machine Learning technology landscape is critical for your success in the team.

This role involves collaborating with domain experts to understand analytics objectives and identifying which process will be most efficient in extracting the insight.

Candidate must know the underlying mathematical foundations of statistics, machine learning, and analytics. The ability to transform a data analytics objective into a mathematical process is must must-have skill for this role.

1. Proficiency in scripting languages such as Python.

2. Strong command of libraries such as Pandas, Dask, and PySpark.

3. Deep understanding of SQL, NoSQL, and relational database design, and proficiency in using SQL to extract and analyze data.

4. Experience with data pipeline and workflow management tools such as Airflow, Perfect.

5. Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.

6. The ideal candidate should be familiar with working with API servers, including Flask, Django, or FastAPI.

7. Candidates should have demonstrable experience in distributed computing and managing big data systems.

8. They must be self-motivated and stay updated with the latest trends and advancements in technology.

9. Understanding of event-driven architectures and real-time processing.

10. Knowledge of streaming technologies like Apache Kafka, Apache Flink, and Spark streaming