We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data processing and analytics. The ideal candidate has a strong background in distributed data processing, cloud platforms, and extensive hands-on experience with big data technologies.
Key Responsibilities:
- Data Pipeline Development: Design, develop, and optimize large-scale ETL (Extract, Transform, Load) processes and data pipelines.
- Big Data Management: Work with large datasets, implementing solutions to capture, process, and analyze data from various sources.
- Data Warehousing: Build and maintain scalable, reliable, and efficient data warehouses.
- Cloud & Distributed Systems: Implement data solutions using distributed computing tools such as Apache Hadoop, Apache Spark, and cloud platforms like AWS, Azure, or GCP.
- Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand data needs and translate them into technical requirements.
- Performance Tuning: Ensure performance, reliability, scalability, and security of big data platforms.
- Mentoring & Leadership: Mentor junior team members and contribute to the overall success of the team by driving technical excellence.
Skills and Qualifications:
- Technical Expertise:
- 7+ years of experience in big data development, including a deep understanding of data pipelines, architectures, and distributed systems.
- Proficiency with big data tools such as Apache Hadoop, Apache Spark, Kafka, Hive, HBase, and others.
- Strong experience with Python, Scala, Java, or SQL.
- Expertise with cloud platforms like AWS, Azure, or GCP for big data processing (experience with services like EMR, S3, Redshift, Databricks, etc.).
- Solid understanding of data modeling, ETL development, and data warehousing principles.
- Hands-on experience with NoSQL databases like Cassandra, MongoDB, or Couchbase.
- Experience with CI/CD pipelines and familiarity with containerization tools (e.g., Docker, Kubernetes) is a plus.
