Data Specialist

Hi,

Data Engineering Lead -- Hyderabad -- Full Time

Whatyou will do

Create ML-Ops frameworks for model development, training, deployment, and monitoring.

Optimize Databricks clusters for performance, scalability, and resource efficiency.

Monitor cluster health, performance metrics, and resource usage, adjusting cluster sizes as needed.

Design efficient data processing systems and pipelines using Databricks, APIs, and other cloud services.

Act as the Databricks expert and uphold platform standards.

Collaborate with Data and Platform architects on defining the product roadmap.

Build and maintain scalable infrastructure on cloud platforms such as AWS, Azure, or Google Cloud to support data processing and analytics workloads.

Implement data governance best practices to ensure data quality, reliability, and security throughout the data lifecycle.

Monitor and troubleshoot data pipelines to identify and resolve performance bottlenecks, data integrity issues, and other operational challenges.

What you'll bring

Bachelor's degree in computer science or related field.

7+ years of IT experience, including 5+ years in Databricks development and administration, Spark, and cloud platforms (preferably Azure).

Proficiency in Databricks admin and development, with expertise in Resource and Performance Management.

Expertise in test automation, continuous integration, Docker, GitHub, Jenkins, and Unix/Linux.

Leadership in large project initiatives, willingness to mentor, and support for team members from different time zones.

You will have advantage if you bring -

Experience with Kafka and Spark, and strong in Python and Pyspark.

Hands-on experience in machine learning pipelines and scripting languages.

Familiarity with Java, Python, SQL/NoSQL databases, and Big Data tools.

Passion for new tech, continuous improvement, and collaborative problem-solving.Proactive, strong work ethic, and team playe