Business Toys Private Limited is looking for Data Platform Engineer to join our dynamic team and embark on a rewarding career journey
- A Data Platform Engineer is a specialized IT professional responsible for designing, building, and maintaining the data infrastructure and platforms that support data processing, storage, and analytics in an organization
- They play a crucial role in ensuring the scalability, reliability, and performance of the data infrastructure, enabling efficient data management and analysis
Responsibilities:
- Data Platform Design and Architecture: Collaborate with cross-functional teams, including data scientists, data engineers, and business stakeholders, to understand data requirements and design scalable data platforms and architectures that meet the organization's needs
- Data Integration: Develop data integration pipelines to ingest data from various sources (databases, APIs, streaming data, etc) into the data platform, ensuring data quality and consistency
- Data Storage and Management: Implement and manage data storage solutions such as databases (SQL and NoSQL), data lakes, data warehouses, and cloud-based storage systems to store and organize large volumes of structured and unstructured data
- Data Processing: Design and optimize data processing workflows and batch/streaming data pipelines to transform, cleanse, and prepare data for downstream analysis and reporting
- Data Security and Governance: Implement data security measures, access controls, and data governance policies to protect sensitive data and ensure compliance with relevant regulations (eg, GDPR, HIPAA)
- Performance Tuning and Optimization: Monitor and optimize the data platform's performance, identifying bottlenecks and implementing improvements to enhance system reliability and responsiveness
- Cloud Infrastructure Management: If the organization operates in a cloud environment, manage and optimize cloud-based data infrastructure, leveraging services provided by cloud providers (eg, AWS, Azure, GCP)
- Automation and DevOps: Employ automation and DevOps practices to streamline data platform operations, configuration management, and deployment processes
- Monitoring and Troubleshooting: Set up monitoring systems to proactively detect issues and promptly troubleshoot and resolve data platform-related incidents
- Documentation and Knowledge Sharing: Maintain comprehensive documentation of the data platform's design, configuration, and operational procedures
- Share knowledge and best practices with the team to foster a culture of continuous learning
- Requirements:Bachelor's degree in Computer Science, Information Technology, or a related field
- A Master's degree is a plus
- Proven experience as a Data Platform Engineer or a similar role, demonstrating a strong understanding of data infrastructure, data integration, and data management
- Proficiency in programming languages such as Python, Java, or Scala
