Senior Data Engineer--PD

Job Description:

Position: Senior Data Engineer

Budget 17 lpa

Location Bangalore

Notice period Immediate joiner

As a Senior Data Engineer at, you will:

Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed business requirements

Develop systems that ingest, cleanse and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data

Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development

Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc

Unify, enrich, and analyze variety of data to derive insights and opportunities

Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities

Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate

Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform

Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org

Mentor other members in the team and organization and contribute to organizations growth.

What we are looking for:

6+ years work experience and bachelor's degree in Information Science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science.

Hands-on engineer who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security, etc.

3 or more years of active development experience as a data developer using Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc.

Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities

Build, test and enhance data curation pipelines integration data from wide variety of sources like DBMS, File systems, APIs and streaming systems for various KPIs and metrics development with high data quality and integrity

Maintain the health and monitoring of assigned data engineering capabilities that span analytic functions by triaging maintenance issues; ensure high availability of the platform; monitor workload demands; work with Infrastructure Engineering teams to maintain the data platform; serve as an SME of one or more application

Team player, Reliable, self-motivated, and self-disciplined individual capable of executing on multiple projects simultaneously within a fast-paced environment working with cross functional teams

3+ years of experience working with source code control systems and Continuous Integration/Continuous Deployment tools

Independent and able to manage, prioritize & lead workload

Technical Skills:

Azure, Python-Spark, Spark Streaming, Azure SQL Server, Cosmos DB, Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Data Pipelines, Cloud Computing, Relational Databases, Non-relational Databases, CI/CD.