Senior ETL and Feature Engineer

Job Description: Role: Senior ETL and Feature Engineer Job Location: Bangalore Experience: 4-5 years Educational Qualification: Engineering & preferably from a premier institute About the Job: At PrivaSapien, you will have the opportunity to lead this era with Privacy Enhancing & Responsible AI Technologies. You will be an individual contributor working in setting up the big data ecosystem of the worlds first privacy red teaming and blue teaming platform. You will be working on cutting edge privacy platform requirements with customers across the globe and industry verticals. As part of being one of the early employees of the organization, you will be given a significant ESOP option and will be working with some of the brilliant minds from IISc and IIMs. Responsibilities: 1. Develop and maintain ETL pipelines to ingest and process large-scale 2. Develop a Python connector designed for ETL applications, enabling the collection of historical samples and the generation of intelligent samples for AI/ML tasks 3. Demonstrate hands-on experience on Apache Kafka, Amazon Kinesis, and datasets AWS Glue 4. ETL pipeline for AI/ML workload integrations and executions 5. Implement and maintain ETL pipeline and deep understanding of orchestration, scaling, and resource management. 6. Develop ETL samples to manage unstructured data tasks, covering data preprocessing for various types, including emails, Office 365 documents, pictures, voice recordings, videos, and PDFs. 7. Establish a solid understanding and practical knowledge of SQL databases, with a focus on query performance optimization and effective index management. 8. Implement an ETL pipeline with multiple databases to extract samples from both NoSQL and SQL 9. Work seamlessly in a multi-cloud environment, specifically AWS and Azure. 10. Execute hands-on development and deployment of microservices and containerized applications. Requirements: Glue 1. Minimum 4 years hands-on experience in setting up ETL & feature engineering part of the data pipelines on cloud or big data ecosystems. 2. Strong hands-on experience in Apache Kafka, Amazon Kinesis, and AWS 3. Holds expert-level knowledge in at least one ETL tool (ETL/ Workflow - SeaTunnel/ Dolphin Scheduler, SSIS, Informatica, Talend, etc.) 4. Experience with big data technologies such as Hadoop, Spark, and Hive 5. Experience in NoSQL and SQL databases (e.g., MongoDB, SnowFlake, PostgreSQL, DeltaLake, Parquet). 6. Strong programming skills in Python, and experience with data manipulation libraries such as pandas, numpy, etc. 7. Handling unstructured data tasks, including preprocessing for various data types such as Mail, Office 365, Pictures, Voice, Video, and PDF 8. Possesses SQL knowledge including query performance tuning, index maintenance and an understanding of database structure 9. Good to have knowledge in Apache Spark, Apache NiFi, Apache Kafka, Apache Airflow, Apache Beam, KubeFlow, Apache Beam and TensorFlow Transform 10. Expertise in networking, security, and cloud platforms (AWS, Azure, GCP, etc.) About PrivaSapien: Privacy Engineering and Responsible AI are going to be strategic in building guardrails for AI. PrivaSapien is a global pioneer building a spectrum of Privacy Engineering and Responsible AI products to govern data and AI the right way! PrivaSapien has won multiple awards including Niti Aayogs Aatmanirbhar Bharath award, DSCIs Innovation Box award, National Privacy Challenge, Maruti Suzukis MAIL program, NetApps Excellerator, part of NASSCOM AIs first Gen AI cohort, one of the first graduates from a middle east PET sandbox PrivaSapien has built a first of its kind revolutionary PERAI platform with six products Privacy X-Ray (Privacy Threat Modeling), Event Horizon (Expert grade anonymization), Data Twin (Synthetic Data), CryptoSphere (Cryptographic Pseudonymization), PrivaGPT (LLM Governance/ Responsible AI) and Differential Crystal (Privacy Preserved Insights). We have large enterprises as customers in India, Middle East, Europe and US. Our Vision & Work Culture: 1. Vision: PrivaSapiens vision is to innovate and drive the Evolution of the privacy era. PrivaSapien strongly believes that privacy is a fundamental right in the digital era. Without privacy, human beings thoughts, emotions, buying behavior and even voting can be manipulated with their own data and make humans AlgoSalves. Hence, PrivaSapien builds privacy engineering & Responsible AI products, platforms and services that make digital experiences safer for humanity, at the same time creating strategic advantage for businesses in the privacy era. 2. Passion: PrivaSapiens are a group of people who are extremely passionate about privacy and responsible AI, who are crazy enough to believe that we can protect privacy of every person across the world and build responsible AI, using our innovate products and services. 3. Integrity & Trust: At PrivaSapien, Integrity is a fundamental uncompromisable tenet. Integrity creates trust and trust is the foundation for any long-term relationship & ultimately success, especially in the area of privacy, where customers trust us with their data. 4. Customer first, Customer last: We are obsessed with customer value creation. Every action and decision at all levels will be based on the customers requirements and preference in mind. 5. Adaptability: As Charles Darwin stated, The species that survives is the one that is able best to adapt and adjust to the changing environment. In these interesting and disruptive times, more than your skills, your adaptability to emerging requirements is the key to success. 6. Innovation: We are an innovative Company at the core of everything we do. Each employee is empowered to be innovative in every aspect of their job and is expected to set the standard for the industry to follow. 7. Perseverance: PrivaSapien strongly believes that there is no substitute for hard work. Privacy is the right thing to do, but it is an uncharted territory. At PrivaSapien, we persevere to create and show the world the right path to privacy. But its not easy, so be prepared for lots of learning, challenges and burning a lot of midnight oil. 8. Experimentation: PrivaSapien encourages structured rapid experimentation which can accelerate learning, fail quickly, identify the best option under given constraints, guide each one of us in taking strategic and impactful decisions, provide competitive advantage and thus be a pioneer. 9. Collaboration: PrivaSapien is a close-knit family with shared vision and purpose. Its important that we jump-in along with our colleagues in solving issues and support each other at all times. United by the purpose, we will make this world a better and safer place, even in the face of seemingly impossible challenges. 10. Fun: Life is a journey, and every moment is precious. So, follow your heart and have fun while achieving your professional and personal dreams. We at PrivaSapien encourage a positive, transparent, non-hierarchal and fun filled working environment.