Senior Big Data Developer (Informatica, Hive, Spark, SQL)

Job Title : Senior Big Data Developer (Informatica, Hive, Spark, SQL)

Locations : Bengaluru - Thanissandra (BCIT)

Software Requirements: Informatica, Big Data Technologies (Hive, Spark), SQL and NoSQL databases, Oozie Scheduling, Linux OS and Shell Scripting & Agile, DevOps, CI/CD (GitHub, Jenkins)

Note: We are looking for candidates who can join immediately or have a maximum notice period of 15 days

Overall Responsibilities:

• Design and develop ETL jobs using Informatica Big Data Management.
• Manage and optimize data workflows using Oozie and ensure efficient data partitioning and storage within HDFS.
• Implement and troubleshoot complex SQL queries and NoSQL data storage solutions.
• Collaborate with cross-functional teams to integrate and manage distributed services.
• Analyze and address security requirements within data transfer processes, including establishing security controls.
• Adhere to industry best practices and organizational policies regarding data handling and integrity.
• Mentor junior team members and share best practices across teams.

Category-wise Technical Skills:

• Data Management:
• Expertise in Informatica for ETL processes and data integration.
• Proficient in querying and manipulating data using SQL and HQL.
• Familiarity with NoSQL databases.
• Big Data Technologies:
• Strong knowledge of Big Data frameworks such as Hive and Spark.
• Experience with Oozie scheduling and HDFS operations.
• DevOps and CI/CD:
• Hands-on experience with GitHub and Jenkins for continuous integration and deployment.
• Understanding of Agile methodology for software development.
• Security Management:
• Experience in designing security systems and implementing security protocols in data architectures.
• Knowledge of data sensitivity and compliance requirements.
• Operating Systems: Proficient in Linux OS configurations and shell scripting.

Experience:

• Minimum 4+ years of experience in development and design, specifically with Informatica Big Data Management.
• Demonstrated experience in managing distributed services and data processing environments.
• Background in the banking or financial sector is preferred.

Day-to-Day Activities:

• Develop and maintain ETL Pipelines for large datasets.
• Conduct regular code reviews and performance tuning of data processing jobs.
• Collaborate in daily stand-ups and sprint planning sessions to ensure team alignment.
• Engage with stakeholders to gather requirements and ensure that architectures support business goals.
• Participate in incident resolution and rotations to maintain system uptime and reliability.

Qualifications:

• Bachelor's degree in Computer Science or a related field.
• Postgraduate degree in related disciplines is a plus.
• Equivalent industry experience will be considered.

Soft Skills:

• Excellent interpersonal and communication skills for engaging with team members and stakeholders.
• Strong listening skills and the ability to articulate technical concepts clearly.
• Positive attitude towards work with a focus on team collaboration and trust.
• Enthusiastic team player who fosters a motivating and productive work environment.
• Ability to work independently and drive initiatives without supervision