Role Summary & Role Description
As a senior data engineer, you will play a crucial role in our data infrastructure and analytics initiatives. You will contribute to the design, development, enhancement & maintenance of data pipelines, ensuring data integrity, scalability and security. This is an opportunity to work within a very strong development team and help to develop new challenging features in the Collateral management using cutting edge technologies.
Hiring for a skilled data architect with 5-8 years of hands-on ETL experience using distributed computing frameworks as Apache Spark, Hadoop, Hive. Additionally, cloud experience particularly with Azure platform is highly desirable.
- Collaborate with stakeholders to design, build and maintain robust and scalable data pipelines to efficiently collect, transform and load high volume of data from various sources using data ingestion tools
- Develop and optimize data models & schema for analytical and operational use cases
- Establish data quality standards & best practices, implement data governance process and ensure data integrity, security and compliance
- Create visual representations of data to help business and organization understand complex information
- Work closely with infrastructure and DevOps teams to ensure the reliability, performance and scalability of our data infrastructure
- Build data integrations between on-prem/cloud-based systems
- Evaluate new technologies, frameworks and tools in the data engineering space to drive innovation and improve data processing capabilities
Core/Must Have Skills
- Extensive experience in data engineering, with a deep understanding of data processing frameworks and solid experience with ETL development using distributed computing frameworks as Apache Spark, Hadoop, Hive
- Good hands-on experience in Scala/Python with strong experience in Spark scala/PySpark and Spark SQL for data processing
- Hands-on experience in SQL, PL/SQL, Unix Shell Scripting, Autosys/Airflow
- Experience with relational databases such as Oracle, PostgreSQL as well as NoSQL databases
- Data integration techniques and tools including APIs and data streaming
- Strong analytical and problem solving skills, with the ability to identify and resolve issues in data pipelines and systems
- Experience with Agile software development methodology and version control systems like GitHub
- Expertise in Data Modeling and DB design with the skills in performance tuning
- Support existing data platforms and products in production for any critical issues and provide resolution fixes
- Have good communication skill, and collaborate well with different teams/stakeholders and vendors
Good To Have Skills
- Knowledge of Azure based data storage & processing services, DevOps would be a Plus
- Knowledge in Spark based data engineering solutions using Databricks and Unity Catalog
- Experience with containerization and orchestration technologies such as Docker and Kubernetes is a Plus
- Excellent work ethics with ability to work independently
- Experience in Banking Domain preferred
Work Schedule: Hybrid
Why This Role Is Important To Us
Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our business to deliver data and insights to our clients. Were driving the companys digital transformation and expanding business capabilities using industry best practices and advanced technologies such as cloud, artificial intelligence and robotics process automation.
We offer a collaborative environment where technology skills and innovation are valued in a global organization. Were looking for top technical talent to join our team and deliver creative technology solutions that help us become an end-to-end, next-generation financial services company.
Join us if you want to grow your technical skills, solve real problems and make your mark on our industry.
About State Street
What we do. State Street is one of the largest custodian banks, asset managers and asset intelligence companies in the world. From technology to product innovation, were making our mark on the financial services industry. For more than two centuries, weve been helping our clients safeguard and steward the investments of millions of people. We provide investment servicing, data & analytics, investment research & trading and investment management to institutional clients.
Work, Live and Grow. We make all efforts to create a great work environment. Our benefits packages are competitive and comprehensive. Details vary by location, but you may expect generous medical care, insurance and savings plans, among other perks. Youll have access to flexible Work Programs to help you match your needs. And our wealth of development programs and educational support will help you reach your full potential.
Inclusion, Diversity and Social Responsibility. We truly believe our employees diverse backgrounds, experiences and perspectives are a powerful contributor to creating an inclusive environment where everyone can thrive and reach their maximum potential while adding value to both our organization and our clients. We warmly welcome candidates of diverse origin, background, ability, age, sexual orientation, gender identity and personality. Another fundamental value at State Street is active engagement with our communities around the world, both as a partner and a leader. You will have tools to help balance your professional and personal life, paid volunteer days, matching gift programs and access to employee networks that help you stay connected to what matters to you.
State Street is an equal opportunity and affirmative action employer.
Discover more at StateStreet.com/careers
Job ID: R-749725
