Data Engineer
Experience Level:Senior
Position Overview:
This role will be responsible for expanding our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler, who enjoys optimizing data systems and building them from the ground up. This position will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
Roles and Responsibilities:
- Create and maintain efficient and scalable data models as per business needs
- Create and maintain optimal data pipelines against multiple data sources lie SQL, BigData on AWS cloud;
- Assemble and process large, complex data sets to meet both functional and non-functional business requirements;
- Analyze and improve existing data models, pipelines, related infrastructure and processes
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics;
- Monitor and improve data quality and data governance policies
- Collaborate with stakeholders including the executive, product, data and design teams to assist with data-related technical issues and support their data infrastructure needs;
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader;
- Work with data and analytics experts to strive for greater functionality in our data systems.
Must Have Skills:
- 8+ years of experience working with distributed data technologies (e.g. Hadoop, MapReduce, Spark, Kafka, Flink etc) for building efficient, large-scale big data pipelines;
- Strong Software Engineering experience with proficiency in at least one of the following programming languages: Java, Scala, Python or equivalent;
- Implement data ingestion pipelines both real time and batch using best practices;
- Experience with building stream-processing applications using Apache Flink, Kafka Streams or others;
- Experience with Cloud Computing platforms like Amazon AWS, Google Cloud etc.;
- Experience supporting and working with cross-functional teams in a dynamic environment;
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with ELK stack.
- Ability to work in a Linux environment.
Good to Have:
- Experience in building distributed, high-volume data services;
- Experience with big data processing and analytics stack in AWS: EMR, S3, EC2, Athena, Kinesis, Lambda, Quicksight etc.;
- Knowledge of data science tools and their integration with data lakes;
- Experience in container technologies like Docker/Kubernetes.
Qualification:
- Bachelor of Science in Computer Science or equivalent technical training and professional work experience.
Location:
About NomiSo:
NomiSo is a Product Engineering company currently focussed on Video Stream Engineering, backed by AI and ML. We are a team of Software Engineers, Architects and Cloud Experts with more than 100 years of combined expertise in Technology and Delivery Management.
Our mission is to Empower and Enhance the lives of our customers, through simple solutions for their complex business problems.
At NomiSo we encourage entrepreneurial spirit - to learn, grow and improve. A great workplace, thrives on ideas and opportunities. That is a part of our DNA. We're in pursuit of colleagues who share similar passions, are nimble and thrive when challenged. We offer a positive, stimulating and fun environment with opportunities to grow, a fast-paced approach to innovation, and a place where your views are valued and encouraged.
We invite you to push your boundaries and join us in fulfilling your career aspirations!
We are an equal opportunity employer and are committed to diversity, equity, and inclusion. We do not discriminate on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability status, or any other protected characteristics.
