- We are looking for a savvy Data Engineer expert to join our analytics team
- The Candidate will be responsible for expanding and optimizing our data and data pipelines, as we'll as optimizing data flow and collection for cross functional teams
- The ideal candidate has python development experience and is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up
- We believe that candidate with solid Software Engineering/Development is a great fit
- Howe'ver, we also recognize that each candidate has a unique blend of skills
- The Data Engineer will work with database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery is consistent throughout ongoing projects
- They must be self-directed and comfortable supporting the data needs of multiple teams
- The right candidate will be excited by the prospect of optimizing data to support our next generation of products and data initiatives
Responsibilities for Data Engineer
- Create and maintain optimal data pipelines,
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing for greater scalability, etc
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
- Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Performing ad hoc analysis and report QA testing.
- Follow Agile/SCRUM development methodologies within Analytics projects.
- Working SQL knowledge and experience working with relational databases, query authoring (SQL) as we'll as working familiarity with a variety of databases.
- Experience building and optimizing big data data pipelines, and data sets.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with unstructured datasets.
- Good communication skills, a great team player and someone who has the hunger to learn newer ways of problem solving.
- Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
- A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
- Working knowledge on Unix or Shell scripting
- Constructing methods to test user acceptance and usage of data.
- Knowledge of predictive analytics tools and problem solving using statistical methods is a plus.
- Experience supporting and working with cross-functional teams in a dynamic environment.
- Demonstrated understanding of the Software Development Life Cycle
- Ability to work independently and with a team in a diverse, fast paced, and collaborative environment
- Excellent written and verbal communication skills
- A quick learner with the ability to handle development tasks with minimum or no supervision
- Ability to multitask
- We are looking for a candidate with 7+ years of experience in a Data Engineering role.
They should also have experience using the following software/tools:
- Experience in Python, Java, etc
- Experience with Google Cloud Platform.
- Experience with bigdata frameworks & tools - Apache Hadoop/Beam/Spark/Kafka.
- Exposure to workflow management & scheduling using Airflow/Prefect/Dagster
- Exposure to databases like (Big Query , Clickhouse).
- Experience to c ontainer orchestration ( Kubernetes )
- Optional Experience on one or more BI tools (Tableau, Splunk or equivalent).
Minimum Qualifications:
4+ years of IT-related work experience with a Bachelors degree in Computer Engineering, Computer Science, Information Systems or a related field.
OR
6+ years of IT-related work experience without a bachelors degree.
2+ years of work experience with programming (eg, Java, Python).
1+ year of work experience with SQL or NoSQL Databases.
1+ year of work experience with Data Structures and algorithms.
