JD for the role Data Engineer
Job Description:
We are seeking an experienced Data Engineer to join our AI search team. As a Data Engineer at ThoughtSpot, you will be responsible for designing, building, and maintaining the data infrastructure that powers our analytics platform and underlying Natural Language Search engine .
Requirements: 4+ years of experience
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases/warehouses
- Experience building and maintaining large data pipelines, data infrastructure, (Kubernetes preferred) and Machine learning infrastructure
- Working knowledge of message queuing, stream processing, and highly scalable data stores
- Experience working with data tools: Spark, Kafka, etc.
- Knowledge of data warehousing concepts and experience with databases like Snowflake, Redshift with experience with data model design, and optimization for data warehousing
- Experience with data pipeline and workflow management tools like Dagster or Airflow with code and no-code ETL implementations
- Experience with cloud services such as AWS, Azure etc
- Proficiency in programming languages such as Python and data processing libraries such as Pandas and Numpy. Experience with Java is good to have.
- Strong problem-solving skills and the ability to work independently and as part of a team.
- Advanced knowledge of development good practices such as testing, code reviews and git.
Key Responsibility:
- Develop and maintain scalable data pipelines to process large volumes of data from various sources.
- Working closely with our product teams and ML engineers to build features that directly impact the robustness and accuracy of our search infrastructure and NLS engine
- Ensure data quality and consistency through rigorous testing and validation processes.
- Monitor and troubleshoot data pipeline performance and resolve any issues
Please reach out to [Confidential Information] if facing any issues while applying.
