Senior Data Engineer Tenure:- 6 Months
- Job Type:- Hybrid (2 days in a week onsite
)
Location:- Bangalore Karnataka - Salary:-80K-90K in hand per month
- Mandatory Skills: GCP Cloud Services, Spark, Scala or Python, Hive, Hadoop, BigQuery Architecture, LookML, DataProc, Data Modeling, Query Optimization
- Developing and maintaining data pipelines for efficient data extraction, transformation, and loading (ETL) processes.
- Designing and optimizing data storage solutions, including data warehouses and data lakes.
- Ensuring data quality and integrity through data validation, cleansing, and error handling.
- Collaborating with data analysts, data architects, and software engineers to understand data requirements and deliver relevant data sets (e.g., for business intelligence).
- Implementing data security measures and access controls to protect sensitive information.
- Automating and improving data processes and workflows for scalability and efficiency.
- Monitoring data infrastructure for performance and reliability to address issues promptly.
- Keeping abreast of industry trends and emerging technologies in data engineering.
- Documenting data pipelines, processes, and best practices for knowledge sharing.
- Participating in data governance and compliance efforts to meet regulatory requirements.
- Providing technical support and mentoring to junior data engineers, if applicable.
- Continuously optimizing data architecture to support the company's evolving data needs.
- Collaborating with cross-functional teams to drive data-driven decision-making within the organization.
- Requirements:
- Strong knowledge of data processing in Spark (Scala or Python), Hive, and Hadoop.
- Experience with data modeling and query optimization.
- Deep understanding of BigQuery architecture, best practices, and performance optimization.
- Proficiency in LookML for building data models and metrics.
- Experience with DataProc for running Hadoop/ Spark jobs on GCP.
- Knowledge of configuring and optimizing DataProc clusters.
Job Types: Full-time, Temporary
Contract length: 6 months
Schedule: - Monday to Friday
Experience:
- total work: 6 years (Required)
Data Engineer: 6 years (Required)
Work Location: In person
