Overview
The Data Engineer with GCP DevOps expertise plays a crucial role in the organization by designing, constructing, and maintaining scalable data processing systems. This position is essential for ensuring data availability and reliability for analytics and business intelligence initiatives. As a key player in the data team, the data engineer will collaborate with data scientists, analysts, and other stakeholders to optimize data workflows and ensure seamless data integration across cloud environments. With the increasing reliance on data-driven decision-making, the role of the Data Engineer becomes vital. This individual will have a strong background in Google Cloud Platform (GCP), data engineering, and DevOps practices to enhance data governance, security, and accuracy, thereby allowing the organization to extract valuable insights from its data assets.
Key Responsibilities
- Design, implement, and maintain data architecture on GCP.
- Collaborate with stakeholders to gather and document data requirements.
- Develop and optimize ETL processes for efficient data handling.
- Build scalable data pipelines to ensure reliable data flow.
- Integrate new data sources into existing systems and workflows.
- Manage and optimize cloud resources for performance and cost efficiency.
- Employ containerization technologies to streamline development processes.
- Monitor data quality and establish protocols for data cleansing.
- Implement data security measures to protect sensitive information.
- Work closely with the data science team to prepare data for analysis.
- Utilize version control for data-related software and scripts.
- Document data architecture and processes for future reference.
- Conduct performance tuning for databases and applications.
- Collaborate with DevOps teams to automate deployment processes.
- Provide support and troubleshooting for data-related issues.
Required Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 6 to 8 years of professional experience in data engineering.
- Strong expertise in Google Cloud Platform (GCP) services and architecture.
- Proficiency in SQL and relational databases.
- Solid programming skills in Python or similar languages.
- Experience with data modeling and database design.
- Hands-on experience with ETL tools and frameworks.
- Knowledge of containerization tools like Docker or Kubernetes.
- Familiarity with monitoring and logging tools for cloud applications.
- Understanding of big data technologies like Apache Spark or Hadoop.
- Experience with version control systems like Git.
- Strong analytical and problem-solving skills.
- Ability to work collaboratively in a team environment.
- Excellent communication and documentation skills.
- Certifications in GCP or data engineering are a plus.
Skills: devops,data modeling,monitoring and logging tools,etl processes,database design,sql,python,hadoop,gcp,data quality,data engineering,apache spark,version control (git),data security,containerization (docker, kubernetes),google cloud platform (gcp)
