Data Engineer - GCP

Responsibilities:

• Design, develop, and deploy scalable and efficient data pipelines, ETL processes, and data integration solutions using GCP technologies such as Dataflow, BigQuery, and Cloud Storage.
• Collaborate with data scientists and analysts to understand data requirements and ensure proper data modeling, schema design, and optimization for analytical purposes.
• Implement data quality and validation processes to ensure accuracy and consistency of data throughout the data pipeline.
• Optimize and fine-tune data pipelines for performance and efficiency, taking into account factors such as data volume, velocity, and variety.
• Monitor and troubleshoot pipeline performance, data processing issues, and system failures to maintain data integrity and availability.
• Implement security best practices to ensure data privacy and compliance with relevant regulations.
• Stay current with emerging GCP technologies and industry trends, and recommend relevant tools and techniques to improve data engineering processes.
• Collaborate with DevOps teams to ensure smooth deployment, automation, and continuous integration of data pipelines.
• Document data pipeline architecture, design decisions, and technical specifications for internal reference.
• Participate in code reviews, mentor junior team members, and actively contribute to the overall data engineering strategy.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field. Advanced degree is a plus.
• Proven experience as a Data Engineer with a focus on Google Cloud Platform (GCP).
• Strong proficiency in GCP services such as Dataflow, BigQuery, Cloud Storage, Pub/Sub, and Data Catalog.
• Proficiency in programming languages such as Python, Java, or Scala for building ETL processes and data pipelines.
• Solid understanding of data modeling, schema design, and data warehousing concepts.
• Experience with data integration and transformation frameworks, version control systems, and CI/CD pipelines.
• Familiarity with data governance, security, and compliance best practices.
• Strong problem-solving skills and ability to troubleshoot complex data pipeline issues.
• Excellent communication skills, both written and verbal, to collaborate effectively with cross-functional teams.
• Relevant GCP certifications (e.g., Google Cloud Professional Data Engineer) are a plus.