data engineer (GCP)

Описание

Capco is a global technology and management consulting company that helps financial institutions address challenges across banking, payments, capital markets, wealth, and asset management.

Задачи

• Design, develop, and maintain scalable data pipelines and data processing solutions
• Develop data transformation and processing workflows using Python and Apache Spark
• Work with large-scale datasets in distributed environments using Hadoop and related technologies
• Build and support cloud-based data solutions on Google Cloud Platform (GCP)
• Develop reliable ingestion processes integrating data from multiple source systems
• Implement data transformations, validation rules, and data quality checks
• Troubleshoot data pipeline issues and support performance optimization
• Work with Linux-based environments, including scripting, deployment, and operational activities
• Collaborate with Data Engineers, Architects, Analysts, and other project stakeholders to translate business requirements into technical solutions
• Participate in code reviews and follow software engineering and data engineering best practices
• Create and maintain technical documentation covering data flows, dependencies, configurations, and operational procedures
• Support deployment, testing, stabilization, and ongoing maintenance of data solutions

Требования

• 2–4+ Years of commercial experience in Data Engineering or a similar role
• Good hands-on programming skills in Python
• Practical experience with Apache Spark, including building and maintaining data processing jobs
• Experience working with Hadoop or distributed data processing ecosystems
• Good knowledge of Linux and command-line environments
• Commercial experience with Google Cloud Platform (GCP) and relevant data services
• Good understanding of ETL/ELT processes, data pipelines, and data transformation concepts
• Working knowledge of SQL and relational data concepts
• Understanding of data quality, monitoring, and troubleshooting practices
• Familiarity with Git and modern software development practices
• Ability to work effectively in an Agile environment and collaborate with distributed teams
• Good communication skills
• English at a minimum B2 level
• Будет плюсом: Experience with GCP services such as BigQuery, Cloud Storage, Dataproc, Dataflow, or Pub/Sub, experience in the financial/banking domain, experience with orchestration tools such as Apache Airflow, familiarity with CI/CD processes for data solutions, knowledge of data modelling and data warehouse concepts, experience working with financial services or banking clients, familiarity with containerization technologies such as Docker or Kubernetes

Условия

• B2B contract
• Innovative AI and automation initiatives for leading financial institutions