Senior Data Engineer

Location: London (3 days per week in office)

Our client is seeking a Senior Data Engineer to join a high-performing product team delivering enterprise-scale data platforms and data products that drive critical decision-making across a complex trading environment.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and data products using Python, PySpark, Databricks, SQL Server, and Azure.
• Build and support FastAPI services and REST APIs that enable data access and integration.
• Collaborate with product owners, architects, and business stakeholders to deliver robust technical solutions.
• Lead development across the full software lifecycle, including design, testing, deployment, monitoring, and support.
• Implement and optimise CI/CD pipelines, automated testing, and release processes using GitOps and modern DevOps practices.
• Develop and support containerised applications using Docker and Kubernetes.
• Drive engineering excellence through code reviews, observability, security, and operational best practices.
• Troubleshoot production issues and continuously improve platform performance and reliability.

Must-Have Skills:

• Python
• PySpark
• Databricks
• FastAPI
• SQL Server
• Docker
• GitOps
• CI/CD and DevOps practices
• Data platform and data pipeline engineering experience

Nice-to-Have:

• Trading, commodities, or energy sector experience
• Data architecture and data modelling expertise