Principal Data Engineer

Overview

In this role you will lead the evolution of Companies House's cloud data platform, shaping data services at scale. You will guide cross-functional teams, set engineering standards, and turn data into a strategic asset across the organisation. You will influence senior stakeholders, drive technology roadmaps, and deliver measurable data outcomes for users and the business. The position centers on building a platform with Snowflake and dbt to underpin analytical products and drive organisation-wide adoption. You will mentor engineers and champion modern engineering practices to support a data-driven culture.

Pay / Benefits

• Flexible working with hybrid options
• 30 days annual leave
• 8 bank holidays plus privilege day
• Civil Service Pension scheme
• funded training and professional qualifications
• career progression opportunities

Responsibilities

• Shape the strategic direction and evolution of the data platform to deliver long-term value and drive adoption
• Lead design, development and optimisation of scalable data engineering solutions and trusted datasets
• Provide technical leadership, establish standards and best practices for quality, automation, security and reliability
• Drive continuous improvements in data architecture, processes, performance and operations
• Maintain governance, documentation and knowledge sharing to support continuity and compliance
• Build relationships with senior stakeholders to promote data-driven decision making
• Coach and develop a team of data engineers, fostering an inclusive, high-performing culture
• Manage performance, resource planning and capability development to meet current and future needs

Key requirements

• Experience establishing data engineering best practices (coding standards, automation, testing, reproducibility, quality assurance, continuous improvement)
• Advanced SQL and practical experience with Python, Git, dbt, Snowflake, Databricks or comparable technologies
• Strong data modelling knowledge (Kimball methodologies, star schemas, fact/dimension modelling, slowly changing dimensions)
• Ability to own a data product/platform/service and shape its strategy and roadmap
• Commitment to fostering an inclusive, collaborative, high-performing team culture

• leadership
• stakeholder influencing
• collaboration
• SQL
• Python
• Git