Data Engineer - AWS - SC Cleared

Data Engineer - AWS - SC / NPPV3 Cleared

We are seeking an experienced Data Engineer to support the delivery of an AWS-based data solution within a UK Government programme. The role involves building scalable data pipelines, transforming complex datasets, and implementing data-matching and record-linkage solutions.

You will work closely with the Technical Lead and Lead Data Scientist to deliver reliable, production-ready data solutions.

Key Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines using AWS Glue, Redshift, Python and SQL.

• Ingest, transform and validate data from multiple sources and formats.

• Implement data matching, deduplication and record-linkage processes.

• Develop automated testing, monitoring and data quality controls.

• Optimise data pipelines for performance, scalability and cost.

• Apply engineering best practices, including Git, CI/CD and secure development.

• Collaborate with technical teams and stakeholders within an Agile environment.

Essential Skills & Experience

• Active SC clearance (NPPV3 clearance desirable)

• Strong hands-on experience with AWS Glue, Amazon Redshift, Python and SQL.

• Proven experience building production-grade data pipelines.

• Strong understanding of ETL/ELT, data modelling and data quality.

• Experience with data matching, record linkage or complex data transformations.

• Experience with automated testing, Git and CI/CD.

• Experience working within Agile/Scrum delivery teams.