Data Engineer with Databricks experience

Role – Data Engineer

Location – London/ Woking - (3 days in office)

Salary: Upto £80,000

We’re working with a forward-thinking organisation looking to bring in a Data Engineer to shape and drive their data strategy, building a scalable, business-critical analytics layer.

Key Responsibilities

• Should have atleast 8+ yrs of experienc in design, build, and maintain scalable, high-performance data pipelines and modern Lakehouse architectures using Databricks, ensuring reliable data integration, orchestration, and operational excellence.
• Develop robust, business-focused data models and warehouse solutions that support analytics, reporting, and enterprise data initiatives, leveraging dimensional modeling, Star Schema, SCD, CDC, and Medallion Architecture best practices.
• Engineer efficient ETL/ELT workflows using SQL, Python, Apache Spark, and Databricks technologies, including Delta Live Tables, Workflows, Unity Catalog, DataFrames, UDFs, and Delta Sharing.
• Optimize data platforms through proactive monitoring, root cause analysis, performance tuning, and issue resolution while ensuring data quality, reliability, and timely stakeholder communication.
• Leverage modern Databricks capabilities such as Lakeflow Connect, Lakebase, Metric Views, Genie, and other emerging platform features to drive innovation and continuous platform improvement.
• Collaborate closely with cross-functional teams including Business Analysts, Data Architects, BI Developers, and Product Owners to deliver scalable, secure, and fit-for-purpose data solutions aligned with business priorities.
• Implement DevOps best practices by utilizing Azure DevOps, Git repositories, CI/CD pipelines, and Agile/Scrum methodologies to streamline development, testing, and deployment processes.
• Contribute to engineering excellence by producing comprehensive technical documentation, adopting AI-assisted development tools (e.g., Cursor), and promoting coding standards, knowledge sharing, and continuous improvement across the data engineering team.

Preferred Qualifications

• Strong hands-on experience with Databricks, SQL, Python, Apache Spark, and Pandas.
• Proven experience designing and delivering enterprise data pipelines, data warehouses, lakehouses, and analytics platforms.
• Master's degree in Data Science (or equivalent) is desirable.
• Azure certifications (AZ-900, DP-900, DP-203, DP-500) and Databricks certifications (Lakehouse Data Engineer Associate/Professional) are advantageous.