Azure Data Engineer Databricks Pipelines

Build Azure data pipelines with Databricks, Python, and SQL for the Government of Ontario’s Ontario Data Platform, working five days onsite. Support enterprise data solutions across government organizations through scalable engineering and reliable processing.

This Data Engineer / Software Developer role focuses on designing, developing, and operating cloud data solutions. You will use Azure Data Factory, Azure Data Lake Storage, Databricks, Delta Lake, and Spark to enable analytics and reporting. The work also supports data quality, governance, integration, and platform improvement.

Key Responsibilities:

• Design batch and near real-time pipelines with Azure Data Factory
• Develop Databricks notebooks, workflows, and Spark processing jobs
• Build ingestion, ETL/ELT, validation, and reconciliation processes
• Manage Delta Lake tables and optimize workloads
• Automate processes through reusable frameworks and DevOps practices

Requirements:

• Degree or diploma in computing or related field
• Experience with ADF, ADLS, Databricks, Delta Lake, and Lakehouse architecture
• Strong Python, SQL, and Spark proficiency
• Experience with Azure DevOps, Git, CI/CD, and deployment automation
• Knowledge of governance, security, privacy, and access control

Apply Azure Data Factory, Databricks, Spark, Python, and SQL expertise to Ontario Data Platform delivery onsite.