Data Engineer (Databricks)

JOB SUMMARY

We are seeking an experienced Data Engineer with strong hands-on Databricks expertise to support the design, development, and optimization of enterprise data solutions. The successful candidate will have deep experience building and maintaining ELT pipelines, working with large-scale data platforms, and applying modern data architecture principles. This role requires alignment to UK business hours and on-site presence at our delivery center.

JOB RESPONSIBILITIES

• Design, develop, and maintain ELT pipelines using Databricks and related technologies. 
• Build and optimize large-scale data processing workflows using PySpark and SQL. 
• Develop and manage data models to support analytics, reporting, and business intelligence requirements. 
• Implement and support data Lakehouse architectures and data warehousing solutions. 
• Collaborate with data analysts, architects, and business stakeholders to understand and deliver on data requirements. 
• Ensure data quality, integrity, and governance across all data platforms. 
• Monitor and troubleshoot data workflows, pipeline performance, and platform issues. 
• Contribute to continuous improvement of data engineering standards and practices. 
• Support and work during UK business hours as required.

REQUIRED TECHNICAL SKILLSET

• Strong hands-on experience with Databricks (Delta Lake, Databricks Workflows, Unity Catalog). 
• Proven experience building and supporting ELT pipelines at scale. 
• Strong proficiency in PySpark and SQL. 
• Experience with data modeling techniques and best practices. 
• Good understanding of data Lakehouse architecture and data warehousing concepts. 
• Experience with version control tools (e.g., Git). 
• Familiarity with cloud platforms (Azure preferred; AWS or GCP considered). 

PREFERRED EXPERIENCE

• Experience with orchestration tools such as Apache Airflow or Azure Data Factory. 
• Exposure to data governance frameworks and tools (e.g., Unity Catalog, Microsoft Purview). 
• Experience in enterprise consulting or regulated industry environments. 
• Experience working with globally distributed teams across different time zones. 

CERTIFICATIONS PREFERRED

• Databricks Certified Associate Developer for Apache Spark. 
• Databricks Certified Data Engineer Associate or Professional. 
• Microsoft Certified: Azure Data Engineer Associate (DP-203) or equivalent. 
• Additional data or cloud certifications are a plus.