
Lead Azure Data Engineer
- London
- £500 per day
- Permanent
- Full-time
Rate: £500/day
IR35 Status: Outside IR35
Start Date: ASAP - Interviews next week
Duration: 6 months with potential for extensionThe RoleYou'll lead the design and delivery of scalable data products that support financial analysis and decision-making. Working closely with BI and Analytics teams, you'll help evolve our data warehouse and implement best-in-class engineering practices across Azure.Key Responsibilities
- Build and enhance ETL/ELT pipelines in Azure Databricks
- Develop facts and dimensions for financial reporting
- Collaborate with cross-functional teams to deliver robust data solutions
- Optimize data workflows for performance and cost-efficiency
- Implement governance and security using Unity Catalog
- Drive automation and CI/CD practices across the data platform
- Explore new technologies to improve data ingestion and self-service
- Azure Databricks: Expert in Spark (SQL, PySpark), Databricks Workflows
- Data Pipeline Design: Proven experience in scalable ETL/ELT development
- Azure Services: Data Lake, Blob Storage, Synapse
- Data Governance: Unity Catalog, access control, metadata management
- Performance Tuning: Partitioning, caching, Spark job optimization
- Cloud Architecture: Infrastructure-as-code, monitoring, automation
- Finance Domain Knowledge: Experience with financial systems and reporting
- Data Modelling: Kimball methodology, star schemas
- Retail Experience: Preferred but not essential