Data Engineer

Canopius

  • London
  • Permanent
  • Full-time
  • 1 day ago
Job Category: UnderwritingJob Description:The RoleWe are seeking a motivated Data Engineer to support the development, and maintenance of a data-bricks lake house architecture. This role is critical in enabling the organisation to store, integrate, and analyse large volumes of data from diverse sources, ensuring data is accessible, accurate, and actionable for decision-making.Responsibilities:Key ResponsibilitiesData Architecture & EngineeringContribute to the design, development, unit testing, and deployment of data pipelines for extraction, ingestion, and transformation of data.Support the development of scalable data architecture to process and store datasets efficiently.Optimisation of data pipelines from internal and external sources using modern data engineering tools and frameworks.Ensure their deliverables meet the team standards and best practices.Collaboration & Stakeholder Engagement
  • Work closely with data scientists, analysts, and business stakeholders to understand data needs and deliver fit-for-purpose datasets and operational data capabilities.
Collaborate with team members to support applications and resolve technical challenges.Create and maintain high-quality documentation * Data Quality & Automation
  • Optimize automated data quality, cleansing, and validation processes within data pipelines.
Qualifications:Required Skills and ExperienceTechnical ProficiencyExperience in data warehousing, data lakes and data integration, transformation, and modellingHands-on experience with Databricks for scalable data processing and Azure SQL for data storage and querying..Proficiency in SQL, Python, and modern data pipeline tools (e.g. dbt).Data Governance & QualityDemonstrate an ability to implement data quality frameworks and ensure data integrity across systems.Familiarity with data privacy regulations (e.g., GDPR) and security best practices.Security & ComplianceFollow best practices for data security and privacy, ensuring compliance with internal policies and external regulatory requirements.Apply technical knowledge to deliver key projects, including the automation of data deployment, release, and upgrade processes.Cross-functional CollaborationExperience working in cross-functional teams and collaborating with other engineers.Preferred Skills and Qualifications
  • Databricks or Azure Data Engineer certifications (e.g. Databricks Certified Data Engineer Associate or Databricks Certified Data Engineer professional)
  • DBT Analytics Engineering Certification

Canopius