
Data Engineer - Contingent
- London
- Permanent
- Full-time
- Establishing and building processes based on business and technical requirements to channel data from multiple inputs and store using any combination of distributed (cloud) structures, local databases or other applicable storage forms
- Develop and own technical tools leveraging big-data to cleanse, organise and transform data to maintain data structures and integrity on an automated basis in real time
- Develop solutions that are functional, reliable, maintainable, scalable and extensible
- Resolving complex issues that may impact multiple business areas
- Maintaining relationships with internal and external stakeholders
- Collaborating with Data Engineers, Software Engineers, and Data Scientists both within the team and across the business
- Writing clear and relevant documentation
- Expert level (5+ years' experience) Python, including data manipulation packages
- Expert level (5+ years' experience) SQL
- Object Oriented Programming (OOP)
- Familiar with Airflow
- Familiar with the Software Development Lifecycle
- Creative problem-solving
- Meticulous attention to detail
- Comfortable with working independently and taking ownership
- Willingness to work outside of area of expertise
- Data processing frameworks (e.g. Spark, Hadoop, Kafka, FluentBit, RabbitMQ)
- Data warehousing solutions (e.g. PostgreSQL, BigQuery)
- Cloud Platforms (e.g. Google Cloud Platform, AWS)
- CI/CD tools (e.g. GitHub Actions, Jenkins)
- Containerization solutions (e.g. Docker and Kubernetes)
- Version control systems (e.g. Git)
- Package managers (e.g. Poetry)
- Jinja2 SQL templating
- Linux scripting
- Agile change management processes
- Comfortable with coaching members in the team
- A passion for developing creative solutions to real user needs and business problems
- Experience in aviation connectivity
- Experience in telecommunications industry