Lead Data Engineer

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure, high performance, cloud-based data infrastructure for the enterprise. Our proprietary technology solutions drive performance and reduce costs while helping our clients to improve the management and sharing of data across their organisations.

In 2024, Aker Systems won the Breakthrough Culture Awards highlighting growth companies putting culture first. In 2020 Aker Systems was recognised as a ‘One to Watch’ on the Sunday Times Tech Track. The Company was also recognised at the Thames Valley Tech Awards 2020; winning the Thames Valley Tech Company of the year, the Emerging Tech Company and High Growth Tech Business categories. We encourage people of all different backgrounds and identities to apply. We are committed to maintaining an inclusive, and supportive place for you to do your very best work.

We are seeking an experienced and hands-on Lead Data Engineer to design, build and evolve secure, scalable data platforms for clients across the UK Public Sector and Utilities sectors.

You will work closely with architects, engineers and client stakeholders to deliver modern cloud-based data solutions that support operational, analytical and AI-driven use cases. The role combines technical leadership with hands-on engineering, helping clients modernise their data estates while ensuring solutions are secure, reliable and scalable.

Working across multiple projects, you will lead the development of batch and real-time data pipelines, support architectural decision-making, and help establish engineering best practices across delivery teams.

Strong experience with real-time streaming technologies is essential.

Key Responsibilities

• Lead the design, development and deployment of secure, scalable data pipelines and data platform components.

• Design and implement batch and real-time streaming solutions using modern event-driven architectures.

• Own technical design decisions across projects, balancing scalability, security, performance and delivery requirements.

• Lead technical workshops, discovery sessions and architecture discussions with client stakeholders.

• Assess existing client data platforms and design practical approaches for modernisation, migration and integration.

• Build and maintain data ingestion, transformation and integration capabilities across a variety of data sources.

• Develop cloud-native data platforms using AWS services and open-source technologies.

• Ensure data quality, governance, lineage and security requirements are embedded throughout the data lifecycle.

• Drive engineering best practices including code reviews, automated testing, CI/CD and Infrastructure as Code.

• Support multiple client engagements, helping delivery teams maintain consistent engineering quality and standards.

• Mentor and support engineers across project teams, providing technical guidance and leadership.

• Perform data modelling, performance tuning and root cause analysis to resolve complex data and platform issues.

Essential Skills & Experience

Real-Time Streaming & Data Engineering

• Strong commercial experience designing and implementing real-time streaming data platforms.

• Hands-on experience with technologies such as Apache Kafka, AWS Kinesis, Apache Flink, Spark Streaming, Apache Pulsar, or similar event-driven technologies.

• Experience building high-volume, low-latency, resilient data pipelines in production environments.

• Strong understanding of modern data engineering patterns, including batch, streaming and hybrid architectures.

Data Engineering & Databases

• Extensive experience developing and operating enterprise-scale data platforms.

• Strong SQL skills and experience designing data models for analytical and operational workloads.

• Proficiency in Python or other modern programming languages used for data engineering.

• Experience configuring, tuning and optimising relational and NoSQL databases.

Cloud & Software Engineering

• Strong experience working with AWS cloud services, including EC2, RDS, Lambda, Athena, Glue, S3 and IAM.

• Experience working within Agile delivery environments.

• Strong understanding of Test-Driven Development (TDD) and automated testing practices.

• Experience implementing CI/CD pipelines and engineering automation.

• Strong understanding of Infrastructure as Code principles.

• Experience using version control systems such as Git.

Regulated Environments

• Experience delivering technology solutions within UK Public Sector, Utilities, Energy, Water, Financial Services or other highly regulated industries.

• Understanding of security, governance and compliance considerations within regulated environments.

Desirable Skills

• Experience with Terraform.

• Experience with Docker and Kubernetes.

• Knowledge of Ansible and Packer.

• Linux and Windows administration experience.

• Experience supporting AI, machine learning or advanced analytics platforms.

• Experience working on programmes requiring UK Government security clearance.

What We're Looking For

• Strong technical leadership and problem-solving skills.

• Ability to communicate effectively with both technical and non-technical stakeholders.

• Experience working in complex, regulated environments.

• A collaborative approach and willingness to support and mentor others.

• A focus on delivering practical solutions that create value for clients.

• A passion for building reliable, scalable and secure data platforms.

Education

Bachelor’s degree in computer science, Software Engineering, Data Engineering, Mathematics or a related technical discipline, or equivalent practical experience.
Aker Systems Attributes…

Source: Arbeitnow (https://www.arbeitnow.co.uk/jobs/companies/akersystems/remote-lead-data-engineer-315863)