LiftmyCVLiftmyCV
TL

Principal Data Engineer

The London Metal ExchangeOn-siteLondon, GB
AI job summary

The Principal Data Engineer role focuses on shaping the LME's strategic data platform to efficiently process and serve high-volume trading data. The position requires over 10 years of software development experience, particularly with JVM-based languages, and involves building scalable data platforms and modern Data Lakehouse architectures. Successful candidates will demonstrate strong problem-solving abilities and a proactive attitude in leading team efforts while managing stakeholder expectations.

Job description

Principal Data Engineer Shift Pattern:

  • Standard 40 Hour Week (United Kingdom)

Scheduled Weekly Hours:

  • 40

Corporate Grade:

  • C - Vice President

Reporting Line:

  • (UK Division) Information Technology

Location:

  • UK-London

Worker Type:

  • Permanent

The purpose of this role is to help shape the LME’s strategic data platform which can reliably process, store, and serve high-volume, high-velocity trading data at scale, while providing engineering teams with reusable, self-service capabilities that accelerate the delivery of trading and analytical solutions. Success in this role means delivering a highly scalable and resilient data platform that becomes the engineering foundation for all data-intensive trading workloads, enabling faster innovation, improved reliability, reduced operational complexity, and better utilisation of market and trading data across the organisation. Strong Required Demonstrable Experience of:

  • A candidate for this position must have got working experience in IT organisation as a technologist who has evolved from a strong background in hands on software development:
  • 10+ years of experience with software development, preferably working with JVM based languages (Java/Scala/Kotlin).
  • Provide expertise on the development of highly scalable data platforms, data warehouses (SQL server, Postgres), and modern data lakes.
  • Solve complex scalability, reliability, latency, and performance challenges associated with large-scale distributed systems.
  • Provide self-service platform capabilities that allow engineering teams to onboard, process, discover, and consume data efficiently.
  • Influence technology strategy and roadmaps across Trading Technology, Data Engineering, and Platform Engineering functions.
  • Build high-volume distributed data platforms capable of processing billions of events per day and managing terabytes scale datasets.
  • Develop modern Data Lakehouse platforms leveraging S3-compatible object storage, Apache Iceberg, Spark, and Trino.
  • Build real-time streaming integration using Apache Kafka and Apache Flink for low-latency event processing, enrichment, and data distribution.
  • Drive performance engineering and scalability across data ingestion, storage, streaming, query, and analytics platforms.
  • Design and build self-service data platform capabilities including APIs, data products, metadata services, and developer enablement frameworks.
  • Implement cloud-native platform engineering practices using Kubernetes/OpenShift, Terraform, CI/CD, GitOps, and Infrastructure as Code.
  • Lead metadata, lineage, and data discovery capabilities using technologies such as DataHub, OpenMetadata, or Apache Atlas to improve platform transparency and usability.
  • Ensure platform security and regulatory compliance through fine-grained access controls, encryption, secrets management, auditing, and secure-by-design engineering principles.

Bonus for knowledge of:

  • Automation/configuration management using toolsets such as Puppet, Chef, Ansible or equivalent
  • Docker and Kubernetes.
  • Knowledge and understanding of financial markets.
  • Setting up CI/CD pipelines using Bamboo.
  • Confluent certified developer for Apache kafka.
  • Dollar Universe

Personal Qualities:

  • Ability to work under pressure with changing priorities, with a view to resolving issues innovatively, and meeting key stakeholders expectations.
  • A dynamic and self-motivated attitude
  • Accountable and proactive
  • Able to provide leadership and motivate team demonstrating strong interpersonal skills
  • Must display strong analytical skills and attention for detail.
  • Demonstrates pragmatic judgement, balancing risk and business value to reach decisions which are well informed and actionable.

Job details

Published
September 7, 2026
Job source
myworkdayjobs
Workplace
On-site · London, GB
First seen
September 8, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at The London Metal Exchange

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.