LiftmyCVLiftmyCV
TH

MA - Ssr. Databricks Engineer - 189

ThalozRemoteTELECOMMUTE, Colombia

Job description

We are seeking a highly skilled Databricks Engineer to join our dynamic data engineering team. This role is pivotal in driving the migration of existing Hadoop workloads to the modern Databricks Lakehouse platform, enabling our organization to leverage scalable, efficient, and innovative data solutions. The ideal candidate will be instrumental in designing, developing, optimizing, and monitoring data pipelines and architectures that support our business intelligence and analytics initiatives. This position offers an exciting opportunity to work with cutting-edge cloud technologies, collaborate across multiple teams, and contribute to a strategic phased migration roadmap that will transform our data infrastructure.

Responsibilities

  • Identify and classify current Hadoop jobs and data sources to understand migration scope and complexity.
  • Collaborate with stakeholders to prioritize and select use cases for Minimum Viable Product (MVP) migration initiatives.
  • Set up and manage Databricks environments on Azure, ensuring optimal configuration for performance and security.
  • Lead pilot projects to migrate Hadoop workloads to Databricks, validating data integrity and system performance post-migration.
  • Design scalable, repeatable ETL processes using Apache Spark (Scala or PySpark) to transform and load data efficiently.
  • Implement monitoring solutions to track pipeline health, query performance, and cost management.
  • Utilize Databricks capabilities such as Delta Lake, Lakehouse Federation, Liquid Clustering, and Unity Catalog to enhance data governance and performance.
  • Work closely with data scientists, analysts, DevOps, and business teams to ensure alignment and successful delivery of data solutions.
  • Define and monitor KPIs related to migration success, data quality, system performance, and cost efficiency.
  • Assist in developing a phased, strategic roadmap for the full migration from Hadoop to the Databricks Lakehouse platform.
  • Implement and maintain data governance policies to ensure data security, privacy, and regulatory compliance throughout the migration process.

Job details

Published
October 2, 2025
Job source
jworkable
Workplace
Remote · TELECOMMUTE, Colombia
First seen
October 2, 2025
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Thaloz

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.