LiftmyCVLiftmyCV
UG

Resident Solution Architect Databricks

Unison GroupOn-siteSydney, AU
AI job summary

As a Resident Solution Architect for Databricks at Unison Group, you will design and implement large-scale Azure and Databricks Data Lakehouse solutions. The role involves building and optimizing ETL/ELT pipelines, developing scalable data ingestion frameworks, and ensuring robust data governance and security. You will collaborate with cross-functional teams to translate business requirements into scalable data solutions, establish best practices, and mentor junior engineers, making a significant impact on data engineering and analytics initiatives within the organization.

Job description

Job Description

  • Design and implement large-scale Azure and Databricks Data Lakehouse solutions.
  • Build and optimize ETL/ELT pipelines for batch and real-time streaming workloads using Azure Data Factory, Databricks, and Apache Spark.
  • Develop scalable data ingestion frameworks and integrate diverse structured and unstructured data sources.
  • Optimize data storage and query performance using Delta Lake, Parquet format, partitioning, and Spark performance tuning techniques.
  • Implement robust data governance, security, and access control using Unity Catalog, Azure Key Vault, and least-privilege principles.
  • Build and maintain data quality frameworks using Great Expectations or similar validation tools for batch and streaming data.
  • Automate pipeline deployment and CI/CD processes using Azure DevOps, Git, and configuration management tools.
  • Enable advanced analytics and ML workflows by preparing curated datasets and integrating with ML platforms like MLflow.
  • Develop real-time streaming solutions using Kafka, Azure Event Hubs, and Databricks Structured Streaming.
  • Create reusable PySpark libraries and frameworks for data curation, reconciliation, notifications, and Delta table automation.
  • Migrate legacy Hive or on-prem data sources to cloud platforms while ensuring security, compliance, and performance.
  • Monitor pipeline performance, troubleshoot issues, and implement alerting and logging for proactive management.
  • Collaborate with cross-functional teams and stakeholders to translate business requirements into scalable data solutions.
  • Establish best practices, standards, and documentation for enterprise-wide data engineering and analytics initiatives.
  • Mentor junior engineers and promote knowledge sharing within the data engineering team.

Job details

Published
March 19, 2026
Last checked
April 23, 2026
Job source
jworkable
Workplace
On-site · Sydney, AU
First seen
March 19, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Unison Group

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.