LiftmyCVLiftmyCV
WD

Data Quality Engineer

Western DigitalOn-siteBengaluru, IN
AI job summary

Western Digital is seeking a Data Quality Engineer to enhance the governance and trust of its enterprise Data Lakehouse platform. This role involves designing data quality pipelines and ensuring that data across various layers is certified and AI/BI-ready. Key responsibilities include building automated validation frameworks, defining certification workflows, and collaborating with AI/ML teams to integrate conversational analytics. Ideal candidates will have experience with data engineering, AWS tools, and data quality testing frameworks. A commitment to quality across nine dimensions of data is essential.

Job description

About the Role

We are seeking a skilled and forward-thinking Data Quality Engineer to advance the data trust, governance, and certification framework for our enterprise Data Lakehouse platform built on Databricks, Apache Iceberg, AWS (Glue, Glue Catalog, SageMaker Studio), Dremio, Atlan, and Power BI. This role is critical in ensuring that data across Bronze (raw), Silver (curated), and Gold (business-ready) layers is certified, discoverable, and AI/BI-ready. You will design data quality pipelines, semantic layers, and governance workflows, enabling both Power BI dashboards and Conversational Analytics leveraging LLMs (Large Language Models). Your work will ensure that all 9 dimensions of data quality (accuracy, completeness, consistency, timeliness, validity, uniqueness, integrity, conformity, reliability) are continuously met, so both humans and AI systems can trust and use the data effectively.

ESSENTIAL DUTIES AND RESPONSIBILITIES

Data Quality & Reliability

  • Build and maintain automated validation frameworks across Bronze → Silver → Gold pipelines.
  • Develop tests for schema drift, anomalies, reconciliation, timeliness, and referential integrity.
  • Integrate validation into Databricks (Delta Lake, Delta Live Tables, Unity Catalog) and Iceberg-based pipelines.

Data Certification & Governance

  • Define data certification workflows ensuring only trusted data is promoted for BI/AI consumption.
  • Leverage Atlan and AWS Glue Catalog for metadata management, lineage, glossary, and access control.
  • Utilize Iceberg’s schema evolution & time travel to ensure reproducibility and auditability.

Semantic Layer & Business Consumption

  • Build a governed semantic layer on gold data to support BI and AI-driven consumption.
  • Enable Power BI dashboards and self-service reporting with certified KPIs and metrics.
  • Partner with data stewards to align semantic models with business glossaries in Atlan.

Conversational Analytics & LLM Enablement

  • Prepare and certify datasets that fuel conversational analytics experiences.
  • Collaborate with AI/ML teams to integrate LLM-based query interfaces (e.g., natural language to SQL) with Dremio, Databricks SQL, and Power BI.
  • Ensure LLM responses are grounded on high-quality, certified datasets, reducing hallucinations and maintaining trust.

ML Readiness & SageMaker Studio

  • Provide certified, feature-ready datasets for ML training and inference in SageMaker Studio.
  • Collaborate with ML engineers to ensure input data meets all 9 quality dimensions.
  • Establish monitoring for data drift and model reliability.

Holistic Data Quality Dimensions

  • Continuously enforce all 9 dimensions of data quality:
  • Accuracy
  • Completeness
  • Consistency
  • Timeliness
  • Validity
  • Uniqueness
  • Integrity
  • Conformity
  • Reliability

Required

  • Advance experience in data engineering, data quality, or data governance roles.
  • Strong skills in Python, PySpark, and SQL.
  • Hands-on with Databricks (Delta Lake, Unity Catalog, Delta Live Tables) and Apache Iceberg.
  • Expertise in AWS data stack (S3, Glue ETL, Glue Catalog, Athena, EMR, Redshift, SageMaker Studio).
  • Experience with Power BI semantic modeling, DAX, and dataset certification.
  • Familiarity with Dremio or query engines (Trino, Presto).
  • Knowledge of Atlan or equivalent catalog/governance tools.
  • Experience with data quality testing frameworks (Great Expectations, Deequ, Soda).

Preferred

  • Exposure to Conversational Analytics platforms or LLM-powered BI (e.g., natural language query over Lakehouse/Power BI).
  • Experience integrating LLM pipelines (LangChain, OpenAI, AWS Bedrock, etc.) with enterprise data.
  • Familiarity with data observability tools (Monte Carlo, Bigeye, DataDogs, Grafana).
  • Knowledge of data compliance frameworks (GDPR, CCPA, HIPAA).
  • Cloud certifications: AWS Data Analytics Specialty, Databricks Certified Data Engineer.

Equal Opportunity Statement

Western Digital is committed to providing equal opportunities to all applicants and employees and will not discriminate based on their race, color, ancestry, religion, sex, gender identity, age, national origin, sexual orientation, or any other legally protected characteristics.

Application Deadline

Based on our experience, we anticipate that the application deadline will be 10/25/2024, although we reserve the right to close the application process sooner if we hire an applicant for this position before the application deadline. #LI-RR1

Job details

Published
March 3, 2026
Last checked
November 1, 2026
Job source
smartrecruiters
Workplace
On-site · Bengaluru, IN
First seen
March 3, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Western Digital

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.