Dataset Quality Profiler

Data quality problems are the most common cause of failed analytics projects and unreliable model outputs. Duplicate records, inconsistent formats, null values in critical fields, out-of-range values, and referential integrity violations can silently corrupt analysis results if they go undetected. This AI role specializes in systematic, multi-dimensional data quality profiling — producing a clear, actionable picture of exactly where your data falls short and what to do about it.

The assistant profiles data quality across the six standard dimensions recognized by data governance frameworks: completeness (what percentage of values are populated versus null), uniqueness (duplicate record detection and key constraint violations), validity (value ranges, format conformity, domain constraint checks), consistency (cross-field and cross-table logical consistency), accuracy (where ground truth reference is available), and timeliness (data freshness relative to business requirements). Each dimension is assessed separately and scored to produce an overall quality scorecard.

You describe your dataset — its schema, intended use, and any known issues — and receive a structured profiling plan along with executable code in Python (using Great Expectations, pandas, or custom profiling logic) or SQL for database-native profiling. The assistant generates a quality issue inventory that catalogs every detected problem: its dimension, affected column or row subset, severity, estimated business impact, and a recommended remediation step.

Beyond detection, the assistant helps you design data quality rules that can be embedded in pipelines as ongoing checks, preventing quality degradation over time. It produces documentation suitable for data governance reviews, quality dashboards, and stakeholder communication.

Ideal for data engineers building ingestion pipelines, data stewards conducting governance reviews, analytics teams inheriting legacy data, and organizations preparing datasets for regulatory reporting or machine learning.