Explore data types, quality dimensions, and their impact on AI performance. Assess what good data looks like.

The Model Doesn't Know Your Data Is Broken

When an AI system performs poorly, the instinct is to question the model: the architecture, the prompting, the fine-tuning. But in practice, the leading cause of AI failure in production environments is not the model—it's the data the model was trained on, evaluated against, or asked to process. Garbage in, garbage out is not a cliche. It's the most reliable pattern in applied AI.

What makes this hard is that the model won't tell you. An LLM trained on biased historical records will generate biased outputs with complete fluency and apparent confidence. A classifier trained on mislabeled data will learn the wrong task and score well on the wrong metric. A recommendation system fed duplicate records will overweight certain users without any visible error in the pipeline. The system works. It just doesn't work correctly—and you may not discover this until it matters.

This lesson gives you the vocabulary, frameworks, and diagnostic habits to identify data problems before they compound. That's the real skill here: not just knowing what good data looks like in theory, but being able to spot bad data in the wild.

Why This Matters for AI Practitioners

Data quality is not a data engineering concern that gets handed off to someone else. AI practitioners are in the decision loop at every stage where data problems can emerge: selecting training sources, scoping evaluation datasets, interpreting model outputs, and advising on deployment. If you can't recognize a data quality issue when you see one, you can't catch it before it propagates into a shipped system.

The stakes are asymmetric. A data quality problem caught during dataset review costs hours. The same problem caught after deployment can mean retraining from scratch, regulatory exposure, reputational damage, or decisions made at scale on faulty outputs. In high-stakes domains—healthcare, finance, legal, hiring—the downstream costs of poor data can be severe and, in some jurisdictions, legally actionable.

Understanding data types also matters more than it first appears. The structural properties of data—whether it's numerical, categorical, textual, temporal, relational—determine which models are appropriate, how preprocessing should work, what failure modes are likely, and how results should be interpreted. Choosing the wrong model architecture for a data type is a category error, not an engineering tradeoff. It's the kind of mistake that wastes months of work.

Core Concepts

Structured, Semi-Structured, and Unstructured Data

Structured data is organized into a defined schema—rows and columns in a relational database, spreadsheet cells, or well-defined fields in a form. Every record has the same fields in the same positions. This makes structured data easy to query, aggregate, and feed into tabular ML models. Most classical machine learning (gradient boosting, logistic regression, random forests) was developed and optimized for structured data.

Unstructured data has no predefined schema. Text documents, images, audio recordings, and video fall into this category. The majority of data generated in the world is unstructured, and it's the primary input to LLMs, vision models, and speech systems. Working with unstructured data requires preprocessing pipelines that extract structure from raw content—tokenization for text, feature extraction for images—before models can act on it.

Semi-structured data sits in between: it has some organizational properties (tags, hierarchies, key-value pairs) but doesn't conform to a rigid relational schema. JSON, XML, HTML, log files, and email headers are common examples. Semi-structured data appears frequently in API responses, web scraping, and event streams. Handling it well requires understanding its native schema conventions and the ways they tend to drift or break in practice.

Data Types Within Datasets

Within a dataset, individual features have types that constrain how they should be processed and modeled:

Numerical data is either continuous (measurements like temperature, price, or probability scores that can take any value in a range) or discrete (counts like the number of transactions or items in a cart). The distinction matters: averaging a continuous measurement makes sense; averaging a discrete count often does not.

Categorical data represents membership in a group—product category, user role, country, sentiment label. Categorical features require specific encoding strategies (one-hot encoding, label encoding, target encoding) because most ML algorithms operate on numbers. Applying numerical operations to raw categorical codes—treating "category 2" as arithmetically greater than "category 1"—introduces spurious ordinal relationships the data doesn't actually have.

Ordinal data is categorical with a meaningful order: satisfaction ratings (1-5), education levels, risk tiers. Unlike purely categorical data, the order carries information. Unlike continuous numerical data, the intervals between values aren't necessarily equal. Treating ordinal data as purely numerical (assuming equal intervals) or purely categorical (discarding the order) both introduce distortions.

Temporal data includes timestamps, dates, time series, and sequences. Time introduces dependencies that violate the i.i.d. (independent and identically distributed) assumption that underlies most standard ML. A model trained on past data and evaluated on past data will produce optimistic performance estimates—the evaluation must respect temporal ordering to be valid.

Text data is sequential, context-dependent, and requires its own preprocessing pipeline. The distinction between text as a feature (a field in a structured record) and text as the primary input (a document, conversation, or article) matters for how you handle it architecturally.

The Five Dimensions of Data Quality

Data quality is not a single property—it's a set of distinct dimensions that can be assessed independently and that fail independently:

Accuracy is whether the data correctly represents the real-world entity or event it's supposed to represent. An address field that contains an outdated address is inaccurate. A label assigned by an annotator who misunderstood the task is inaccurate. Accuracy failures are often silent—they look like valid data, they just reflect the wrong reality.

Completeness is whether all required values are present. Null values, missing records, and partially filled fields are completeness failures. Completeness is visible in a way accuracy often isn't: you can detect missing values programmatically. What matters is understanding whether the missingness is random or systematic—missing at random versus missing not at random—because the two require different treatment strategies.

Consistency is whether the same entity or concept is represented the same way across the dataset. "New York", "NYC", "New York City", and "new york" all refer to the same place, but a model that counts them separately has a consistency problem. Schema drift—fields changing format between data collection periods—is a consistency failure that frequently emerges in long-running datasets.

Timeliness is whether the data reflects the current state of the world for the use case it's being applied to. A customer database from three years ago may be accurate as of three years ago but misleading for today's decision. Training on stale data and deploying against current reality creates distribution shift—one of the most common causes of model degradation in production.

Validity is whether values conform to the expected format, type, and range. A birthdate of "1875-03-02" for an active user account is technically parseable but almost certainly invalid. Negative transaction amounts when only positive values are expected, email addresses without an "@" sign, or free-text entries in a field supposed to contain controlled vocabulary are all validity failures.

Real-World Examples

Label Quality in Classification Systems

A financial services team builds a document classifier to automatically route incoming contracts to the right review queue. They have 50,000 labeled historical documents and achieve 94% accuracy on their held-out test set. They deploy. Within weeks, the routing accuracy drops to around 70% and the team starts getting complaints.

The investigation reveals two problems. First, the test set was sampled randomly from the same historical batch as the training set—same annotators, same time period, same document formats. It wasn't independent in any meaningful sense. Second, around 15% of the historical labels were wrong, applied during a period when the labeling guidelines were unclear. The model learned the labeling errors as well as the legitimate patterns. The test set reflected those errors too, so accuracy looked high.

The fix required going back to the labeling process: clarifying guidelines, re-annotating a clean evaluation set, and auditing the training labels for the most error-prone categories. The data problem masqueraded as a model problem for weeks before anyone looked at the labels themselves.

Training Distribution vs. Deployment Distribution

A healthcare AI team trains a triage model on three years of patient intake records. Performance on the historical validation set is strong. After deployment, the model consistently underperforms on new patients in ways that don't match the validation results.

The cause: the training data was collected before a significant change in intake form design. Several fields were renamed, two new fields were added, and one field that had been free-text was converted to a dropdown. The model was trained on the old schema. When it encountered the new schema in production, the feature alignment broke silently—the pipeline didn't crash, it just fed the wrong values into the wrong model features.

This is a consistency and timeliness failure compounded by inadequate schema versioning. Catching it required comparing feature distributions between training data and recent production data—a check that should have been part of the deployment process from the start.

Where People Get This Wrong

Practical Takeaways

These are the habits and practices that distinguish practitioners who catch data problems early from those who discover them in production:

• Audit before you train. Before any model development begins, profile the dataset: check null rates per feature, value distributions, label balance, duplicate records, and schema consistency across time slices. Most data quality problems are visible in exploratory analysis—they just require someone to look.
• Separate completeness from accuracy in your assessment. Missing values are easy to detect. Inaccurate values look identical to correct ones. Make sure your data quality checks address both: programmatic null detection for completeness, and human or cross-reference validation for accuracy in critical fields.
• Respect temporal ordering in evaluation splits. If your data has a time dimension, your train/test split must reflect it. Train on earlier data, evaluate on later data. Random splits on temporal data produce optimistic results that won't hold in deployment.
• Track distribution drift in production. After deployment, monitor the statistical properties of incoming data against the training distribution. Feature means, null rates, category frequencies, and label distributions should all be monitored with alert thresholds. Drift doesn't always mean your model is wrong—but it means you need to investigate.
• Treat data types explicitly, not by assumption. When ingesting data, validate that each feature conforms to its expected type and range before it enters any model pipeline. Type coercions that happen silently (a date field read as a string, an integer field parsed as float) are a common source of subtle bugs that degrade model performance without obvious errors.
• Document provenance for training data. Where did the data come from, when was it collected, who labeled it, what guidelines were in place, and what was the original intended use? Without this metadata, you cannot assess fitness for a new use case or diagnose a quality problem that surfaces months later.

Before You Move On

Make sure you can answer these questions with confidence before proceeding to the next lesson:

• What is the difference between structured, semi-structured, and unstructured data—and why does the distinction affect which models and pipelines are appropriate?
• What are the five dimensions of data quality, and how can each fail independently while the others appear fine?
• Why does "missing at random" require different treatment than "missing not at random"?
• What is distribution shift, and why does a strong held-out test set score not guarantee strong production performance?
• What are the key checks you would run to profile a new dataset before beginning model development?