✅ The 6 Elements of Data Quality

✅ Data Quality

Analogy

Think of data like drinking water. If the water is muddy or polluted (bad data quality), it will make you sick if you drink it. If you build a business decision on bad data, the business will suffer.

"Garbage In, Garbage Out." If you put garbage data into a report, you get a garbage decision out.

❓ Why Do We Need Data Quality?

Bad data costs companies millions of dollars. If data quality is poor:

Marketing sends emails to the wrong addresses.
Finance calculates the wrong profit.
The CEO makes bad decisions because the dashboard is lying.

⚠️ Key Data Quality Issues

These are the most common "pollutants" in data:

Duplicates: The same person or transaction appears twice. (e.g., Two profiles for "Raj Kumar").
Nulls (Missing Data): Empty fields where there should be information. (e.g., A customer profile with no email address).
Inconsistency: The data disagrees with itself across different systems. (e.g., CRM says Sales are $100, but Finance says$ 80).

💎 The 6 Elements of Data Quality

How do we measure if data is "good"? We check these 6 things:

Element	What it means	Simple Example
1. Completeness	Is anything missing?	Does every customer have a phone number? (No Nulls)
2. Accuracy	Is it correct in the real world?	Is Raj's phone number actually his real number?
3. Consistency	Does it match everywhere?	Does the HR system and Payroll system both show the same salary?
4. Validity	Is it in the right format?	Is the email address formatted like `name@email.com`?
5. Reliability	Can we trust the source?	Did we get this data directly from the customer, or a sketchy 3rd party?
6. Timeliness	Is the data fresh?	Are we looking at today's sales, or last month's?

🛡️ How to Check Data Quality

To stop bad data from entering the Data Warehouse, we run checks (tests) during the ETL process.

1️⃣ Data Flow Checks

Making sure the right amount of data moved from point A to point B.

Example: If we extracted 10,000 rows from the source, did we load exactly 10,000 rows into the warehouse?

2️⃣ Structural Integrity Checks

Making sure the "shape" of the data is correct.

Example: Checking that a column meant for "Age" only contains numbers, not letters like "Twenty".

3️⃣ Business Rule Validation

Making sure the data makes logical sense for the business.

Example: A bank account balance cannot be negative.
Example: An employee's joining date cannot be in the future.

4️⃣ Transformation Quality Checks

Making sure that when we changed the data (Transformation), we didn't accidentally break it.

Example: When mapping M to Male, did we accidentally leave any Ms behind?

🧪 Practice Drill

text

// Try answering these:
// Q1. What does the phrase "Garbage In, Garbage Out" mean in data warehousing?
// Q2. An employee accidentally types their email as `john.smith.gmail.com` (missing the `@`). Which of the 6 elements of data quality did this fail?
// Q3. A report shows today's total sales, but the data hasn't refreshed since yesterday. Which of the 6 elements did this fail?
// Q4. A system checks to make sure that nobody's Age is listed as `-10`. What kind of quality check is this?
// Q5. Match the definition to the Data Quality Element:
// | Definition | Element |
// |---|---|
// | The data matches across all different systems. | ? |
// | There are no blank or missing fields. | ? |
// | The data represents the real-world truth. | ? |

💡 Click for Solutions

A1. It means if you load bad, dirty data into your warehouse, the reports and decisions that come out of it will also be bad.

A2. Validity. The email is not in a valid format.

A3. Timeliness. The data is not fresh.

A4. Business Rule Validation. (Age cannot logically be a negative number).

A5.

Definition	Element
The data matches across all different systems.	Consistency
There are no blank or missing fields.	Completeness
The data represents the real-world truth.	Accuracy

← Previous Topic | Next Topic → Next Topic