Understanding Data Quality Cycle

What is data Quality?????

The process of conditioning data to meet the specific needs of business users.
Data Quality ensures data is correct, consistent, and complete by monitoring, analyzing, and reporting.

Data Quality Life Cycle can be broadly classified into two steps

1. Data  Profiling/Data Quality Assessment
2. Data Cleansing

Data Profiling:Data Profiling is the analysis of data to identify data quality issues w.r.t various defined data quality attributes which are listed below.

1.Completeness
2.Conformity Of Data
3.Consistency
4.Accuracy
5.Duplicates
6.Address validity
7.Timeliness

Once the Issues are identified in the data w.r.t above attributes,we start the process of Data Cleansing.

Data Cleansing:Data Cleansing in the process of correcting the issues identified during the Profiling exercise.Few of the One-time cleansing activities are listed below.

1.De-Duplication:To identify the Survivor record among the identified duplicates and merge the duplicates into Survivor Record

2.Address Enrichment:To enrich or correct the address data.To enrich the addresses.

3.Field Level Cleansing:To apply cleansing logic specific to few fields.

4.Data Standardization:Standardizing the data w.r.t standards defined.Ex:Country Name Standardizing to ISO 3-Digit Format.Ex :United States to USA.

5.Parsing:parsing is the process of splitting the values in a field to multiple items.
Ex:Name to First Name,Middle Name,Last Name

Once the Data Cleansing exercise is complete,we prepare the data management Strategy.

Impact Of Poor Data Quality-Real Time Scenario

In this Competitive world,organizations are running fast to outpace each other and grab customer market.

What happens when the customer Information they have is incorrect?

How it impacts their Business?

Lets analyze the below real time scenarios...

A organization which has a good customer base is willing to offer discounts to customers in this festival season,hence they wish to send promotional Emails and Sms to Customer Phone Numbers.

If the Quality of the data present in Email and Phone Number Columns is upto the mark(may be around 80%) then organization might be successful in Promoting their Discount Sale to Atleast 70-80% of their Privileged Customers...

Let us discuss about few of the data quality Issues with respect to above scenario.

Completeness Of Data:If Phone Numbers and Email Addresses are not populated(If they are Null),then Organization will not be able to reach out to them.

Duplicates in Customer Data: If the Customer records are repeating in the System holding Customer records,organization might end up sending Multiple emails and Sms to customer which might irritate him.

Accuracy Of Customer Data:Though data is populated in Email and Phone Number Fields,if it is not accurate then Organization may not be able to reach out to customer.
Ex:Domain Names not Proper.ex:Instead of abc@yahoo.com,email id recorded as abc@yaho.com

Let us assume a case where above organization plans to send out postal mails to their privileged customers about the ongoing special offers.
If the addresses for those privileged customers are not complete and not accurate,chances of Postal mails not reaching customer is high.In this case company's operational cost in sending postal mails to customers is a waste of money

Business Impact: Loss of Revenue,Operational Cost

Impacts of Poor data quality can be many,listing few of them:

1. Loss of Revenue
2. Increase in Operational Cost
3. Poor Customer Experience
4. Poor Conversion of Leads to Customers
5. Forecasting and decision making

List Goes on......Improve the quality of data and Increase your Business Efficiency...