Data Cleaning

Data warehouses, which are repositories of data collected from several data sources, form the backbone of most current CRM and decision support applications. Since data sources are independent, they may adopt independent, and potentially inconsistent, conventions. For example, one source may adopt the use of standard descriptions while another source adopts the use of fully expanded descriptions. Moreover, data entry mistakes at any of these sources introduce more errors. Since high quality data is essential for gaining the confidence of users of CRM and decision support applications developed over data warehouses, ensuring high data quality is critical to the success of data warehouse implementations. Therefore, significant amounts of time and money are spent on the process of detecting and correcting errors and inconsistencies.

The process of cleaning dirty data is often referred to as data cleaning. Since the types of errors and inconsistencies can be domain-specific, it is important and challenging to develop generic domain-independent data cleansing solutions. Our technique is to use a set of domain-independent tools to perform effective and efficient data cleaning across a broad spectrum of applications.

Data cleaning looks simple. For example, before conversion an address could have been submitted in the following ways:

The results make the process seem simple. It isn't. We make it simple for you.