Functional Requirements for DQ and Data Profiling Tools

Jul 20 2010 (6:30 PM - 8:30 PM)

As more organizations recognize the important of data quality and its improvement, a host of vendors are ready to sell software and services to aid in that effort. However, before you start talking to the vendors, you must have a clear vision of what the data quality function will do. Understanding and documenting roles, sub-processes, and interfaces with other parts of the organization must be set up before any package is purchased. Data quality functions in an organization may be divided into three general areas:
1. Understanding of data behavior and finding data quality problems.

2. Correction of business processes and application design to improve high quality data capture.

3. Correction of latent (or incoming) data records against authoritative reference data.

The third is what most vendor products address—in name and address improvement referencing a variety of commercial and government sources. Indeed, much of the DQ vendor hype focuses around name and address corrections. And, indeed, there is much to do here. Where tools are less useful is in tabular data which is proprietary to the organization—essentially where there are no other versions of “truth” to check against.

This lecture looks at all of these aspects of data quality, with particular focus upon how to deal with vendors and establishing functional expectations of both the DQ department and any acquired DQ tools.


Michael Scofield, Asst. Professor, Health Information
Management, Loma Linda University

Michael Scofield is an Asst. Professor of Health Information Management at Loma Linda University. His work experience spans banking, manufacturing, and education. He is a popular speaker in the data management discipline, speaking for Data Management Association chapters and conferences, Oracle User Groups, Quality Assurance Association chapters, Institute for Internal Auditors, Data Warehousing Institute conferences and chapters, Association of Government Accountants, and other professional and general audiences. He has spoken in 26 states, Australia, and in the U.K. His articles appear in a variety of professional journals in data management. He is the recipient of the DAMA-International Community award in 2008.