Analyzing profiling in data warehouse
Analyzing profiling results
Data profiles provide tremendous volumes of metadata and our success in ensuring data quality is directly proportional to our ability to process and understand this information and make appropriate conclusions.
One approach is to manually go through profiles, one at a time, and skip through the basics while focusing on anything unusual or unexpected. Of course nothing beats human intelligence in the ability to mine knowledge out of wheel-presented data. However, the task may prove overwhelming.
An alternative approach is to use automated algorithms to narrow down data profiles to those with information deemed of further interest. Various approaches can be used. For instance, we can automate cross-referencing valid values in data dictionary or lookup tables against frequency charts in data profiles. We can also automate search for candidate substitutes for default values. We can even write a program that will identify any distribution charts with signs of clustering. All these techniques prove useful on a large project or as a part of a toolset of a data quality professional.
Dependency profiling uses various pattern recognition techniques to find hidden relationships between attribute values.
Hierarchy profiling shows the relationship between parent level and children level. One parent can have some children; one child can have one and only one parent at the same time.
Hierarchy profiling result shows a department is linked to two faculties, so a break in the hierarchy exists. It will be necessary to investigate.
- 4State-transition model profiling examines life...
- Timeline profiling looks for patterns in histor...
- Analyzing profiling results Data profiles provi...
- Mining basic statistics Attribute profiling pro...
- Attribute profiling examines values of individu...