Timeline profiling in data warehouse
Timeline profiling looks for patterns in historical data (such as temporal distribution of the data, patterns of values for different time periods, etc…).
Profiling techniques discussed thus far provide key input to data mapping and quality assessment decisions. Specifically, we can identify databases, entities, and attributes containing desired data and understand their meaning. However mere presence of a certain attribute is not enough. Most data maintained in the databases is historic in nature and a certain depth of history is expected.
Timeline profiling is the process of understanding the actual depth of the historic data stack and any patterns that data values have over time.
Entity timeline profiling
The objective of entity timeline profiling is to identify which data is available for which time periods. For each entity we build a distribution of records effective dates. Compare the minimum and maximum dates on the distribution charts and adjust for distribution tails.
Event timeline profiling
The objective of event timeline profiling is to identify availability of data about different kinds of events for different time periods.
Value timeline profiling
The objective of value timeline profiling is to verify that an attribute did not take different meanings over time. This situation is common when a code usage changes historically, or when a past data consolidation brought data from sources with different uses of the same code.
- 4State-transition model profiling examines life...
- Timeline profiling looks for patterns in histor...
- Analyzing profiling results Data profiles provi...
- Mining basic statistics Attribute profiling pro...
- Attribute profiling examines values of individu...