Most of the time, we do not have options for which systems to source data from. We need to get the data from the systems that generate the relevant data. However, the source system analysis is a critical part of determining the amount of time the ETL will take –- and therefore, how long the entire data warehouse project will take.

Source system analysis can be performed when certain information (next tip) can be gathered about the source system. Usually this takes place on a system currently in production that can be observed and potentially tested with. Products that are not in-house, or not even built, require vendor interface and the analysis can be based on speculation, rather than fact. Any source system analysis that is performed before this period of observation and hands-on interface occurs should be reevaluated after the system is in house.

Non-built source systems may provide an opportunity to influence the build of the system such that it is more data warehouse compatible. Time-stamping new and changed records and partitioning source databases are examples of this. Time-stamping gives the ability to more easily source only those records that have changed since the last extract. Partitioning source databases increases the availability possibilities for the source database while extract is occurring, thus increasing the chances that real-time, non-impacting data warehouse extracts can be performed someday.

For more information,

    Requires Free Membership to View

check out SearchCRM's Best Web Links on Data Warehousing.

This was first published in July 2002

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.