Training? We don't need no stinkin' training …

While data warehousing technologies and methodologies have evolved over the years, ideologies, in many cases, have not. Taking a narrow-minded approach to your data warehousing strategy can be like trying to drive with your parking brake on.

Rick Sherman, Athena IT Solutions

One of the biggest mistakes project teams make is in refusing to acknowledge the need for training. This is just one of the ways companies undermine their own efforts to take data warehousing to its full potential.

When faced with an unfamiliar task we generally use our previous experience as a point of reference to determine how we should react. Likewise, when people built their first data warehouse, they often modeled it after their familiar transaction processing systems. They built what they knew and were comfortable with instead of using the new paradigm offered by data warehousing.

As a result, the first generation of data warehouses often shared many data modeling and architectural characteristics with their data source systems. This tendency to use outmoded yet familiar techniques was exacerbated in the 1990's. At that time, there was such a rush to build IT systems that people often did not have time to learn new techniques and appro

aches. But the downside of building something in "Internet time" meant doing it fast and taking shortcuts. Each initiative – ERP, CRM, supply chain, data warehousing and many others – had marching orders to produce and to produce fast. There was not much time to learn new and more suitable approaches.

Compounding the problem was that with so many IT systems being built, there was a strain on the IT workforce. There were not enough experienced people for projects; many were stretched too thin. In order to obtain the appropriate resources, many companies relied on consultancies to build their IT systems. However, consulting firms were experiencing growing pains of their own, and had trouble hiring enough IT experts. In addition, many consultants, themselves, were struggling to get acquainted with data warehousing. Demanding timetables left little time to be trained in the new approaches.

For more information Read Rick's previous column on data warehousing. Find more data warehousing resources here

So, many data warehouses were not really built in the new data warehousing paradigm, which focuses on storing and setting up data for consumption. No one knew any better – not IT, business groups, consultants, or even some industry analysts. They, too, were struggling to "learn on the fly." And unfortunately, although many of these data warehouses were incorrectly designed, they continue to grow and expand. In fact, the very people who built them incorrectly are now "experts" who continue to build new data warehouses using bad techniques that focus on transmitting data. Without comprehensive data warehousing education, the first generation mistakes are being repeated in succeeding generations.

Getting the right kind of training is important. Technical, i.e. software product training, is not enough. Often, software vendors offer courses in addition to their product-specific training, or devote a portion of a class on BI or ETL tools to data warehousing concepts. Although this is useful, it is generally not detailed enough to really give people an understanding of what is involved in developing a data warehouse. And although the people teaching vendor-sponsored classes are quite knowledgeable in their tools, they may not have many years of data warehousing experience to draw upon in their teaching.

It's best to seek out data warehousing training that reviews concepts, architectures and best practices. This training should involve discussing data integration and data modeling, because that is the crux of the new data warehousing paradigm. There are also some very fine books that can teach you this information. But get books that have chapters on architecture, data integration and data modeling. Here are a couple of book suggestions:

• The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling by Ralph Kimball and Margy Ross
• The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning by Ralph Kimball and Joe Caserta

So how do you justify getting data warehousing training or "expensing" data warehousing books, especially if you have been involved in your company's data warehouse program? Pitch your training as necessary to support your company's performance management program, regulatory initiative (such as Sarbanes-Oxley) or effort to provide real-time data to the business. Your investment of time and expense in really understanding data warehousing will be abundantly rewarded when you improve your company's CPM, BI or DW initiatives in the future.

About the author
Rick Sherman has more than 18 years of business intelligence and data warehousing experience, having worked on more than 50 implementations as an independent consultant and as a director/practice leader at a Big Five accounting firm. He founded Athena IT Solutions, a Stow, Mass.-based business intelligence consulting firm. He can be reached at rsherman@athena-solutions.com.