I’m working on a project where we have to load over 700GB of raw data into a Data Warehouse to provide querying and analytics to the business unit. The choice was made to build this DW on an SQL Server 2005 3-node cluster with a 6TB network attached SAN. (The excess disk space was purchased in order to support ~1GB of nightly loads and 3 years of growth. I’ll cover more on that later.)
Over the next few posts I’ll discuss what I’ve learned through the challenges I’ve faced in this project. I think the most important pieces of the project can be broken out into these 5 topics:
- Project scope, deliverables & documentation
- Hardware
- SQL Server 2005 configuration
- Designing our ETL
- Lessons learned
I hope that through my experiences (and mistakes) another team could save time on their project deadline. Because, hey, nobody likes working on weekends and losing time w/family and friends.
- Reagan