Data warehouse performance killers -- and how to avoid them

In a podcast Q&A, consultant William McKnight offers advice on steering clear of common hazards that can puncture the performance of data warehouses.

A well-designed data warehouse system can help boost the corporate bottom line by ensuring that business intelligence data gets delivered to business users on a timely basis. But if you aren't careful, data warehouse performance can bog down in "a lot of different places," according to William McKnight, president of McKnight Consulting Group in Plano, Texas. For example, McKnight pointed to data modeling and system architecture shortcomings as potential drags on performance, during a podcast interview with SearchDataManagement.

In addition, many IT managers don't take advantage of new technologies that might ease performance bottlenecks in their data warehouses, McKnight said. He cited tools such as columnar databases, solid-state drives, data virtualization software and Hadoop systems. Hadoop, in particular, can be useful in big data environments involving unstructured or semi-structured data that isn't a good fit for conventional databases: "If we've been trying to push that into the data warehouse, we've struggled," he said.

Throwing money at data warehouse performance problems isn't necessarily the answer, McKnight said. "I like to say it's more a matter of smart money you spend. This isn't investment for investment's sake." But he added that in many cases, companies err on the side of not doing enough to address throughput issues. "A lot of shops will do the lowest-denominator kind of thing to try to take [their systems] up one inch of performance, when there are bigger things out there," he said. "What folks should do to improve performance problems is to really look at the bigger picture while they're looking at the smaller picture."

In the 10-minute podcast, McKnight further discussed the performance issues that can affect data warehouse systems and how to sidestep or mitigate them. Listeners will:

  • Learn about some of the major factors that can slow down a data warehouse environment.
  • Hear McKnight's take on whether the growing focus on big data ratchets up the potential for performance problems in data warehouses.
  • Get a list of steps that IT and data warehouse managers can take to avoid performance potholes.
  • Find out whether the solution to data warehouse performance problems usually involves adding more technology to better handle the warehousing workload.

Dig Deeper on Data warehouse project management