While the importance of workload management (WLM) has grown in recent years, not everyone thinks the data warehouse vendors' WLM
"Workload management is kind of a mixed topic," said Silvio Schurig, a consultant at Switzerland-based Datelligence GmbH, in an email interview. "I don't think tools are doing a very good job."
Schurig, who has worked on numerous data warehouse implementations, said the WLM tools he has worked with have proved somewhat inflexible. He would like to see data warehouse vendors make their WLM tools easier to manipulate and require less coding.
"There is no single tool to manage and handle workloads in a scalable way that can be -- at least to some degree -- parameterized without massive programming/scripting effort," Schurig said. "At least, not that I'm aware."
Vendors like Netezza beg to differ, however. Netezza's data warehouse appliance includes WLM capabilities that DBAs can interact with via an intuitive graphical user interface, according to Phil Francisco, vice president of marketing at the Marlborough, Mass.-based vendor. DBAs can set business rules and define user groups in as little as 10 minutes, he said.
But some of the smaller, newer data warehouse vendors do indeed lack the sophisticated WLM capabilities of their larger, more established competitors, according to Merv Adrian, a data warehouse analyst and consultant with IT Market Strategy. Instead, they rely on raw speed to win customers.
"Emerging analytic DBMS vendors typically prove themselves in proof of concept trials that are focused on running one or two jobs and comparing those to the incumbents doing the same thing," Adrian said. "And they often win handily -- but that's not about workload management."
"The more users you bring on the system, the more conflict you'll have."
Randy Lea, Vice President of Products and Services Marketing, Teradata
To a degree, though, it's understandable that the younger data warehouse vendors often don't measure up when it comes to WLM. "They have not been in place long enough to have accumulated rich, complex streams of concurrent work," Adrian said.
But that doesn't mean they won't have to develop strong WLM capabilities eventually if they want to compete in the hot data warehouse market.
"Even smaller data warehouses often see substantial amounts of concurrent work: large reports that are running while ad hoc queries are being submitted, along with backup tasks," Adrian said. "When these tasks start to pile up, the picture can be different."
Workload management critical to data warehouse ROI
Improving data warehouse performance is the practical goal, of course, but optimal WLM is also the key to getting your money's worth from a data warehouse investment, according to Jim Kobielus, an analyst with Cambridge, Mass.-based Forrester Research.
The simpler, more effective the WLM tools, for example, the fewer DBAs will be needed to run the data warehouse. Fewer DBAs mean lower salary costs and a quicker return on investment (ROI) from your data warehouse, Kobielus explained.
"WLM is what DBAs use day in and day out to keep this thing humming," he said. "That's the secret behind getting a near-term return on your data warehouse investments."
Randy Lea, head of products and services marketing at Teradata, agreed. WLM helps you get the most performance out of your data warehouse, he said, and the more performance you can "squeeze out of the box," the longer you can delay costly upgrades.
Lea said most Teradata customers are running their data warehouses at between 85% and 100% of capacity thanks to the vendor's WLM capabilities. "This gives our customers a huge advantage," he said.
Gartner analyst Donald Feinberg puts it this way: "It's not just about who can do my query the fastest or load my data the fastest, but who can give me my complete workload with least investment."
Workload management presents political challenges
Though the technology side of WLM is pretty straightforward, actually coming to decisions about the business rules that WLM will observe is not.
Whose queries take precedence in any given situation? How much CPU will the sales department be allotted? When will large batch jobs run? All of these questions need to be answered before WLM parameters and rules are implemented.
"It really is a business decision on how to best utilize the resources of your environment," Lea said. "The more users you bring on the system, the more conflict you'll have."
"It can get a little political," he added.
Companies considering their data warehouse options should also understand that WLM is not a one-time job that takes place at the beginning of a deployment, said Netezza's Francisco.
Inevitably more users and more tasks will be heaped onto the warehouse, he said, requiring constant WLM monitoring and tweaking.
"It's a job that's never done," Francisco said. "You can always make it better."