Catalina Marketing Corp. casts a big shadow behind the scenes of the retail industry. The St. Petersburg, Fla.-based company collects and crunches huge amounts of point-of-sale data, and its systems are used in thousands of supermarkets, drugstores and mass merchandise stores in the U.S. and abroad to do real-time analytics on purchases and target coupon offers to shoppers in an effort to drive additional sales.
In recent years, Catalina has built its business around more than 30 data warehouse appliances, which sit at the core of what might be the largest real-time data network in the world -- one that processes information on 260 million transactions each week. Eric Williams, until last month Catalina's executive vice president and chief information officer, said in an interview before retiring from the company that the optimized combination of hardware and software built into appliances has been ideal for Catalina from both a performance and cost perspective.
More on data warehouse architectures and appliances
Get advice from TDWI conference speakers on overcoming data warehousing challenges
Read about a data warehouse appliance deployment at the Foxwoods casino in Connecticut
Listen to a podcast Q&A on the pros and cons of hub-and-spoke data warehouse designs
For a growing number of other organizations, the logic of using appliances as part of their data warehouse architectures has also become compelling. The devices, which bundle together data warehouse software and hardware in preconfigured packages, have moved deeper into the business mainstream because of their ease of deployment, potential cost advantages and ability to quickly scale out to support business and data growth. In SearchDataManagement.com’s 2011 online reader survey, for example, 42% of the 340 respondents with data warehouses in place or planned within the next 12 months said they were using or planning to deploy data warehouse appliances.
Williams said that in the 1990s, Catalina built its own appliances using proprietary software and commodity hardware. But when the first data warehouse appliance vendors started entering the market, the company changed its strategy and decided to buy systems from an external supplier while also sharing its considerable development experience with the chosen vendor.
In addition to the performance and cost benefits that Catalina has seen from using data warehouse appliances, Williams noted that instead of having to endure rounds of finger-pointing between different software and hardware vendors when technical problems arise, “there is just one throat to choke” with appliances.
Randy Mattran, vice president of professional services at Lancet Software, a business intelligence (BI) consulting and services firm in Burnsville, Minn., said he’s seeing widespread interest in and adoption of data warehouse appliances among his clients in the retail, manufacturing and health care industries.
Addition by addition to data warehouse architectures
Mattran added that the organizations he works with are mostly installing appliances to supplement existing data warehouse platforms. One reason for that, according to Mattran, is that appliances often are special-purpose machines that rely on having a well-structured and reliable operational data store (ODS) behind them, acting to consolidate and integrate enterprise data from multiple source systems. The ODS feeds data to the appliances, which in turn deliver the information to end users for use in BI and analytics applications. That division of responsibility “offloads the ODS and spares it from the pressures of user-driven analytical queries,” he said.
Analysts who focus on data warehousing and other data management technologies also point to growing adoption of appliances -- like Mattran did, primarily in a supplementary role.
The thing with appliances is that once you get them in, you’re kind of stuck. They’re hard to get out of because it’s a complete package.
Julie Lockner, analyst, Enterprise Strategy Group
“What is playing out is accelerating acceptance -- it is a well-understood principle and people have been buying [appliances] and having success,” said Gartner Inc. analyst Merv Adrian. Indeed, some appliance vendors have doubled their sales of the devices over the past 18 months, according to Adrian.
However, Julie Lockner, an analyst at Enterprise Strategy Group in Milford, Mass., cited data from a survey that ESG conducted last July that shows data warehouse appliances being adopted somewhat selectively by organizations.
Lockner said the survey results, which were released as part of a report on “big data” and data analytics that was published last September, highlight the importance that companies attach to enhancing their data analytics capabilities and processes. Six percent of the 270 respondents ranked that as their most important IT priority over the next 12 to 18 months, while 45% said it was one of their top five IT priorities.
Data warehouse appliances have a role to play in helping to achieve that goal, but not necessarily the leading one for many of the respondents to the ESG survey. Only 6% said they were currently using workload-specific appliances as their primary data analytics systems. Out of 102 respondents whose organizations planned to deploy new analytics systems within 12 to 18 months, 21% said appliances were part of their plans. But four other technologies got higher response rates: custom-developed systems at 45%, cloud-based analytics services at 35%, massively parallel databases at 29% and general-purpose databases tuned for specific workloads at 28%.
Lockner thinks data warehouse appliances are here to stay, but she added a note of caution: Be sure they’re right for your organization before you commit to a purchase and build them into your data warehouse architecture. “The thing with appliances is that once you get them in, you’re kind of stuck,” she said. “They’re hard to get out of because it’s a complete package.”
Alan R. Earls is a Boston-area freelance writer focused on business and technology.