Home > Data management / BI News > Text analytics, search bolster business intelligence software stack, says TDWI
Data management / BI News:
EMAIL THIS

Text analytics, search bolster business intelligence software stack, says TDWI

By Hannah Smalltree, News Writer
19 Apr 2007 | SearchDataManagement.com

News on data management trends and technology
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google

Two unstructured data technologies can seriously bolster business intelligence software stacks -- and one can be a "no-brainer" to implement, according to an analyst.

For more on business intelligence, unstructured data and search
Learn how business intelligence vendors are addressing search

Read five myths about business intelligence and search
Text analytics and search have both been around for a while, but both were recently examined in the context of BI in a study and survey of 370 data management professionals by The Data Warehousing Institute (TDWI), based in Renton, Wash. The two technologies are different but related because they're typically applied to unstructured data, such as that found in documents, emails and text fields, explained Philip Russom, senior manager of research and services with TDWI and author of BI Search and Text Analytics: New Additions to the BI Technology Stack, published in the second quarter of this year. Companies are increasingly integrating these technologies with BI software, he said, albeit in very different parts of the stack.

"Text analytics is a very deep technology piece that end users never see, whereas BI search is something every report consumer can see and use," Russom said.

Business intelligence search increases productivity and self-service

On the front end, BI search enables productivity and efficiency gains, by helping end users find reports and information faster, Russom said. Some companies seem to agree, with 62% of TDWI's survey respondents rating the perceived business value of BI search as high or very high. Though only 9% of respondents had deployed BI search technology, 15% are in the design or implementation phase, and 40% are in the exploration phase. Russom is "amazed" that more companies haven't implemented BI search.

"Search capabilities are now built into most BI platforms. [Many] have switches -- you just turn on the switch and it enables search capabilities for their BI platform," Russom said. "If it's that easy to implement, just go ahead and do it. In fact, with many licenses, you don't actually pay extra money for it."

Companies that haven't implemented BI search may be strapped for time or may be dealing with competing priorities, he said. But he thinks organizations might move BI search to the top of the queue if they considered its potential long-term benefits, such as reducing end user requests for new reports or tech support.

"BI search enables a wide variety of self-service capabilities that offload work from IT to the report-consumers," Russom said.

Text analytics increases knowledge and insight

Organizations can gain valuable insight by extracting and analyzing information from unstructured data, Russom said. In the BI context, text analytics is on the back end, part of the underlying data integration infrastructure, he explained. It can be complex to implement, in part because of the development work required and the volumes of unstructured data at most companies -- though the volumes may not be as big as previously thought, he found.

An oft-cited statistic is that, on average, unstructured information makes up 80% of the data in an enterprise, Russom said, but this TDWI survey indicated that 31% of enterprise data is unstructured, 22% is semi-structured (as in XML, etc.) and 47% is structured. Though it's a change from previous figures, it doesn't change the importance of unstructured data to BI, he said. A company's view of corporate performance is incomplete without the contextual information that's provided by unstructured and semi-structured data.

One example of this was an insurance company Russom surveyed that uses text analytics to examine call center records. Call center representatives entered a status code (structured) for each call, which helped the company measure metrics such as customer satisfaction. To gain insight beyond the codes, the organization used text analytics to look at the comments field (unstructured text). The company found that customer satisfaction metrics looked very different depending on whether they considered the status code alone or combined that information with data extracted from the text comments field.

"In the text, you could find things about the customer that the structured data from that call did not capture. So they were actually able to create some new metrics that had to do with softer quantification of customer satisfaction," Russom said.

Some companies seem to recognize the importance of using text analytics in BI, the survey found. It said that 51% of respondents rated the business value of text analytics as high or very high. As for usage, 6% reported having text analytics already deployed, while 7% are in the design or implementation phase, and 43% are in the exploration phase. Russom was not that surprised by the low implementation numbers.

"Text analytics is a full-blown development project. It is not the kind of thing that you can install and be using in a week, like BI search. Text analytics takes a lot of design time before you even select a tool. And, because of the development work, it involves a lot of payroll costs," he said.

The costs and complexities of text analytics can be worth it, he said. Early adopters have used it for warranty analysis, fraud detection and call center analysis, but many companies can benefit.

"If you don't have data in your warehouse representing information that originates from unstructured sources, your data warehouse may be a single version of the truth," Russom said. "But it's not a complete version of the truth."

Tags: Business intelligence technology platformText analytics software and text miningBusiness intelligence and searchBusiness intelligence searchEnterprise content managementEnterprise search softwareVIEW ALL TAGS

Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google



RELATED CONTENT
Business intelligence technology platform
BI project management tips for implementation success
Oracle, SAP and IBM top Forrester's corporate performance management rankings
Microsoft details self-service business intelligence, data warehouse releases
Birst takes SaaS BI out of the cloud, battles data security fears
Hurdles for SaaS BI vendors include data integration, low recognition
IBM launches private analytics cloud
Atlanta YMCA turns to SaaS BI software over 'complicated' Cognos
Choosing BI software: Use your ERP vendor or go with third-party BI?
Data integration for Software as a Service business intelligence software evolves
SAP inks another partnership to enhance NetWeaver BW

Text analytics software and text mining
Text analytics market small but growing amid recession, according to IDC
Business intelligence software market looks to hold its own during recession
Calculating text analytics ROI: Start small and focus on customer data
Text mining tools and prediction markets extract insight from unstructured data
Unlocking and integrating unstructured data, with Bill Inmon
IBM brings enterprise search, business intelligence to Super Bowl coverage
Data management's top 10 stories of 2007
Business Objects adds text analytics to BI with Inxight acquisition
Enterprise search platforms: Seven evaluation tips
IBM's free OmniFind Yahoo edition: In search of "the catch"

Business intelligence and search
Enterprise search and business intelligence: What's in store
Enterprise search software enables AMR's analysis
Business intelligence search: Five myths
BI vendors search for new meaning

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
corporate performance management  (SearchDataManagement.com)

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary



Data Management: Business Intelligence, Data Integration, Data Compliance