News Stay informed about the latest enterprise technology news and product updates.

Open source data integration tools good choice for 'standard' ETL jobs

Open source data integration tools and software could be the best choice for companies looking to keep costs low in standard extract, transform and load (ETL) scenarios.

When Balfour Beatty Construction needed data integration software to connect its ERP and project management systems, the company considered many of the usual commercial data integration vendors.

But, thanks to advice from colleagues at General Growth, decision makers at the Dallas-based construction firm also included open source data integration vendor Jitterbit in its evaluations.

Since 2006, General Growth, a real estate investment trust based in Chicago, had been using Jitterbit to integrate data from acquired companies into its JD Edwards ERP system, the same ERP system that Balfour Beatty used.

"Being open source, [General Growth] was willing to share their model for how they used Jitterbit," said Jason Bentley, director of business systems at Balfour Beatty. In addition to its low price compared with commercial products, the ability to draw on advice from a community of users, along with its functionality, sold Bentley on Jitterbit.

More on data integration tools and software
Find out how data integration tools could play an integral role in modernizing healthcare IT  

Learn how real-time data loading helps make operational business intelligence a reality  

Find out whether poor data integration license negotiation has cost your company money

As the economy continues on its downward slide, more and more companies like Balfour Beatty are considering open source IT technologies, including data integration software, to keep costs down, according to Ted Friedman, an analyst with Stamford, Conn.-based Gartner Inc. But open source data integration tools aren't right for every situation, Friedman warned.

"We definitely are seeing an increased interest in open source tools in the data integration space," he said. "But open source solutions are not nearly as well proven in large-scale, mission-critical implementations."

Friedman said most open source data integration software is geared to traditional extract, transform and load (ETL) methods. Other integration styles, including data federation (also called data virtualization and enterprise information integration) and real-time data integration methods like change data capture, are not well represented, at least at this point, in the open source community. Open source metadata management capabilities are also "lighter" than their commercial counterparts, Friedman said.

Nor are open source data integration tools cost-free. Customers still must pay for support and maintenance, he said, as well as for internal manpower to run and monitor the software.

Still, lower cost is a major driver of open source data integration software, as with other types of open source technology. Interest continues to grow in open source technologies of all kinds, Friedman said, as the recession continues to put pressure on IT departments to do more with fewer resources.

At Balfour Beatty Construction, however, "cost was an important criterion, but not a final factor" in the decision to go with open source data integration software, Bentley said. "We felt [that] the flexibility was definitely there that we needed [with Jitterbit]. And we knew there was a community there, should we have any questions."

Balfour Beatty uses Jitterbit for a number of data integration scenarios, but primarily to connect its JD Edwards EnterpriseOne ERP system with AutoDesk Constructware, project management software that subcontractors use on job sites. Before the two systems could communicate, Bentley said, workers often had to input billing information twice, once in each application.

After integrating the two with Jitterbit, which batch loads data from one system to the other every five minutes, data input in one application is now integrated with the other. And the integration software can also validate "cost codes" in one system against the other, to ensure that the data is accurate. "That was a change for us," Bentley said, "because we were pretty loose with cost code creation in the past."

Still, even though functionality and the assistance of the open source community were the driving factors in Balfour Beatty's decision to go with Jitterbit, cost did play a role, Bentley said. "Integration solutions are expensive, and we needed something that was affordable," he said. Balfour Beatty pays for maintenance and support services, plus an annual subscription fee, but did not have to buy licenses for the Jitterbit integration software.

Gartner's Friedman recommends that companies consider open source data integration software to reduce costs "when your requirements map well to the level of maturity of the open source solution," which, as he reiterated, are mostly geared to standard ETL.

"If it's a bulk and batch problem," he said, "and data sources and targets in question are pretty much industry standard, then go for it."

Dig Deeper on Extract transform load tools

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.