Integrating and reporting against disparate data among 60+ school districts that use many different applications
and databases does not sound like an enviable job. But it’s the one Joe Fitzgerald’s got.
SIF agents are like driving to the store for a quart of milk with a Mack truck.
Joe Fitzgerald, Systems Integration Manager, LHRIC
Fitzgerald is a systems integration manager at the Lower Hudson Regional Information Center (LHRIC), a nonprofit consortium that provides IT services to 62 school districts spread across three counties in suburban New York City. Among Fitzgerald’s top tasks is facilitating data integration among and between each district’s student information systems, financial and human resources applications, and other data sources to support day-to-day operations and meet state and federal education reporting requirements.
Each school district’s student information system, for example, must be able to communicate and integrate data with food services applications so students that qualify for meal assistance get the benefits they’re due. The districts also must integrate data from multiple systems to create academic achievement reports as required by the No Child Left Behind law. Without proper reporting, the schools could lose funding.
Integrating data with SIF not an elementary task for school districts
Until recently, LHRIC relied on the Schools Interoperability Framework (SIF) to integrate data among each school district's systems, according to Fitzgerald. SIF is an XML-based, open data integration standard created in 1999 to allow elementary and secondary schools to pass data between previously siloed applications and databases.
The problem, Fitzgerald said, is that SIF is a complex standard that sometimes requires manual customization to meet the integration needs of the districts’ various systems. Each district averages around 10 different applications and databases, he said, and supporting data integration among them is putting significant pressure on LHRIC staff.
It was critical for LHRIC to find a simpler way to allow its districts’ different applications and databases to communicate in as near to real-time as possible -- both on an event-basis and in batch -- to ensure that students received the services they deserved and to maintain continuity in case of emergency, Fitzgerald said.
So in 2009, Fitzgerald began his search for a new data integration tool that fit the bill. During a six-month proof-of-concept, he evaluated three vendors -- open source Talend, data virtualization specialist Denodo, and Microsoft’s BizTalk.
Denodo proved to have some “reliability issues,” Fitzgerald said, while BizTalk’s “complexity was beyond the expertise that we wanted to develop here.”
Talend’s user interface, on the other hand, was intuitive enough for Fitzgerald’s staff to learn quickly, he said. And the tool simplified the job of automating many data integration jobs previously done manually -- just what Fitzgerald was looking for.
While cost was a factor -- Fitzgerald didn’t even bother evaluating IBM, for example, which he called “wildly expensive” -- Talend’s open source model and associated price structure was an important -- but not the most significant -- factor in his decision.
Open source data integration tool reducing errors, saving labor costs
Automating previously manual tasks is where Talend will help LHRIC the most, Fitzgerald said. In addition to improving data integration reliability and reducing man-made errors, it will also save LHRIC on labor costs associated with supporting SIF.
“In a K-through-12 environment, where teachers are getting laid off, saving the cost of labor is a big, big issue,” he said.
When deployed later this year, Talend’s open source software will help LHRIC with a number of data integration jobs. Among them, it will integrate data between the districts’ multiple student information systems and an eScholar data warehouse that LHRIC uses to report student achievement.
Talend will also automate and simplify data integration for the districts’ multiple finance, human resources, food services and transportation systems, leaving behind the complexity of SIF.
“We need to operate independently of the SIF architecture,” Fitzgerald said. “SIF agents are like driving to the store for a quart of milk with a Mack truck.”