Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Build vs. buy equation changes, as open source big data tools surge

Listen to this podcast

The build vs. buy decision is a fluid one, as open source big data tools proliferate. This edition of the Talking Data podcast looks at the new quandary.

Build vs. buy is a decision that has long been intrinsic to the mission of the IT leader. In the age of big data and open source software tooling, the familiar dilemma has taken a different tone, according to reporters speaking in the latest edition of the Talking Data podcast.

Important elements in making build vs. buy decisions are the total cost of ownership of software over time, the competitive benefit to be gained by adding new features quickly and the skill sets available within the organization for any given new technology initiative.

Mature markets are often marked by packaged software that is bought, while new markets tend toward innovative software that is built. But, in practice, the choices are seldom so black and white.

Big data analytics software, for now, is a place where the elements in the build vs. buy decision are still very fluid. Today's open source technologies, such as Hadoop and Spark, were spawned by web upstarts like Facebook, Google and Yahoo -- all of which showed a heavy commitment to working quickly to build in-house and, in many cases, placing the software tools in the open source domain, where others could contribute to bug catching and feature enhancements. But these companies' paths are not a direct fit for the typical software shop.

Sometimes, buy remains the favored choice. As an example, the podcasters discussed a recent SearchDataManagement article about a music site pursuing a recommendation engine project. In that case, the lead engineer opted to buy an end-to-end Hadoop platform that already included well-honed templates for recommendation engines. The thinking was engineering staff energies could be better used if assigned to other aspects of the project.

A new wrinkle on build vs. buy is cloud computing, which alters the equation for decisions: build vs. buy vs. rent. The latter was the path chosen, according to the podcasters, in the case of a software test services company that opted for a cloud-based, managed data warehouse service.

Like the engineering head at the music site, the technology leader behind this decision was confident his team could have built and run its own data warehouse. But the motto he followed here was: "Just because you can do it, doesn't mean you should."

Listen to the Talking Data podcast and find out more about the changing world of build vs. buy decisions in big data and the cloud.

Next Steps

Read more about build vs. buy choices for digital transformation

Watch a video review of recent software evolution

Listen to a podcast on big data ethics