Nmedia - Fotolia

Flexible MongoDB connects to Spark analytics, adds DBaaS

Speed to production underlies interest in MongoDB. At its annual confab, the company behind the NoSQL database rolled out improvements, including a Spark analytics link.

A growing desire to quickly build new applications has spawned broad interest in DevOps and, with that, flexible approaches to data that avoid relational schema. A top example is the open source MongoDB NoSQL database, which, together with JSON, has become a common means to build applications that are easier to change after their initial deployment.

Enterprises' use of MongoDB for such apps was on display last week at MongoDB World in New York City, where the database's eponymously named originator, MongoDB Inc., rolled out a connector to Spark analytics. The company also discussed its first database-as-a-service offering.

Speed to production was a major benefit of MongoDB, according to Gary Hoberman, executive vice president and CIO at insurance giant MetLife, based in New York. He told the MongoDB World attendees he and his team were able to demonstrate a customer relationship management system that looks like Facebook in two brief weeks in 2012, and launch into production in the span of just three months.

A flexible JSON schema worked perfectly with MongoDB, Hoberman said. MetLife's developers were able to accomplish things the 20-plus-year technology veteran said he had never seen before.

Also on hand was Oron Gill Haus, managing vice president for consumer bank engineering at Capital One, based in McLean, Va. He said MongoDB enabled the bank and credit card company to ''act quickly on data," as it built a visualization dashboard, based on MongoDB, that provides DevOps personnel an immediate view into software projects.

 "It enables our delivery teams to see what's working and what isn't working," he said.

Hooked on Spark analytics

Operations have been the first major use for NoSQL databases like MongoDB, with analytical uses of such databases still in early stages of maturity. With its new Spark connector, MongoDB Inc. more formally joins a club of NoSQL players -- including Basho Technologies, Couchbase, DataStax and others -- that have created integrations with the fast-rising Apache Spark analytics engine.

"The truth is that, with the exception of a fairly narrow range of applications, most analytics still require you to put data in another store for queries," said IDC analyst Carl Olofson. Spark links to NoSQL stores, such as MongoDB, can address this issue and reduce the need for data movement. According to Olofson, Spark showed promise as a potential standard analytical platform for what he described as near-real-time data.

MongoDB Spark connectors have been available before, but this one has received certified application status from Databricks, the company founded by Spark's inventors. The connector is written in Scala, according to Kelly Stirman, vice president for strategy and product marketing at MongoDB, based in New York. Scala is Spark's native language.

Cloud predicted

MongoDB's move to release MongoDB as a service -- one to be known as Atlas -- comes at a time when, in Stirman's words, many users, "would rather focus on building a winning application than focus on operating infrastructure." In his estimation, MongoDB and the cloud will only get closer to one another over time.

"We don't know where everyone is running MongoDB, but our best guess is that well over half of the deployments are in the cloud," he said. Atlas is offered on Amazon Web Services, although not all AWS cloud geographic regions are supported in the initial version, with support for Microsoft Azure and Google Cloud Platform to follow.

The company enters a MongoDB-on-the-cloud market that is far from a lonely place. Cloud providers Amazon, Google, IBM and Microsoft all support MongoDB on the cloud. As well, specialists, including MLab, Rackspace ObjectRocket and IBM company Compose, are already there.

Stirman knows "there are companies that do this today," but said his company will have advantages because of its familiarity "with tuning the software to run successfully."

MongoDB is ''coming out of the starting gate with something competitive,'' according to Tony Baer, principal analyst at London-based Ovum. Pricing for the service, which is a single-tenant-style cloud offering, will likely be most competitive at high, 100-GB-and-up levels," he said.

"I expect the rivals will respond in kind," he said.

Hi, I'm developer-friendly MongoDB

Much of MongoDB's present value, whether it is in open source or commercial clothing, is that, used in conjunction with JSON, it is malleable to developers' needs. Most especially, programmers do not have to work with rigidly fixed schema of the kind often associated with SQL systems. This leaves more room for changes as designs evolve.

"The MongoDB-style of NoSQL has proved a good fit with JSON development. From developers' perspective, JSON is representing data in the way it wants to be represented," Baer said.

"MongoDB is developer-friendly," he said. "Its biggest strength is its hold in the developer space."

IDC's Olofson concurred on this aspect of MongoDB. "You can make rapid changes and you don't have to go through the formal process that you have with relational systems," he added.

Olofson noted, however, while the company MongoDB may look to impress people that its software can be a foundation database for enterprise applications, its core use may be a bit narrower than that. Still, he indicated, these uses are important.

"It is especially useful for customer-facing, user-intimate applications," he said. "These are ones that need to change all the time. For those, you need a system that is able to keep up with change."

The need for speedy development is an important MongoDB driver, said Brian McNamara, principal at CloudyOps, a consultancy that focuses on automated infrastructure building. But, he emphasized, upfront design savvy still must accompany NoSQL database development.

"Everyone is looking to iterate as fast as possible, and tools like MongoDB allow you to be really flexible there," said McNamara, who led a session on running MongoDB with Kubernetes' containers at the MongoDB conference.

"But the popular notion that Mongo is 'schema-less,' is not entirely true," he said. "It's just that the responsibility now for how the data is used in applications has become the responsibility of the development team."

That kind of forethought is required for MongoDB in the cloud as well, McNamara said. "You are still responsible for architecting your database to be tolerant to different sorts of failure scenarios," he said.

Next Steps

Review MongoDB additions unveiled at MongoDB World 2015

Listen to a podcast that discusses apt uses of NoSQL databases

Watch a video that looks at SQL and NoSQL in context

Dig Deeper on Database management system (DBMS) software and technology