Guide to NoSQL databases: How they can help users meet big data needs
A comprehensive collection of articles, videos and more, hand-picked by our editors
New database technologies are coming to market with increasing regularity, and if these products live up to the hype as superfast and crazy cheap, hundreds of thousands of workhorse relational databases in use today will be put out to pasture. Who needs a 20th-century relational database when you can have a decidedly more modern NoSQL, columnar or in-memory database -- or even the Hadoop Distributed File System?
Most organizations, it turns out. At least for now.
While the seductive powers of the new database technology are not to be denied, you should resist the siren song of the new, post-relational database vendors. Not because the new database options lack merit -- on the contrary -- but because making your company's next database move a technology decision is the wrong way to go about it. The choice of database should be secondary. Your business goal -- that comes first.
A very good place to start
Consider a battery of practical questions about your project: Are you creating net new applications in support of net new business processes or merely upgrading the ones you already have? Engaging new types of users, data or analysis? Supporting a new line of business or reinvigorating an existing one? Answers to these questions will provide essential criteria for understanding which new database technology, if any, to deploy.
Only then should you look around to see whether a new database is better for the job than something you already have.
Implementing a database of any kind isn't cheap. While many of the new varieties are open source, they aren't free -- and even more costs enter the equation when a project involves migrating an existing relational database to one of the newbies. Myriad complexity issues also stand in the way.
New database technologies, particularly in-memory ones, often need new hardware. Many of the available options promise to lower total cost of ownership over time -- but new hardware will have to be obtained, and that up-front cost must be taken into consideration.
The fine print
Finding people with the right skills is an even bigger issue. The new models may require fewer administrators -- most proponents insist that their databases are less expensive to implement and manage than old-school relational databases are. And in many cases that's an easy argument to make: Top-tier database administrators are some of the highest-paid people in the IT department, and their numbers -- most relational databases are notorious for the number of admins required to keep them finely tuned -- clearly add significant costs.
But the likelihood of finding a Hadoop or columnar database expert in a traditional relational database shop is slim, which means you'll have to go out and hire these in-demand people or get the required skills from a consulting company.
And, as anyone who has worked to bring a major application project to fruition can attest, the bulk of the complexity is centered on everything but the cost of the software license. Creating new algorithms, analytical models, transactional components and business processes that need to be engineered and implemented is where the real expense is. Until they're well understood and the necessary stakeholder input and approvals have been obtained, the choice of database technology is at best a distraction. At worst, it's a great way to knock a project off its axis and send it spinning out of control.
This is particularly true in the era of big data, which is driving a considerable percentage of the new application projects in organizations. For many, big data projects involve data types that are new, unfamiliar and often unstructured -- time-series data, Web server logs, text. While some new database technology might eventually need to be deployed, figuring out what the new data sources are and what the new algorithms should look like must be the first order of business, right after you've reached agreement on what the new business processes are all about. To do otherwise is to march your company down the path of cost overruns, scope creep and eventual if not inevitable failure.
About the author:
Joshua Greenbaum is an independent industry analyst and founder of Enterprise Applications Consulting in Berkeley, Calif. Email him at firstname.lastname@example.org or follow him on Twitter: @josheac.
Take a quiz on the new database technology called NoSQL
New database technology adapting to cloud computing
Swedish bank banks on a new database technology: in-memory