Essential Guide

Browse Sections


This content is part of the Essential Guide: Trend watch: Data management and business intelligence technologies
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Is a second machine age on the way, driven by big data?

MIT's Erik Brynjolfsson and Andrew McAfee see elements that may betoken another machine age, including big data and advances in predictive analytics. And it might not be a mirage.

Science fiction writer Arthur C. Clarke's "third law of prediction" holds that the best technology is like magic. If so, that's both good and bad for IT and data managers, as well as business users: Good, because it truly describes the transformative effect technology can have, and bad because dressing up a new technology as magic is tempting for vendors. To the present: Thanks partly to big data technologies, we could be on the verge of a second machine age, with exponential change on par with the first machine age -- but it could also turn out to be an illusion.

Few technologies fall utterly flat, but many fail to justify their initial hype. Object-oriented databases, neural networks and artificial intelligence are just some examples of technologies that didn't live up their proponents' promises, though they ultimately did amount to something. Now, despite some success stories, the jury is still out on big data as a transforming technology.

Once you go beyond online advertising and marketing, successful big data use cases thin out. Some other killer applications may yet emerge. There are promising signs in agriculture, application management and other data-driven disciplines. But it hasn't quite happened yet.

As always, human factors are in play. Digitization is piling up data of all kinds at a prodigious rate. But the useful analysis of that data typically still relies on human judgment, and people have neither grown an extra brain nor exhibited giant leaps in IQ.

The technologist's answer is automation of intelligent operations, machine learning and predictive analytics. And the latter is a particularly charged topic -- foretelling the future has always been a challenge. What's different this time around for predictive analytics to find wider utility?

MIT professor predicts bright predictive future

That and other big data topics were much discussed at last month's MIT Sloan CIO Symposium in Cambridge, Massachusetts. According to Erik Brynjolfsson, an MIT professor and head of the MIT Center for Digital Business, predictive technology is improving at a significant clip. He thinks Internet-scale data is a key difference today. "We do a lot of predictive analytics in my group, especially using Google searches," he said. "I think it's like collective ESP."

Google searches are a window not so much into what people find, but what they're thinking of and what they want, Brynjolfsson said. He added that his group's work has shown you can use that information to predict how people will behave.

"For instance, we can predict with good accuracy what housing prices are going to be in the future," Brynjolfsson said. That is a prediction of a general trend, not what a specific individual will do, he acknowledged. But he cited recommendation engines and online advertising as predictive analytics success stories that are tailored to increasingly more narrow demographics.

"Whether it is who is going to click on an ad, or who is going to buy a book … that's working pretty well," Brynjolfsson said. He sees broader use of big data analytics in the future too, for example, to help predict who is going to get diabetes or other diseases. The data required for that is already accumulating, thanks to the use of loyalty cards that track food shopping preferences.

Notes on the new machine age

Brynjolfsson recently co-authored The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies, which outlines reasons why something new may be happening in milieus like predictive analytics. He and Andrew McAfee, also of MIT, wrote that incremental advances in both personal and business technologies may be adding up to great effect.

They said upcoming changes based on digital technology could be as momentous as those at the beginning of the steam engine era -- when James Watts' dedicated tinkering suddenly led to efficient steam engines that drove the Industrial Revolution forward, eventually leading to what came to be called the Machine Age in the late 19th and early 20th centuries.

The authors focused on several elements that they say indicate a new machine age is unfolding:

  • Digitization has given computers more and more to crunch on.
  • Exponential improvements like those seen in smartphone processing chips are creating more diverse data and more untethered clients.
  • Different technologies feeding on those advances are now being combined in forms where the value exceeds the sum of the parts.
  • Brynjolfsson and McAfee also cited the 2011 victory of IBM's Watson system on the TV game show Jeopardy!, the results of DARPA's autonomous vehicle challenge in 2004, the development of Apple's Siri personal assistant, and other examples as indicators that we're on course for a second machine age.

If you suggest to Brynjolfsson that effective big data and predictive analytics may be farther off than some software vendors imply, he's philosophical.

"We have come upon an inflection point to do that kind of thing -- whether it is Siri or Google Now or predictive analytics," he said. It's important to remember, he added, that flipping an on-off switch won't usher in the new era; it's continuous improvement in making small predictions that will help illuminate the way. That might not sound like magic, but it could become reality.

Jack Vaughan is SearchDataManagement's news and site editor. Email him at, and follow us on Twitter: @sDataManagement.

Next Steps

Learn about fast-rising cognitive computing -- its pros and cons

Catch up on your machine learning

Check out coverage of MIT's 2013 CIO Symposium

How DevOps may help to cope with changing ML models

Dig Deeper on Enterprise data architecture best practices

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

It seems premature to declare any technology's impact disappointing. To anyone, just now, any innovation can be disappointing compared with what can be imagined. But many otherwise humdrum technologies could ultimately give rise to innovative and life-changing advances. Technology evolves and in the process, new solutions may be found using adaptations or iterations of "old" technology.