kentoh - Fotolia

Systems of record take back seat to systems of engagement, innovation

Systems of engagement represent a hotbed of activity in data management these days. Flexibility and scalability are watchwords.

When your hotel or Airbnb reservation is booked and paid for, and it goes into a system of record, that's a bit...

of technology magic, yes. But that record-level transaction is not the wonder it once was. Today, the magic is in systems of engagement.

Such systems, ones that choreograph a complicated web-based dance of seller and buyer, are the center of attention today.

Systems of engagement must be able to work at large scale and change features flexibly. Recommendation engines, special offers and mobile apps are among examples of today's systems of engagement. Underlying these systems are data management tools for diverse data types, fast data processing and novel analytics.

Reimagining business

IDC analyst Carl Olofson at the firm's Directions 2017 conference in Boston last month outlined striking data management changes that accompany the push toward systems of engagement. Expect more change, he counseled.

"People are reimagining business processes, and our systems have to be flexible enough to deal with that," he said. "The idea is to move from systems of record to systems of engagement."

Such systems need to scale big time, handling millions of users at breakneck speed. They must also be flexible. 

Flexibility is required because data has many more dimensions than it once had, and system designs have to turn on a dime. These needs, Olofson said, have helped spawn the greater variety of databases and associated components that data managers now encounter.

Olofson said he sees emerging graph databases, key-value stores, document databases, persistent stream managers and more combining to create a scalable data platform that underpins the growing system of engagement and, importantly, links it to the system of record.

This all foretells a significant shift toward operations-oriented business intelligence, or analytical transaction processing, he said.

The wide assortment of data tools -- streaming engines, key-value stores and the like -- have now arisen to engage the customer and make relevant sales pitches in something like real time.

Analytics meet operations

The emergence of these new-style systems has been noted before. The systems of engagement differ in details, but carry a common theme:

  • Hadoop clusters feed recommendation engines that suggest relevant products to retail shoppers in 40 to 60 milliseconds.
  • A key-value NoSQL database delivers low latency for quickly shifting users from static webpages to interactive dialogs.
  • Hadoop and NoSQL technology come together in a big data platform for retail behavior analytics intended to predict future customer behavior and move some work from batch to real-time operations.

Even when analytics go through human hands -- when they are worked on by in-house data scientists -- they are pushing closer toward real-time response.

Employees, after all, come to expect the same throughput at work that they have with their cellphone. Again, what comes into play are combinations of Spark and Hadoop, as well as NoSQL and other scalable data platform components.

Tapping into NoSQL

A shadow army has arisen along with systems of engagement today. The army comprises users armed with smartphones. As the user army has grown, so has use of flexible, built-for-purpose NoSQL databases. Speaking with a witness to the mobile and NoSQL movements gives a view into the foundations of the systems of engagement.

Gary O'Connor saw NoSQL databases grow up along with e-commerce at Betfair, the U.K. bookmaking company that has become an international online wagering giant with massive transactional volume.

He also saw this at the BBC, where content management, streaming on demand and video analytics came to rely on NoSQL.

Now as CTO at Doddle Parcel Services Ltd., he has tapped into NoSQL again -- in this case, with Couchbase Server and its associated mobile development tools. By focusing on the mobile front end and working inward, Doddle can more quickly expand its business -- a network of click-and-collect stores intended to facilitate ready pickups for online sales, he indicated.

"We looked at what we had to do in the parcel stores, and a document database made sense," he said.

The Couchbase software enabled quick changes, he said, allowing Doddle to add new elements to records without first changing overall database design. The system's agility drives broader business agility.

"Our partners don't want to have to make all their decisions immediately. Now, they can make a few key decisions, start moving forward, and then make changes later," O'Connor said.

Clearly, beginning -- and adding as you grow -- is yet another trait of today's systems of engagement, a trait of which data managers are becoming acutely aware.

It would seem as though systems of record have become table stakes. They come with many moving parts to learn and build, but the systems of engagement have, bit by bit, become today's true hotbed of activity.

Next Steps

Discover the role of DevOps in data engineering

Learn about systems infrastructure for deep learning

Find out how big data pipelines are changing Hadoop architecture

Dig Deeper on Hadoop framework