The world of data is changing at a rapid pace, and the most successful data management professionals will be those who manage to evolve with it, according to speakers at the recent TDWI conference in Chicago.
A rash of relatively new data management technologies like data warehouse appliances and columnar databases have already caused a major shift in the kinds of tasks that data management professionals perform on a daily basis, the speakers said. And with the advent of cloud computing, many traditional tasks, such as database tuning and indexing, may all but disappear eventually. In a world where computing power is treated much like a water or gas utility, how will data management professionals survive?
We need to be solving business kinds of puzzles. But the fun is applying the technology and the data to solve those puzzles.
Laura Reeves, co-founder and principal, StarSoft Consulting Inc.
According to TDWI conference keynote speaker Laura Reeves, the fate of data management professionals lies largely in their ability to become business-savvy problem solvers -- the type of people who know how to use data and technology to address "big picture" issues.
SearchDataManagement.com got on the phone with Reeves -- a longtime data warehousing consultant and the co-founder and principal of Naperville, Ill.-based StarSoft Solutions Inc. -- to find out just how the world of data management has changed, how it might change in the future, and what data management professionals can do to remain marketable in the long term. Here are some excerpts from that conversation:
What are some of the biggest changes affecting data management professionals today?
Laura Reeves: We have seen huge changes in the kinds of hardware, software and technologies that are available to us [and] the speed that things are changing is obviously increasing. Also, the dynamics of what [we do are changing]. If you're purchasing, [for example, a data warehouse] appliance, then you're no longer doing the table design and the level of management, and what you need to do changes. [That's] the kind of shift that we're seeing right now. And frankly there is a whole range of [job functions being practiced today]. I might be building star schemas. I might be using a columnar database. I might be using cubes. I might be using a warehouse appliance. I might be using Teradata, which is, again, a different kind of a platform. We already have a whole range of technologies that are all completely different in terms of what you have to do on the physical data management side of things.
How should organizations deal with all of this? What does it mean for them?
Reeves: There are all these things going on. Everyone is pressuring you to deliver more quickly with fewer resources. With all of that pressure and with everything we're looking at, the first thing I think people need to do is take a short timeout and say: How are we doing? As part of [that], I suggest that people go out and go talk to all of the business areas. Find out what is working. Are they satisfied? What is not working? What do they like? How is IT helping? Where do we still need to shore things up? Look at your staff. See who you've got and what are they doing. The biggest thing with all that is that you have to listen to what people say. Don't get defensive. [I also] think you need to get an inventory of the pieces and parts of what you have. How many data marts? How many databases? Where are they? What are they doing? What are the technologies? What are all the things that you have? I think that will give you a pretty good idea of where you are.
After checking the pulse of the business and taking inventory, what's the next step?
Reeves: Then you've got to figure out what do we want to do and where do we want to go. What things can we do that will really make a difference regardless of the changes in technology? When you boil it all down, regardless of what technology is doing, a lot of this is dealing with people -- both the business community and your team -- as well as data. Sometimes I still see people in a data management kind of a hat or role being quite technical -- tuning databases and partitioning and things like that. A lot of [those tasks] may go away if these predictions come true. [What I want people] to think about is that you need to practice reading, writing, communication skills, problem solving, creativity, that kind of thing. Those are the skills that we're going to need as professionals going forward and that will prepare you for whatever [the] world will look like in five or 10 years.
What else should data management professionals focus on moving forward?
Reeves: We need to get more into a problem solving, understand-the-business kind of arena. [We] need to partner with the business community, and we have to learn how to talk to them in business terms. If we say that loading [a particular] data source is going to take us three months longer and we'll miss our deadline, [the] business users may say, "OK." But they may not realize that [missing the deadline] may take away the ability to do three critical calculations that help them understand their business impact. So, you have to talk in those business terms. Talk with the business [and] understand their broad requirements. [There has] been research done in the past that shows that executive sponsorship and partnering with the business community are some of the most critical factors for success of a data warehouse. Yet, if you look at the tasks on a [typical] project plan, very few are specifically geared toward cultivating and maintaining that kind of a relationship.
How do you think data management professionals should evolve in terms of handling the actual data?
Reeves: If you think about the news industry, we used to get the daily newspaper. You had reporters and editors and people who were responsible for vetting and making sure that it was accurate. Then we got morning and evening news. Cable news [came along around 1980] and that was 24/7 but still, [those news sources] were still vetted and accountable sources of information. Now we see with the Internet and all the handhelds and things and social media, you can get things from everywhere [and you often] really can't tell if it is opinion or not. I liken that to the explosion of data. As data management professionals, we need to be able to help differentiate factual information from opinion. Regardless of where the data lands, we have to provide that trusted, vetted source of information to the company. [What] is the data? What does it mean? How should it be used? [Those] are the things that are critical long term regardless of all this other technology turmoil.
What is one more piece of advice that data management professionals should think about as they prepare for the future?
Reeves: We need to be solving business kinds of puzzles. But the fun is applying the technology and the data to solve those puzzles. That's what we need more people to be able to do, especially as the [world] continues to change. The people that are going to be the most successful long term are the people who can begin to develop and grow those soft skills.