In the second edition of the Data Management Book of Knowledge (DMBOK 2): “Data Architecture defines the blueprint for managing data assets by aligning with organizational strategy to establish strategic data requirements and designs to meet these requirements.”
Another way to look at it, according to Donna Burbank, Managing Director at Global Data Strategy:
“Data Architecture, in its broadest sense, asks, ‘What are we trying to do as a business?’ And then from all the diverse technologies ‘what’s the best fit for that purpose and how do they work together?”
Data Architecture Defined
Data Architecture is an offshoot of Enterprise Architecture, which looks across the entire enterprise, Burbank said. “Enterprise Architecture tends to look a bit more broadly at business and IT.” Business processes, business organizational structures, and business goals are important to the data architect, along with security and compliance.
At a closer level, Data Architecture also deals with decisions such as which platform is best based on business goals: moving to a Cloud-based solution or not, security risks with product decisions, and the choices such as the use of graph or relational database. “The architect role is unique in that they have to understand all of the new emerging tech to be able to graphically draw that out,” she said. Without a data architect, the organic growth that most companies struggle with is inherently chaotic.
“At many of the companies I visit, the first thing I do is draw a picture of their existing architecture and you’ll see the spaghetti diagram. Then when we’re done, there is a nice, clean Data Architecture.”
“What’s unique about data is that it’s partly a business role and partly a technology role,” Burbank said. A data architect might be locked in an IT-only role, focusing on the platform. Their purview is limited to decisions about specific tools, such as what type of server to use, or backup and recovery options.
“Those are all valuable, but I see that as less of a true architect, because a true architect, like a true Chief Data Officer, has to align with the business.”
There can be frustration between business and IT when the data needs of the business surpass IT’s ability to meet those needs. Burbank had a client who wanted a Data Lake and IT was too busy, so they outsourced it themselves. “Is business correctly skilled to do that? You don’t want business people trying to architect the solution,” because they don’t have all the information to make a good decision. “And you don’t want IT so focused on the technology that they lose sight of the business needs.”
Organizations that are holding out for a magical data architect that can do everything well, understands every platform, and everything about the business may be in for a long wait because, “They tend to not exist,” she said. Many companies are addressing this situation by creating a new data department, a hybrid that works with both business and IT, “Because either one in a vacuum doesn’t work well, and you do need both skillsets.”
Data Modeling Defined
The DMBOK 2 defines Data Modeling and Design as “the process of discovering, analyzing, representing and communicating data requirements in a precise form called the data model.” Data models depict and enable an organization to understand its data assets through core building blocks such as entities, relationships, and attributes. These represent the core concepts of the business such as customer, product, employee, and more.
Burbank defines Data Modeling as designing data from both the business and the technology perspective. “A data modeler might be great at modeling a specific system or a specific business case. But a data architect has to look more broadly.” Data Modeling typically focuses on the design of a specific database at the physical level, or a particular business area at the logical or conceptual level.
Aligning Data Architecture and Data Modeling with Organizational Processes Together
Data Architecture and Data Modeling should align with core businesses processes and activities of the organization, Burbank said. When the sales department, for example, wants to buy a new eCommerce platform, it needs to be integrated into the entire architecture. Without knowing what the existing data import and export processes are, it’s difficult to know whether the new platform will be a good fit. “And that’s often where a data model comes in. I’m trying to compare the data from two systems, how do we integrate? Is there data that’s missing?”
At a high level, a model entails developing simple business rules about what the business has: customers, products, parts, etc. “You don’t have to take months to do it—even just asking some simple questions and documenting can go a long way.”
Burbank worked with a customer who was about to implement a new system, and because they had previously had a bad experience, they decided to call everyone together to do a two-hour pre-implementation Data Modeling session. In the process, they found some significant errors. A few weeks later, they told her they were glad they’d taken the time to create a model because:
“They nipped a few issues in the bud that would have been a nightmare down the road. And they knew that because they had not done it the right way before,”
Burbank is working with a small nonprofit that has added a comparison of their data model as part of their governance around purchases. “These are not technical people at all, but now that they understand data, they ask every vendor that comes in to look at their data model” and compare it against their enterprise canonical model. “If more companies did that, there would be fewer issues, because once you start to implement and these systems don’t work, you realize it’s because the data isn’t aligned.”
Another customer had bought several systems but was unaware that they needed a rule specifying that a customer can have more than one email address. “It cost them thousands of dollars to clean that up. Had that been done proactively, they would have known that the system didn’t match their business rule.”
Yet another company was trying to implement an HR system and had staff filling multiple job roles, which required a data model rule stating, “The employee may have more than one role.” “This is Data Modeling 101, but their system didn’t employ that.” With that lesson learned, the company now checks every tool that comes in and compares it to their data model because, she said, “Their data model is how their business is run.”
Another much larger client now requires vendors to comply to a certain data import/export standard before purchasing software. Because they are a big company, they have purchasing power and vendors comply. With any architecture decision, the more understanding a company has of its systems, and the more willing it is to compare it to that common data model of your company, the more likely it is that integration will go smoothly, she said.
The Future: Back to the Basics and Then Some
Burbank said that she sees companies going back to basics, with no signs of that trend slowing down. Data Governance and Enterprise Architecture used to be considered “old school,” and companies who haven’t focused on it for a while have realized that it’s coming back, she said. “They’re dusting off old enterprise models because it works. It’s tried and true.”
The industry went through a “teenage phase,” Burbank said, “where we’re going to break all the rules and say ‘oh, we don’t need those silly data models anymore,’ but I think people are now coming full circle.” Data Governance was passé for a while and now it’s back in force because organizations realize the problems caused by not having it, she said. “A little bit of planning goes a long way.”
Business people are becoming more aware and involved, which she sees as a positive trend. She also welcomes some new additions, such as Design Thinking, and the merging of business and IT. “Some of these new ways of working are really lightening up modeling and bringing it to the masses. They’re working, active sessions and more tied to the business, which is a good thing.”
The environment is so much more complex, and the choices are so much more varied than in the past, she said. “The more complex things get, the simpler you need to make them.” Planning doesn’t have take months, and it helps for people to understand that systems talk to one another, so they can see what questions to answer. “Some of these very basic things that are so critical, like whether a sales rep has more than one region. You can just get together and quickly figure them out before you go running,” she said.
Due to the desire for new technologies, demand is increasing for Burbank’s help with basics like Data Governance, Data Architecture, and Data Modeling, “People want to get to the new stuff, and they realize they can’t get there without building the foundation first.”
Image used under license from Shutterstock.com