Businesses face lightning-fast and massive data growth “at a time when analytical capabilities cannot even come close to meeting them,” said Dr. Peter Aiken at his recent Data-Ed webinar, “Conceptual vs. Logical vs. Physical Data Modeling.” An acknowledged Data Management authority and president of DAMA International, Aiken explained that a lack of understanding and documentation of a company’s Data Architecture – coupled with data structures that need updating – fail to return useful information.
Instead, many enterprises have cobbled systems that don’t speak a common data structure language and require development gymnastics to get meaningful data through an exponential number of system and user interfaces. This results in organizations spending 80% of their budget on improving existing architectures, Aiken noted.
Getting meaningful information requires data structures organized in a good framework. Using a three-dimensional evolutional Data Modeling approach to reverse engineer existing systems, update requirements, and forward engineer with the new requirements promises an excellent approach.
During the webinar, Aiken discussed this three-dimensional evolutionary data approach, what it is, and how to use it.
Why Consider a Data Modeling Approach at All?
By default, a Data Architecture has some sort of data model, whether understood or documented, because it has at least a physical data structure to get data from a system to a user. According to Aiken, most organizations get stuck in this physical model, starting with a flawed architecture and trying to fix the technical structure without “a single thought” about what the Data Architecture does or why.
So, as many companies reverse engineer existing systems without understanding the existing systems, their strengths, and weaknesses, they risk passing along flawed data structures or failing to re-create data structures that work. The initial data structure problem gets passed along through all the different fixes. Aiken said:
“A serious product defect can haunt it throughout its life cycle. Several companies have had these ‘oopsies,’ starting with a flawed data model. Consequently, this approach locks in these imperfections and restricts data investment benefits in the future. “
The organization has inadequate consumable data from migrating, converting, and improving the existing system. This lack means updating existing data structures takes longer and costs more to deliver valuable information.
To break this process, companies need to model their data to solve a specific business problem and to have a shared understanding between business and IT. Yes, organizations need a physical model for their technical people to build; however, companies also need models with the same common vocabulary between the business stakeholders and the technical people building the data platform to get meaningful information.
Why Consider Three Data Models: Conceptual, Logical, and Physical?
Aiken acknowledged that Data Modeling needs to be done in components to have a data structure connecting the data’s value to the information required by the business activity or situation. He notes that data and information must be considered when a consumer requests the data to do something.
Organizations create each data model to answer a specific business question in this process: the needed information. One data model that tries to represent all the business’s Data Architecture ends up unwieldy and unusable, as the figure below shows:
Instead, think of data models and Data Architecture as developed for specific needs based on the problem to solve. Since business needs are evolving, consider data models as iterative to better achieve a particular goal.
To get to a model’s purpose, some types are better suited. While a physical model captures a data solution as built, companies need to know how to make this solution and what they are fundamentally building for the business. A logical data model responds to how to build it, and a conceptual model describes what needs to be made to solve the business problem or case.
Why Add Dimensions to Three Types of Data Models?
Typically, companies do not start from scratch upon modeling a business problem because data systems exist. So, IT takes the available Data Architecture and makes changes to it, necessitating an alteration in the data models.
Suppose developers and engineers move forward with updating and creating a new architecture. In that case, IT assumes that they have validated what exists and that everyone working on the system has that understanding.
So, this reasoning explains why many companies end up reverse engineering their platforms. They want to get to a place from the not validated or understood existing system to the validated, understood place.
However, how does IT know what exists and meets the business needs? What if the requirements changed because the business evolved since the data solution updates?
According to Aiken, the people building or updating the system need to know about any new business requirements and what has changed. This requires chatting and aligning with the businesspeople and stakeholders who need the information from the data for their work.
Consequently, organizations need to understand what already exists and know what will be necessary. They need to reverse engineer as is and build the data architecture according to what it needs to be. See the diagram below:
A Three-Dimensional Model Evolution Framework
Adding in the validation status of the data model provides a third dimension for Aiken. See the image below:
Each data model – conceptual, logical, and physical – joins with one another to form a whole Data Architecture component. However, each model type has a different purpose and perspective on business needs and gets used differently.
The Conceptual Data Model – The Data Requirements Needed to Do Business
In specifying what is to be, the conceptual data model provides “the focus and scope,” said Aiken. It organizes the existing data concepts, analyzes them with the organization’s strategy, notes tradeoffs due to technical strengths and weaknesses, and builds future capabilities.
Most importantly, the conceptual data model harmonizes vocabulary between technical staff, less technical businesspeople, and among systems. With stakeholders and engineers at the table, Aiken suggests the following:
- Identify the entities
- Identify the key for each entity
- Draft the connections between the entities
- Identify data characteristics
- Map these data attributes to the entities
Through this process, Aiken advises, the conceptual model will evolve slightly. If it starts changing radically, consider a different way to view the context, such as describing different user views through a logical model and then returning to the conceptual model.
By laying out the architecture conceptually, participants agree on what the entities mean, harmonizing their vocabulary and communications. These discussions form the basis of an enterprise taxonomy with meaningful business definitions and valid entities required by business.
The Logical Data Model – How to Meet Business Data Requirements
According to Aiken, logical data models assist with the transition between the conceptual and physical data models. They provide information about how to build the conceptual model and the effort involved.
Through this process, the logical model may challenge the existing conceptual data models since logical data modeling provides information about effort, such as size and shape. So, business and IT are involved in discussing logical data models to refine proper relationships with entities, simplify and standardize agreement about the architecture, and facilitate a shared vocabulary between business and technical analysts, noted Aiken.
The Physical Data Model – Technical Blueprints to Build the Business Solution
Upon understanding the existing business requirements, knowing what the business wants, and how to build it logically, companies have a specific goal. These businesses have evidence on how to recreate the data structures, flow, and entity relationships to construct a solution that aligns. This information forms the physical data model, the implementation, and the blueprint.
Typically, IT can apply semi-automated techniques, using built-in algorithms or creating programs “to uncover or check the data structures to be created, updated, read, or deleted,” noted Aiken.
Questions may arise requiring technical professionals to return to the business with conceptual and logical models to get more information to solve the business problem through the physical model.
Conclusion
“Companies need to do Data Modeling to solve a specific business problem or answer a business question,” summarized Aiken. IT and businesses need to share goals and understanding to get to a data solution. Moreover, there needs to be a common language between systems for data to flow smoothly.
However, slapping together any model or a big overarching enterprise architecture will not be helpful. A data model needs to achieve a particular purpose, and getting there requires a systematic process.
Aiken’s three-dimensional model evolution framework provides resources for an improved data platform. It considers the existing architecture and the evolution needed to meet business needs and validates that stakeholders and builders are on the same page.
A combination of conceptual, logical, and physical data models promises meaningful and useful results, especially where business and IT need to achieve a common objective. Doing the data modeling correctly and understanding requirements frees up 20% time and money for corporations to leverage their data capabilities and get more value from them.
Watch the webinar:
Image used under license from Shutterstock