The second wave of interest for a Master Data Management (MDM) solution is here. Are you thinking of implementing a new MDM or replacing your existing MDM solution? There are some dos and don’ts when designing your next MDM solution. In this article, we will share a few key considerations based on the current trends, technology availability, and what we think your MDM solution should deliver for your company in the future.
First of all, when choosing your next MDM solution, look for products that are built on technologies that will carry your company through the next 20 years and not based on technologies that have been around for the past 20. Microservices, graph database, and AI are no longer just technology buzzwords; they are becoming mainstream, more mature, very effective, and readily available technologies for MDM. If you think about technologies that are available today that can continue to deliver for your company in the next 20 years, you will need to consider these top three in your design: cloud-native architecture, graph database, and AI. The time is right for a cloud-native MDM solution using graph infused with AI.
A cloud-native MDM solution should be built on microservices architecture, which by design allows fast and reliable delivery of complex applications continuously, providing companies the ability to release new features much quicker than traditionally designed software applications based on monolithic architecture. Microservices also allow companies the flexibility to adopt new technology stack easily when building new services or making changes to existing services, reducing the risks associated with a long-term commitment to any technology vendor.
A cloud-native MDM solution should also leverage container technologies like Docker and Kubernetes. Containers have a very small footprint and are designed to provide agility and scalability to your applications. It will allow multiple MDM developers to write code in separate self-contained environments, resulting in quicker development, faster deployment, and unlimited scalability.
Many companies, or some parts of the company, have been adopting a more disciplined approach by applying the reliable and proven agile and DevOps methodology to cloud applications. Automation is key to a more predictable execution and achieving operational excellence offered by methodologies like Agile and DevOps. It reduces human error, improves quality, and speeds up processes. Your cloud-native MDM application will need to support deployment automation so you can execute continuous integration and continuous delivery (CI/CD). The goal of having a CI/CD strategy is to achieve ongoing operations and zero downtime so you can keep updating the software or deploy new features without disrupting the application or services.
CI/CD types of agile capabilities will be key enablers to meet the constant learning and changing required by all application systems in the upcoming business world. Gone are the days of 12-month, six-month, or even three-month release cycles. MDM release cycles will be more like what you encounter in your mobile phone apps – new features can be released on any day and at any time of the day and, in fact, likely multiple times a day, if needed. Successful companies nowadays rely heavily on technology for their day-to-day business from product development, marketing, sales, service, etc. Companies that take three months to react to competitive threats are no longer competitive.
Companies must stay agile and quick to compete in order to survive, and therefore their technologies supporting their business need to be agile and fast. Continuous learning, changing, and delivering are a matter of survival in the new business world.
Companies that are used to the waterfall approach or even have dabbled with agile methodology will find developing, maintaining, and supporting any cloud-native application challenging. A whole different mindset needs to be adopted, or the journey will be very painful. That being said, it is better to start now than later. The level of adoption pain will be proportional to the skills set you can acquire and the willingness to change. Trying to upskill existing staff for cloud-native solution applications is very difficult. A whole new approach is needed to work on cloud-based tools, applications, and infrastructures. It is advisable to import new talent in these areas if you don’t have them in-house. Once successfully implemented, a cloud-native MDM solution integrating the critical data across the company will play a key role in business transformation initiatives that continuously learn and adapt through trusted master data.
Graph database has come a long way to support serious, never-go-down, heavy-duty real-time applications like MDM. Its object-oriented concept with nodes and edges (relationships) maps very well to the master data domains (person, organization, product, location) and the relationships among themselves and others. Each microservice matches well with each node and relationship that are put together to form the graph. By nature, as companies build their business one product, one customer (person or organization), one location, and one relationship at a time, they execute on these microservices one at a time to form the knowledge graph to support their business. The graph database has the additional advantage of being more visually appealing natively without knowing how to run SQL queries to join and collate tables together. Representing master data as a graph allows enterprises to discover relationships and patterns that could have been ignored if we modeled using relational or NoSQL-based databases.
Nowadays, as AI technology becomes more mature and readily available, it would be crazy not to infuse AI into every new application that you have, deriving intelligence and insights from your data. The additional automation benefits from AI will further drive cost reductions and improve business efficiencies. Modern MDM applications on cloud should natively support machine learning to drive AI capabilities. It should be part of your intelligence architecture on the cloud supporting continuous enterprise learning – the nexus of architecting enterprise intelligence. Although not mandatory, having the ontologies and data in a graph database to form a knowledge graph does help in facilitating deep learning using machine learning tools. The AI-assisted decision can be leveraged in many aspects of MDM, including matching, de-duplication, events notification, relationship forming, product recommendation, product bundling, and many more. The MDM knowledge graph can be combined with other data to form a trusted enterprise knowledge graph that will further enhance the advanced analytics results to drive AI-based automation and business acceleration across the company.
MDM application, by definition, is real-time operational in nature. Otherwise, data will be out of date, and MDM loses its effectiveness of being the Master Data Management system it’s designed to be. Using graph and advanced analytics like machine learning, real-time analytics can be incorporated as part of each event that happens to any piece of master data. Looking at a piece of master data in a graph model can inherently answer many insights that are not immediately available in a legacy MDM application. Traditionally, insights can be derived and constructed only through table joins and queries in traditional relational and NoSQL databases in a legacy MDM system. In addition, it would require multiple teams working together and weeks or months of effort, from defining requirements, developing queries to analyzing results. The real-time analytics made possible by graph database natively is becoming apparent.
Additionally, in a cloud environment, platform resources can be allocated automatically to handle additional workloads needed or to create the scalability and performance required by the business for real-time analytics – keeping in mind that master data does not include all data required for analytics. Therefore, more complex analytical requirements can only be meaningfully accomplished by having the master data integrated with the rest, including the data outside the company’s boundary, such as social media data.
The number of cloud-native MDM vendors is only a few, but it’s growing, so companies will soon have more options to choose from. Major vendors are offering cloud-based MDM solutions such as SaaS, managed services, or licensed to run and manage by yourself on your own cloud. Most vendors today do not support full-blown data on the graph database. Infusing AI as part of the MDM solution is still at an immature state for the time being, but it will get better as demands grow and more companies realize the benefit of it and the competitive advantage it adds to their organization.
Some vendors with traditional on-prem monolithic MDM solutions have ported their legacy MDM products to run on cloud. They might have added some cloud-native capabilities, but these adapted legacy MDM applications are not the same as full-blown cloud-native MDM. These re-platformed legacy MDM solutions may have traditionally proven features, capabilities, and reliability that may still look appealing. But, keep in mind that technologies based on legacy architecture and design would definitely not be able to support the agile, DevOps, CI/CD methodologies needed for those companies that need to be flexible and change quickly to compete. If done successfully, cloud-native MDM can enable fast deployment of features many times per day. Through rigorous automated testing and deployment pipelines, it’s easy to deploy changes across many functions on a continuous basis, which means features are released to users more frequently, bugs are fixed more quickly, operations are optimized continuously, and so on. Companies that can successfully manage these changes consistently with predictable outcomes will come out as winners in the marketplace.
Some companies are not waiting for the vendors and are proceeding with bespoke development of this next-generation MDM solution in-house. Companies typically view MDM as a very strategic IT solution to support their continuous business transformation. They are also the ones with in-house IT skills to build cloud-native applications or willing to hire external consultants to assist. If these companies have an existing MDM solution that they are used to and model against, the job will be easier but still a pretty steep hill to climb. Others who do not have MDM experience will find it extremely challenging, as they will struggle with both technology and domain experiences. They should engage external expertise!
MDM has many different data domains such as a person, organization, location, product, reference data, etc. Obviously, which data domain to use depends on the business requirements and priorities. For bespoke development, if given the benefit of choosing a domain to start (i.e., there are business requirements and urgency for all of them), then start with the reference data domain first. This is the most simplistic MDM domain to build. One should learn to crawl before walking and running. So, bespoke development of a cloud-native MDM starting with reference data management would be a wise initial step. Mistakes will be made in the first phase of MDM development, but learning can be accomplished, and experience will be gained with this initial build to get you better prepared when it’s time to tackle the other more complex MDM domains.
For more advanced MDM practitioners, and for those who need to comply with corporate mandates on new product design and deployment, some of the other key design tenets for cloud-native MDM include:
- Container-based micro-service design
- Well scoped, proven, and flexible data model
- Graph DB
- Supports CI/CD pipeline
- M/L enabled (data integration, data quality, match, auto-merge, ever-greening, event management, data remediation, etc.)
- Security and access control
- Authentication integration
- Audit trail
- International languages (include double-byte languages)
- Multi-tenant
- Data residency compliant readiness
- Edge-computing readiness
- Extensibility
- Performance and scalability
- High-availability and DR capable
You may not need all the above design tenets on day one, and some may not be needed at all, like international language and multi-tenant, but do consider them carefully, so they don’t come back to haunt you later. To play it safe, you might want to incorporate all of these in the design but execute them when needed. This will minimize major rewrite efforts in the future.
Other technical issues that may not relate to MDM alone but cloud applications in general that need to be considered include:
- Development languages and environment
- Platform software
- Operating system
- Network, server, and storage
- Virtualization
- Infrastructure management
- Security management
The choices for a cloud-native MDM solution are available now. Make sure your choice includes graph database and AI. If you are going to invest in MDM, invest in a solution with technologies that will last you the next 20 years.