When it comes to metrics, “Less is more,” said Kelle O’Neal, Founder and CEO of First San Francisco Partners, presenting at DATAVERSITY® Enterprise Data Governance Online 2017. You don’t have to measure everything, she said, you just have to “choose what’s important and meaningful to your stakeholder group and your program.” O’Neal’s presentation showed how to create meaningful metrics to measure the impact of a Data Governance program, and offered examples and practical knowledge for incorporating metrics in a Data Governance program.

“Metrics are one of the hardest aspects of Data Governance [to grasp] and also one of the most important,” she said, because metrics provide an opportunity to establish a baseline to “know what bad data means, and potentially, what good data could mean,” creating a starting point to get the broader organization to understand the need for better data. Metrics can help get an organization aligned with a set of shared goals, “and this alignment of expectations is really important.” Metrics can also provide an opportunity for engagement with stakeholders.

A Data Governance Framework

O’Neal shared a slide outlining First San Francisco Partners’ Data Governance Framework, which showed areas where Data Governance operates and what tasks are involved. Governance helps define priorities, she said. “We believe that governance is an organizing framework, and that that framework helps to establish the strategy objectives – and therefore the policies – around effectively managing corporate data.” That framework also helps to ensure that your data is of high quality, that it is usable, that it has integrity across the right use cases and applications, that it stays consistent when aggregated, that it is secure, and that you’re protecting the privacy of the data owners. “It is definitely not a ‘one size fits all,’” and so some of the metrics may not apply to your situation.

“Most organizations do start with a strategy, and we recommend that as a mechanism to align your Data Governance program to corporate goals, objectives, and priorities.” Although Data Governance includes policies, processes, guidelines, and standards, without taking into consideration the corporate environment where it will be implemented, it’s unlikely the program will be successful, she said.

Governance exists within your overall information management program, and that is, of course, within your overall business strategy and the way that your organization goes to market in general. “The way that we think about enterprise information management is that it provides the information strategy that supports the rest of your business.”

Why are Metrics Important?

Metrics can show whether or not you are aligned with your business strategy, and can “help to ensure that the silos of business process within your organization understand consistently and are aligned on the expectations of the governance program,” she said. Metrics can articulate the relevance and value of your Data Governance program on an ongoing basis. “You’ll find that the definition of value, the definition of relevance, and how you align your organization changes over time; which means your metrics will also change over time,” as your strategy evolves.

O’Neal stressed that it’s important to remember that not every measurement is significant. Although anything can be measured, not every metric is necessarily a Key Performance Indicator (KPI). A true KPI “helps to identify whether the program is actually performing to expectations,” and it’s best to prioritize your efforts in these areas, she said.

Measuring Progress and Impact: Two Key Perspectives

Progress toward a set of goals can be measured as well as the impact of that progress on the organization.

Starting with the measurement of progress, O’Neal elaborated how progress can be measured in four contexts: People, Process, Technology, and Data.

Progress Metrics: People

Using an example of the rollout of a Data Governance program, she showed ways that progress could be measured as it relates to people. To start, consider “tasks and activities that go into aligning, assigning and on-boarding people as part of the program,” as well as the number of people who have been trained, and their ongoing participation. Other ways that this area can be measured is by tracking the number of resolved issues, the number of data owners identified, the number of projects approved, and the program adoption rate by company personnel.

Adoption rate falls into the KPI category because it “tends to indicate whether the program is consumable by people, so they understand it enough to adopt it and they care about it enough to adopt it, which means that it’s meaningful to the organization,” she said.

Progress Metrics: Processes

“Measuring processes is a great way to measure how well governance is embedded into the organization,” she said. Consider which processes have an impact on your goals and focus your measurement on those. Identify and articulate how processes are created, how implementation planning should occur, and how processes will be executed, she said.

“When we’re thinking about the execution of these processes, it’s really around the adoption. So, do people understand what the new process is? Are they using the new process? Are we working through our backlog of processes?” Metrics could also come from tracking the number of data consolidated processes, the number of approved and implemented standards, or the number of consistent data definitions and how those definitions are used in within different processes of the organization.

Progress Metrics: Technology

The progress of technology planning and implementation within the Data Governance program can be measured by the integrity of the data across systems, by the number of consolidated data sources, by the number of data targets using master data, or the quantity of lineage documented, she said. If the goal is “to create a more consolidated, more rationalized, and potentially unique identification of data elements – whether that’s from a master data perspective, [or] maybe it’s from a product data perspective – if you are trying to ensure adherence to regulations . . . maybe the KPI is something like the presence and usage of that unique identifier.”

Progress Metrics: Data

“If we look at every single entity of data within an organization, it’s extraordinarily overwhelming, so we want to look at how we focus and prioritize those data and entities,” she said. Options for measuring progress with data could include looking at common data entities and how requests for improvement of those data entities are managed, or documentation of those data entities. “In the sense that we’re looking at different Data Quality dimensions as it pertains to those specific data entities; maybe we’re looking at process efficiency.” Considering how those Data Quality metrics impact productivity, “That’s really a KPI- it’s the link between the improved Data Quality and the productivity of the usage of that data in the organization,” she said.

Process to Establish Impact Metrics

The process of establishing impact metrics is something O’Neal considers extraordinarily important because “You don’t just want to measure progress, you want to make sure that you’re measuring the value that you’re providing to the organization.” Impact metrics can be established by asking a series of questions to uncover issues, set goals to resolve those issues, and define how you’ll measure the impact of reaching those goals.

Start by dissecting and prioritizing the issues, and then “create the measurement and metrics that address the business need,” she said. Through the process of discussing and clarifying each issue, “you start to get more specificity around what is needed.” Focus on what change you would like to see and asking if that change will create a difference in the work that you are doing, she said. This question makes it possible to “track that change over time, and ultimately come up with a result that you can measure. And that result is the impact of that data change in order to improve something within the business.”

Example: Improving Report Quality

O’Neal used an example of improving report quality to illustrate the process of clarifying an issue, understanding the changes needed to affect that issue, and measuring the impact of those changes. She recommends asking a series of questions, in this case, starting with, “What is the impact that we could provide by improving that report quality?”

She then suggests asking follow-up questions to get an understanding of the specific components of the issue:

• Why is that report of poor quality?
• What is the change that you would like to see?
• How does that report quality impact your organization?
• How do we want to improve that report quality and accuracy?
• What would it mean to your organization if we could provide that change?

She offered the following answers for her example: “Well, I don’t really understand what is in the report, I’m not really sure where the data came from, and the process to get to that report is time consuming.”

Through the line of questioning, she said, “We’ve identified that the reason the reports are not accurate is because there is a lot of duplication of client data, which then results in inaccurate and inconsistent reports. We’ve also identified that those records are not as complete as they need to be,” and that the amount of manual remediation is slowing down the process, making it difficult to ensure that the report is complete by the time it’s distributed, she said.

“The idea here is that we are taking those data change metrics that we’ve identified – things like duplication of client data, completeness of field, of lineage,” and the length of time manual work adds to the process and, “We’ve identified those progress metrics that are important to making a business impact.”

Getting to Business Change/Impact Metrics

By breaking down measurements for the impact of each goal, the path to improvement becomes clearer. “The idea here is to use those data change metrics and those progress metrics to articulate and explicitly show an impact to a KPI.” Once progress has been made, she said, “We’re going back to our stakeholder and demonstrating an improvement in the report quality and accuracy because we have tracked and we have measured and we have improved all of the elements that lead up to the accuracy of the report. And we’re able to demonstrate the change.”

Translating Metrics (Data Value) into Business Value

Lastly, she stressed the importance of effective communication to stakeholders. “One of the challenges we have as a data community is to make sure that we are communicating the data change and the data value in a way that is consumable and understandable by our stakeholders,” she said.

Using the previous example, by telling the stakeholder “that we have reduced the duplication of client information by 95%,” that business owner won’t automatically correlate improvement in productivity to improvement in the report. “So, we need to translate that data change into a business value statement.” It’s better to say,

“We reduced duplications, so your report analysts are spending 50% less time on manual processes. This means the production of that report has gone from seven days down to three, and we anticipate that once we address a completion issue, it will go down to one and a half days, because that productivity gain has meaning to that stakeholder.”

 

Check out Enterprise Data Governance Online at www.datagovernanceonline.com/

Here is the video of the Enterprise Data Governance Online 2017 Presentation:

Photo Credit: Aunging /Shutterstock.com