How to identify the right AI model governance solution for your business

This article was contributed by Harish Doddi, CEO of Datatron.

The pandemic has wreaked havoc on the carefully developed AI models many organizations had in place. With so many different variables shifting at the same time, what we’ve seen in many companies is that their models became unreliable or useless. Having good documentation showing the lifecycle of a model is important, but that still doesn’t provide enough information to go on once a model becomes unreliable.

What’s needed is improved AI model governance, which can help bring greater accountability and traceability for AI/ML models by having practitioners address questions such as:

• Which input variable are entering the model? 
• What are the output variables? 
• How does the model behave in terms of certain metrics? 
• What were earlier versions like? 
• Who has access to it? 
• Has any unauthorized person gained access to it?  

How exactly does AI model governance help tackle these issues? And how can you ensure you’re using it to best fit your needs? Read on. 

Too much manual effort 

Data scientists use a variety of tools to develop their models, whether it’s  SAS, R, Python, or the multitude of machine learning software libraries available today. With machine learning still in its nascency, there are a ton of options to choose from. Some use cases are simply more effective with certain languages or frameworks, for example, and data scientists tend to be relatively loyal to one language over another.  

Since this field is so specialized and data scientists are so few, their work is siloed from the rest of the enterprise. This makes it difficult for the primary IT or oversight body to guarantee appropriate company-wide governance and audit of models. That means this body will need to exert major manual effort to go to all the various departments and gather the needed model governance information. They can overcome this issue by implementing an AI governance solution.

Evaluating AI governance solutions

There are certain expectations, rules, and assumptions that ML models must abide by during the development process. When these models are deployed into production, they can yield quite different results from those in controlled development environments. Governance is critical here.

Those involved in governance must have a means of tracking the different models and the different versions associated with the models. For an AI governance solution to be effective, its catalog must have the ability to track and document the framework that the models are developed in. 

In addition, the catalog must have the ability to ensure model lineage where it associates the models with the functionality features within the models. Importantly, it enables computation of the appropriate governance metrics of the various features. 

In recent years, as more organizations have operationalized ML models, their dark side has emerged in the form of biases and other issues.  An example would be a financial institution whose models recommend offering lower credit limits to women compared to men living in the same home.  

It’s necessary to have the ability to compute and track metrics that might affect these models, such as anomalies, risks, levels of performance, biases, and data drifts. It’s not possible to simply calculate them in a lab; calculations must be done when the models are in production.

You’ll need a dashboard that can display these metrics to your data scientists and your business users. The metrics need to be displayed in such a way as to alert business users to potential issues. And your data scientists need to see the metrics that will guide them to those possible issues. You’ll also need a feature that enables you to identify potential anomalies based on the business-specific thresholds you set — and to notify both parties if something’s off without overwhelming them with false alarms.  

ML models need secure access

Particularly in larger organizations, model security is crucial. Serious problems could occur should a model accidentally get exposed to the wrong department. For instance, imagine that ML models have been successfully optimized to increase revenue by a few points. Now imagine that another department incorrectly uses those models. That could expose the company to fines equaling millions of dollars for regulatory violations.

It’s possible to tweak and reverse-engineer models, but if you don’t understand their original context, your organization could be at risk from possible potential iteration programs. In this case, the tweaked models aren’t doing what they’re documented to be doing. Model governance has left the building.

It’s important to require permission to access sensitive models that shouldn’t be shared with other departments. To make certain that no unauthorized parties — including applications — can get access to your model, you need encryption and an audit trail. You have to set up a method that guarantees traceability, transparency, and accountability.

Standardization is key

It’s obvious that a model governance solution offers many benefits, but implementation can be a challenge. Speed, cost, and effectiveness are likely to suffer from the complex review flows of AI model governance. 

Consistency is a big problem. Your governance solution must be applicable across all models, not just to certain business departments. Not all solutions offer standardization, so add that item to your list when vetting them. 

Making your models successful

Some organizations have had to scramble as the pandemic weakened or destroyed their AI and ML models. This also highlighted the need for model governance to improve accountability and traceability for machine learning models. A model governance solution, properly vetted, will reduce risks and increase the likelihood of successful models that will serve business goals.

Harish Doddi is the CEO of Datatron.