Why businesses take a month or more to deploy ML models and what you can do

Machine learning (ML) is an invaluable asset to modern businesses across the board. However, when it comes to ML models, both B2C and B2B companies face the problem of delayed time to market. According to Algorithmia, a vast majority of companies take at least a month or longer to first develop and then deploy their ML model. 

The reason for this is a complex and often very costly two-stage process. Developing an ML model can be a lengthy and potentially expensive process in and of itself. But what many companies often don’t realize early on is that the initial stage must then be followed by another, arguably more challenging phase – deployment. This second stage involves moving the ready model to production, testing and fine-tuning it, and then scaling up accordingly. 

Only around 10% of all businesses are estimated to possess enough experience, financial resources and technical expertise to deploy a fresh ML model to production within a week after its completion. Many struggle for up to a year, with at least 30% of all companies taking at least three months post-deployment. How long it takes exactly largely depends on which of the three popular model types the company chooses.

Off-the-shelf, custom and custom adaptive models

Of the ML models currently available on the market, there are the following: generic models, custom models and custom adaptive models.

Generic and custom models are basically polar opposites. The difference is that generic models are low in both cost and accuracy, while custom models are high in both cost and accuracy. This is because generic models are designed to suit virtually every business within that industry. These are normally based on ResNet, BERT/GPT, and similar off-the-shelf technologies. As a result, these models are affordable and dependable, but they’re also far from being a perfect fit.

In contrast, custom models are always tailored to the task at hand and are therefore much more accurate. However, they also come with a much higher price tag because of their high development and maintenance costs. Those who start with a generic solution and then attempt to improve their ML model often venture beyond the model’s basic architecture. What they eventually end up with is a custom model. A custom model that can be adapted to wider business needs right away and forego most of the lengthy post-deployment fine-tuning is a custom adaptive model.  

An adaptive model is therefore a type of custom model with some benefits that generic models offer. Like all other custom models, adaptive models are designed with particular business needs in mind. For this reason, they’re very accurate. At the same time, they don’t require that the company figure out MLops after the initial development stage. As a result, they in some ways operate like generic models in the deployment and post-deployment stages, with relatively low maintenance costs and improved time to market.  

Choosing an ML model

Which model your business requires – that is, whether paying extra is worth the stretch – depends on your particular situation. Your business may need something quite straightforward like sending online orders to different warehouses depending on their location. In this case, a generic ML model might just do the trick, especially if you’re a small business. 

On the other hand, if it’s something specific like content moderation for an online community of doctors discussing medical equipment, a custom model will work better. What a generic ML model may view as inappropriate language – for example, mentions of genitalia – is not only appropriate but necessary in the context of medical discussion. The training model in this case needs to be tailored to the company’s distinct needs. And this tailor-made model can be either adaptive or not.

Let’s consider the pros and cons of each model:

Custom adaptive models

Custom ML models are expensive due to the often unforeseen pre- and post-deployment costs. Because of these generally high startup costs, some companies tend to steer clear of the tailor-made option, instead opting for the less accurate but also less costly generic track. How expensive a training model actually gets depends on a number of factors, including the chosen data-labeling methodology, which is reflected in the model’s flexibility or its lack thereof. 

The following case illustrates a crowdsourcing-based custom adaptive model in action, i.e., an adaptive model that relies on human-in-the-loop labeling:

One well-known company that offers a technical editing environment wanted to boost its software’s accuracy and diminish the model’s training costs. The engineering team had to come up with a more efficient solution for correcting sentences in English. Any solution had to be in line with a fully manual labeling pipeline that was in place already.

The final solution entailed using a pre-existing custom model for linguistic processing that was adapted to the client’s needs. Third-party AutoML was used for text classification within the target sentences. Subsequently, phrase verification accuracy rose by 6% – from 76% to 82%. This, in turn, reduced the model’s training costs by 3%. Furthermore, the client did not need to make additional investments – financial or otherwise – into the model’s infrastructure, as is normally the case with most custom models.

Key points to keep in mind

Choosing the right ML model for your business can be a daunting task. Here’s a summary of what you should take into account to make an informed decision:

• Consider how specific your needs are: the more specific the need, the further away from the generic model you should move as a rule of thumb.

• Always consider scalability – if that’s something you know you’ll need, consider paying extra for something tailor-made just for you.

• If you don’t require high accuracy but need fast deployment, consider opting for the generic route. 

• If accuracy is important to you, consider how much time to market you can spare.

• If you’re short on time and require high accuracy, consider taking the custom adaptive route; otherwise, any custom solution can potentially fulfill your needs just as well.

• In terms of the overall cost, the generic route is the cheapest of all – followed by the custom adaptive route that bypasses most MLops expenses – and finally by all other custom solutions whose costs may rise substantially post-deployment (the exact figures differ greatly on a case-by-case basis).

• Consider whether you have in-house data scientists and MLEs at your disposal – if yes, going for the traditional custom option developed internally may be feasible; if not – consider the other two (generic or custom adaptive).

• When choosing between custom vs. custom adaptive options, consider how accurate and specific to the needs of your customer the ML model ultimately has to be. The higher the accuracy and adaptability, the higher the cost and longer the waiting period to prepare and maintain the model.  

Fedor Zhdanov is head of ML products at Toloka AI.