AI systems in procurement

More and more governments are experimenting with AI systems. This involves examining whether an AI system is useful and desirable and whether the system fits the needs of the organization. While experimentation will not easily require a tender, this changes if a government wants to deploy a system for the longer term. What should you look for when tendering for an AI system?

Needs
The first step of any procurement is to determine the user's needs. This is where perhaps the first difficulties will arise. How do you formulate the needs of the user of an AI system? It may be worthwhile to start a market consultation for this purpose. With this you can find out what suppliers think they can deliver and with what guarantees. The City of Amsterdam has formulated some general needs that are worth considering:

Technical transparency: how does the system work. Is it explainable? Is it auditable?
Procedural transparency: what is the purpose of the system and how was it made? What choices and assumptions were made and what data was used.
Explainability: how did the system arrive at a particular decision? This is especially important when a system makes decisions that affect citizens personally.
Goals
When deploying AI systems, it is very important to determine what the goal of the deployment is. Specifically, what policy goal is being pursued? The purpose of an AI system will often also be directly related to the processing basis, if personal data is used. If there are also secondary policy purposes when tendering for the use of the AI system, it is important to include this in the tender, as this will allow vendors to ensure that their system is suitable for this.

Minimum Requirements
Once the need(s) and goal(s) are clear, you can start determining minimum conditions. Here the City of Amsterdam is again a possible source of inspiration. It has in fact drawn up model conditions for algorithmic applications. In addition to these conditions, you can also think of specific functional requirements, such as certain performance guarantees, explainability or control options.

Award and selection
Drawing up the award criteria will have to be done in close consultation with the future user. The price will obviously always play a role here, in addition to quality.

When possibly using selection criteria that reduce the number of candidates for a tender, you can think about the following aspects: has this company ever had a data breach and where is any data stored.

Conclusion
Needs, policy goals and minimum conditions are of great importance when tendering for an AI system because it is difficult to compare multiple systems purely on award criteria. With good minimum conditions, you can be sure that you are buying an AI system that meets clear requirements for transparency, quality and management.