Outsourcing Contracts and AI: The New Considerations

A team from Norton Rose Fulbright run through some of the novel issues advisors need to consider now so many outsourcing projects involve the use of AI

Traditionally, IT outsourcings (ITOs) and business process outsourcings (BPOs) have been centred around the delivery of services by personnel with clear service descriptions and performance requirements and may include the use of “traditional” software. The emergence of AI and its potential use in outsourcing, whether as a standalone service or incorporated into an ITO or BPO, has brought with it its own set of considerations when drafting outsourcing contracts.

While many provisions we ordinarily see in an outsourcing contract remain applicable when AI is used, AI introduces new complexities that require entirely new provisions in a contract (and a new approach to some existing provisions in a contract).

Here we look at the following provisions in an English law governed outsourcing contract, and whether they are fit for purpose for an outsourcing that involves the use of AI:  

• IP and customer data.
• Liability, relief and remediation.
• Exit.
• Maintenance arrangements.
• Third party licensing arrangements.
• Performance monitoring.
• Implementation and acceptance.
• Change management.
• Audit; and
• Subcontracting.

Particular attention may also need to be given to other provisions in an outsourcing agreement (for example, data protection provisions; and insurance in connection with AI), and to the specific regulatory requirements (for example, the EU’s AI Act), including  for particular sectors (for example, regulated outsourcings in the financial services sector or sectors where high risk AI systems may be deployed). These matters are, however, beyond the scope of this article.

IP and customer data

Al is trained on datasets. The more there is suitable, quality data available for training, the more the Al should improve, and therefore the more efficient the outsourced process should become.

Such training can raise a number of questions for the customer, for example:

• Will the customer provide some or all of the training data for the AI?
• Does the customer have the necessary datasets?
• Would the customer benefit from Al that has been trained on third party data, or even the data of other customers of the supplier?
• Is the customer willing to share any improvements to Al that result from training on its data?
• Would such sharing relinquish a competitive advantage for the customer (or even give competitors an insight into the customer’s business)?

In a traditional outsourcing contract, normally each party retains ownership of, and licenses to the other, their background IP for the purposes of the contract and there is a debate as to whether any foreground IP created during the term of the contract is assigned to the customer. Customer data must be held securely (or perhaps not held by the supplier at all) and processed only for the purposes of the outsourcing. Such a traditional approach to IP and customer data does not cater for the complexity of whether data, or subsets of the data, can be contributed to the supplier’s AI data lake.

At one end of the spectrum, a customer might take the view that it wishes to segregate completely any Al deployed in its outsourcing. A separate, “plain vanilla” instance of the Al is accordingly deployed on segregated servers and then trained on customer data (perhaps in combination with datasets that are licensed from the outsourced supplier or third parties). The customer would then require contractually that all improvements to the Al could not then be redeployed elsewhere by the outsourced supplier.

At the other end of the spectrum, a customer might agree to deploy Al and contribute customer data to the data lake on which it is trained. The contract would provide that the outsourced supplier would then own the IP in the Al, and in any improvements to it, so could deploy it elsewhere. This means that any valuable insights from the customer data would be (indirectly) available to potential competitors, but the customer would also benefit from improvements to the Al that come from training on other customers’ data.

Rather than opting for one end of the spectrum or the other, a layered approach may be taken to this issue. The customer may be willing to contribute certain low-value data and relinquish ownership of IP in general improvements to the Al, while at the same time segregating high-value data and retaining ownership of the IP in the consequent improvements. Investigation as to how both parties will meet data protection requirements if there is any personal data and how the confidentiality of contributed customer data will be maintained will be essential, as will tailored provisions reflecting the practical steps the parties will take and the risk allocation in these two areas.

Liability, relief and remediation

A great deal has already been written by legal commentators on the autonomous nature of Al and its implications for liability. This is a complex area and beyond the scope of this article.In the outsourcing context, the autonomous nature of Al raises a series of questions for the customer:

• To what extent should the outsourced supplier be liable for errors caused by AI?
• In what circumstances should the outsourced supplier be granted relief from liability?
• How, in practice, can errors or breaches caused by Al be remedied contractually?

It is common for an outsourcing contract to contain a “Relief Events” or an “Excusing Cause” mechanism under which the outsourced supplier is excused from liability where the customer has caused the outsourced supplier to be in breach of its obligations (for example, by the customer failing to satisfy certain dependencies). Al is only as good as the data on which it has been trained, so if the customer has provided poor quality training data, should this be grounds for outsourced supplier relief?

Similarly, an outsourcing contract often includes a remediation procedure in respect of contractual breaches. Commonly, the supplier is required to conduct a root cause analysis and then prepare and implement a remediation plan.

How would such requirements apply in the case of AI? The functioning of Al is often opaque. It may not be possible to identify a root cause and therefore remedy the issue. The outsourced supplier may be unwilling to share detailed information about the functioning of an Al solution because it views the solution itself as proprietary.

We may also start to see the emergence of:

• A liability “supercap” in relation to breaches that arise from the supplier’s use of AI.
• Warranties around the output produced by the AI. The warranties given may vary depending on how the AI’s output is to be used. For example, whether the outsourced supplier will warrant the accuracy of the output may differ between output that is used by a human which has ultimate decision-making authority versus output that is generated fully autonomously and requires no human intervention.

Exit

Most outsourcing contracts contain detailed provisions for termination assistance and for conducting an orderly transition to a replacement supplier (or to the customer if the function is being brought back in-house) in accordance with an agreed exit plan.

In a traditional outsourcing:

• This will generally require that the outsourced supplier migrates the data to a replacement supplier and deletes any customer data in its possession following exit (and, in some cases, to transfer related licences or equipment).
• Outsourced suppliers will often be required to facilitate knowledge transfer and to maintain a service library or know-how repository for this purpose.
• In many jurisdictions, exit arrangements will include procedures for managing transfer of employees under TUPE or its equivalent.

Deployment of Al may make exit and continuity more complex as:

• The outsourced supplier may be unwilling to provide information in relation to proprietary Al or license it to a replacement supplier (who is likely to be a competitor).
• It may not be possible to provide the training data for use by a replacement supplier because it belongs to the customer / existing outsourced supplier or multiple third parties.
• It may not be possible to extract customer data that sits in a data lake.

Maintenance

The centrality of software to an Al-enabled outsourcing blurs the lines between an outsourcing and a software procurement. When some or all of a business process is being conducted by a piece of software, rather than by humans, the outsourcing contract will need to cater for the maintenance of that software. This may require additional provisions more commonly seen in a software maintenance contract, such as:

• Pro-active monitoring and an incident management framework, with incident classifications and response and fix service levels.
• Provisions for the deployment of upgrades and patches, potentially with commitments as to system downtime and maintenance windows.
• Provisions for secure access to customer systems by supplier personnel performing maintenance (if the Al is deployed on customer systems).
• Software warranties.

While many ITOs already contain these provisions, BPOs often do not.

Third party licensing arrangements

A customer is generally required to grant to the outsourced supplier any licences necessary to access and use the customer’s third party systems. The deployment of Al may impact upon the customer’s ability to grant such licences where licences from third parties are often licensed on a per-user model. This type of licensing arrangement with third parties may no longer be appropriate in the Al context because a single Al solution can do the work of many human users. This may create issues for customers where:

• Software providers may license software on the basis of human users, in which case additional licences for non-human users may be required.
• Because Al is able to operate at a much higher processing frequency than a human, it may actually mean that the customer is using third party software beyond the specified volume tolerances.

Not only will AI impact upon key provisions in the outsourcing contract itself, it will also require the customer to:

• Undertake careful due diligence on its third party licensing arrangements and on whether it can grant the licences to the outsourced supplier.
• Obtain any additional licences where required.

Performance monitoring

The purpose of outsourcing is for the outsourced supplier to take responsibility for a particular function of the customer’s business. In doing so, the customer is not entirely giving up control of the service. For example, the customer may have the right:

• To request an outsourced supplier’s employee be removed from the customer’s account, or to specify that a key person be assigned to the customer’s account.
• For enhanced supervision rights over the supplier.
• To receive regular reports on performance.
• To attend regular governance meetings.
• To require the outsourced supplier to implement a remediation plan to remedy poor performance.

In all these cases, the customer has mechanisms in place to understand how well the services are being provided and to provide input into any remediation required to bring performance of the outsourced services to meet the service levels.

In the case of AI, such customer protections may have little beneficial effect. For example:

• Implementing a remediation plan to remedy issues in the AI could involve a major overhaul of the AI (as opposed to fixing a bug or rolling back a version we might see in traditional software).
• Removing outsourced supplier personnel would not affect the performance of the AI.
• Supervising the outsourced supplier’s personnel would give the customer little tangible benefit where the AI operates autonomously and opaquely.

Such difficulties raise questions about how the customer can get comfort that it has the ability to have oversight over the AI. Additional obligations may need to be imposed on the outsourced supplier to stand behind its AI system, which could involve the outsourced supplier confirming that the AI: 

• Is capable of generating the output that meets the customer’s intended purpose.
• Is logical, defensible, lawful and valid.
• Does not discriminate against protected characteristics.
• Is accurate for the customer’s intended purpose.

Other potential provisions could be for the outsourced supplier to provide technical documentation and instructions on the use of the AI. Such documents could include details such as:

• The characteristics, capabilities and limitations of performance of the AI.
• How the AI was developed (including development methods, design specification, system architecture, data requirements, human oversight measures, how it will react to pre-determined changes, validation and testing procedures and cybersecurity measures).
• Its level of accuracy, robustness and cybersecurity against which the AI has been tested and validated.
• The degree to which the AI can provide an explanation for its output.

Provisions such as those above:

• May overlap contractual provisions driven more specifically by AI regulatory requirements.
• Are unlikely to provide the customer with any tangible benefit on requiring the outsourced supplier quickly to improve the AI at times of poor performance (as to do so may take some time in relation to AI, as with other software). Such provisions will, however, give the customer greater transparency on how the AI is expected to perform and, should any breach occur, give the customer grounds to bring a claim against the outsourced supplier. 

Implementation and acceptance

In ITOs and BPOs there is typically an implementation phase during which relevant assets and systems are transferred to the outsourced supplier and deployed.

Following successful testing and demonstrating to the customer that there are no issues with the service, the services then “go live”. This process is not too different from implementing AI, where:

• The outsourced supplier trains the AI on (typically) the customer’s datasets to ensure that the AI performs in accordance with the customer’s requirements and intended purposes, similar to an implementation phase in a BPO and ITO; and
• The outsourced supplier validates that the AI works as intended to achieve compliance with the customer’s requirements and intended purposes – similar to traditional acceptance testing to achieve “go live” by the milestone date.

In the AI context, such a two phased approach would help the customer to build confidence in how the AI works and in the output it can generate, ultimately determining whether the AI is fit for the customer’s intended purposes.

Change management

Outsourcing contracts contain strict change management procedures dealing with how the parties can request and agree to changes to the services. The outsourced supplier is also typically prevented from making changes to the services unless such changes have been agreed through the change management procedure.

AI systems often have a continuous ability to improve and learn from each interaction (often referred to as “adaptive AI”) which may make traditional change management procedures redundant for the more “day-to-day” changes in the AI. While material changes to the AI system would be captured by any change management procedure (for example, new versions or changes to uptime availability), preventing the outsourced supplier from making changes to the services except through the change management procedure may not work in relation to adaptive AI. It could put the supplier in breach should the AI continually improve and learn (contrary to otherwise potentially static requirements of the contract).

Although changes to the AI may be ongoing without the customer’s oversight or approval, certain contractual provisions could be included in outsourcing contracts to mitigate associated risk, such as:

• The outsourced supplier contractually committing to contractual requirements to the effect that the AI has been designed and developed to: (i) allow the customer (or the outsourced supplier’s personnel, as the case may be) to monitor its operation; (ii) interpret output; and (iii) include controls that allow a natural person to decide whether or not to use the AI or disregard, override or reverse output, and to interrupt or stop the AI’s decision-making.
• The outsourced supplier implementing a quality management system which provides for the ongoing testing and validation of the AI and the updating of technical specifications.

As mentioned above in relation to other AI-related provisions, such provisions may overlap contractual provisions driven more specifically by AI regulatory requirements, reflecting the fact that it may become best practice to include them even when the relevant AI is not regulated as such.

Records and audit

One of the customer’s primary mechanisms to monitor an outsourced supplier’s performance is to exercise its audit rights. Audit rights generally permit the customer to request access to the outsourced supplier’s information, devices, personnel and premises:

• So it can then get a level of assurance as to how the services have been performed.
• If any non-compliance is discovered.

For audit rights to have teeth, the outsourced supplier is typically contractually required to maintain records in relation to the services so that, when the customer wishes to conduct an audit, the relevant records are available.

As AI may not involve intervention or performance by a natural person, access to personnel and premises is unlikely to give the same benefit as it would in the case of a traditional outsourcing. Therefore, the types of records maintained by the outsourced supplier in relation to AI will become much more important in an audit. They may need to be more prescriptive, covering:

• Decision-making the AI has performed (typically in the form of “logs”).
• Anomalies or dysfunctions.
• The method the AI uses to make predictions, content, recommendations and/or decisions.
• The development methodology and testing undertaken to assess accuracy, discrimination against protected characteristics and any other relevant form of unfairness.
• Information about the logic, significance and consequences of the predictions, content, recommendations and/or decisions made by the AI.

As with many of the other AI-related provisions described above, such provisions may overlap contractual provisions driven more specifically by AI regulatory requirements.

Subcontracting

Traditionally, when an outsourcing occurs, the customer has full transparency and control over the use by the outsourced supplier of subcontractors. This is simply because the customer is entrusting the outsourced supplier with providing the outsourced service, and therefore the outsourced supplier should not be subcontracting material aspects of the services without the customer’s consent.

The likelihood of subcontracting where AI is used is high. This is because the necessary compute power to operate AI mean the outsourced supplier may have a high dependency on critical third party providers, such as large language model providers or cloud service providers, to provide the infrastructure to host and operate the AI.

The traditional approach to subcontracting, therefore, may no longer be fit for purpose, as realistically the customer may have little room to restrict the outsourced supplier’s subcontracting of critical systems.

In traditional outsourcings, when subcontracting is permitted, the customer will often seek to impose requirements on the outsourced supplier to the effect that the relevant subcontract should contain certain prescribed provisions on a flow-down basis (e.g. rights for the customer to audit the subcontractor and the right for the subcontract to be transferred to the customer on certain trigger events).

However, in an outsourcing where AI features, given that the businesses that the outsourced supplier will need to subcontract with may have significant bargaining power, the outsourced supplier may have little leverage to negotiate with them. The result may be that the outsourced supplier may simply pass on to the customer the same terms and operational constraints that it has obtained with the critical third party provider.

It remains to be seen whether we will see a two-tiered approach to subcontracting with one regime for subcontracting in relation to the operation of the AI systems (where the customer would have little control over the approval of subcontracting), and another for any other subcontracting (where the customer retains some control over the approval of subcontracting). Such a two-tiered approach is already commonplace for regulated outsourcings (for example, in the financial services industry).

Action required

A customer’s decision to take on an outsourced solution that uses AI requires it to strike a balance between potential difficulties in managing risk associated with transparency, governance and performance of AI-related services against the benefits (efficiencies, cost-savings etc) that AI can bring. 

That balance finds expression in a skilfully drafted and negotiated outsourcing contract. Outsourcing contracts should accordingly contain provisions that are fit for purpose for AI, designed specifically to ameliorate AI-related contractual risks. 

A well-advised customer will seek to understand how the AI solution is deployed within an outsourcing solution, conducting appropriate due diligence, before ensuring that risk is appropriately allocated in the outsourcing contract.