AI in the energy sector guidance consultation

3.1 There is a well-established principle that, regardless of the arrangements for procured services and components, an organisation cannot outsource responsibility or accountability for safety, security (including data privacy and confidentiality), fairness or sustainability. 

3.2 Stakeholders are expected to ensure effective supply chain management is in place to deliver safe, secure, fair, and sustainable use of AI. Arrangements are expected to be put in place to ensure users of AI can influence their supply chain through their procurement, contractual, purchase or developer decisions such that they clearly articulate any legal responsibility and liability. These arrangements may include: 

a. ensuring that AI developers have robust arrangements in place to build confidence in the development of safe and responsible AI 

b. effective and auditable training of AI systems 

c. adequate testing 

d. arrangements for anonymisation of data, governance, triangulation of data risk management (for example, drawing correlations between data sets that are misleading) and incident reporting  

e. data provenance records and adhere to commercial data assurance

3.3 Users of AI may have limited input into the AI system design and development and be heavily reliant on guidance, policies, copyright statements and other materials and communication issued by AI developers. The supply chain, for both external AI services and internal AI-based components and systems, should be understood and should have sufficient transparency, traceability, validation and verification processes in place to ensure the desired outcome is achieved. Users should expect to be able to access information regarding what and where the AI is used, its characteristics, any failure modes and necessary risk control measures needed (including cyber security risk control) and how to identify issues such as incorrect or misleading results (often referred to as hallucinations). 

3.4 The supply chain for AI systems adds additional complexity to developing secure systems, for example exacerbating data issues. NCSC identify securing this supply chain as the first point in their ‘Secure development’ of AI. This challenge is similar to that faced with the cloud. Due to the supply chain challenges, NCSC produced guidance, stating 'the service provider should ensure that its supply chain meets the same security standards that the organisation sets for itself […] if this principle is not implemented, supply chain compromise can undermine the security of the service and affect the implementation of other security principles'. 

3.5 AI can also present new vulnerabilities and threat vectors due to their complexity, rapid development and the potential opacity of the supply chain.  

3.6 Novel vulnerabilities and cyber security threats with AI include:

a. training data and model poisoning: injection of malicious or corrupt data

b. prompt injection: attackers feed the AI malicious prompts

c. model memorisation: models remembering specific detail 

3.7 Stakeholders should ensure that: 

a. where data is exchanged with externally developed, hosted and operated services there is a clear and thoroughly documented understanding of the data flows, data use, data storage and retention  

b. supply chain managers address relevant technical aspects of procured services and components and these are reflected in commercial arrangements, including data ownership, privacy, confidentiality and cyber security

c. the supply chain management arrangements are continually reviewed and updated to reflect the pace of development