Information request and response under the Freedom of Information (Scotland) Act 2002
FOI Reference: FOI/202500478412
Date received: 1 August 2025
Date responded: 1 September 2025
Information requested
Request for Information 1: Any internal evaluation reports or assessments of the AI Document Understanding and Client Advisor Knowledge Assist systems, including performance metrics like accuracy rates, staff time saved or cost savings, and impacts on service delivery or claimant outcomes.
Request for Information 2: Any targets or benchmarks used to assess the success of these systems during trials or in production.
Request for Information 3: Documents or emails discussing changes made to the system after deployment based on performance issues.
Please provide this information from the date of the system's pilot to the present.
Response
AI Document Understanding System:
Request for Information 1:
We have provided the Intelligent Automation pilot assessment report and the AI transparency report document relating to the AI document understanding system. Information outwith the scope of your request has been redacted.
Request for Information 2:
While our aim is to provide information whenever possible, in this instance Social Security Scotland does not have the information you have requested. This is because no formal targets or benchmarks were used to assess the success of the AI document understanding system specifically.
This is a formal notice under section 17(1) of FOISA that Social Security Scotland does not have the information you have requested.
However, the AI document understanding tool is part of a wider intelligent automation project as explained in the attached document number 4.
The AI document understanding tool is a digital reader that uses optical character recognition (OCR) technology. We use the OCR technology to scrape information from PDF application forms which helps reduce processing times.
We have provided the Intelligent Automation pilot assessment report and the AI transparency report registration document. Both specify some of the potential benefits this technology may deliver for Social Security Scotland.
An exemption under section 38(1)(b) of FOISA (personal information) applies to a small amount of the information requested because it is personal data of a third party, e.g. names/contact details of individuals, and disclosing it would contravene the data protection principles in Article 5(1) of the General Data Protection Regulation and in section 34(1) of the Data Protection Act 2018. This exemption is not subject to the ‘public interest test’, so we are not required to consider if the public interest in disclosing the information outweighs the public interest in applying the exemption.
Request for Information 3:
While our aim is to provide information whenever possible, in this instance Social Security Scotland does not have the information you have requested. This is because no changes were made to the system after deployment based on performance issues.
This is a formal notice under section 17(1) of FOISA that Social Security Scotland does not have the information you have requested.
Client Advisor Knowledge Assist System:
Request for Information 1 - 3:
While our aim is to provide information whenever possible, in this instance Social Security Scotland does not have some of the information you have requested.
This is a formal notice under section 17(1) of FOISA that Social Security Scotland does not have the information you have requested.
However, we have included the Client Advisor Knowledge Assist System proof of concept report as this may be considered an internal evaluation report
An exemption under section 38(1)(b) of FOISA (personal information) applies to a small amount of the information requested because it is personal data of a third party, e.g. names/contact details of individuals, and disclosing it would contravene the data protection principles in Article 5(1) of the General Data Protection Regulation and in section 34(1) of the Data Protection Act 2018. This exemption is not subject to the ‘public interest test’, so we are not required to consider if the public interest in disclosing the information outweighs the public interest in applying the exemption.
The Client Advisor Knowledge Assist System was a proof of concept that was not taken further. Therefore, questions 2 and 3 cannot be answered as they are not applicable in this instance.
About FOI
Social Security Scotland is committed to publishing all information released in response to Freedom of Information requests. The Scottish Government also publishes responses to requests. You can view the responses at http://www.gov.scot/foi-responses.