Back to research

Reports

Automating Authority: Artificial intelligence in European police and border regimes

Date

9 April 2025

Topics

Borders and frontiers

Institution

European Union (EU)

Summary analysis

Read the summary analysis as a PDF.

Full report

Publication information

Authors: Chris Jones, Romain Lanneau

Research support: Samaya Anjum, Eloisa Griffiths

Thanks to: Nidžara Ahmetašević, Hope Chilokoa-Mullen, Sara Chitseko, Riccardo Coluccini, Caterina Rodelli, Niovi Vavoula

This report was supported by the European AI & Society Foundation.

Manuscript completed February 2025.

Published by Statewatch, April 2025.

Statewatch produces and promotes critical research, policy analysis and investigative journalism to inform debates, movements and campaigns for civil liberties, human rights and democratic standards. We began operating in 1991 and are based in London.

Support our work by making a donation:

https://www.statewatch.org/donate/

https://www.statewatch.org/about/mailing-list/

Registered UK charity number: 1154784. Registered UK company number: 08480724. Registered company name: The Libertarian Research & Education Trust. Registered office: MayDay Rooms, 88 Fleet Street, London EC4Y 1DH, UK

Next section >
Contents

Summary analysis

Acronyms and abbreviations

1 Introduction

2 Cop out: security exemptions in the Artificial Intelligence Act

2.1 “A historic achievement”

2.2 Summary: exceptions and loopholes

2.3 In detail: the AI Act’s security exemptions

2.3.1 Scope and application of the law

2.3.2 (Un)prohibited practices

2.3.3 Risk and impact assessments

2.3.4 A “silicon curtain” of secrecy

2.3.5 Conformity assessment

2.3.6 Data protection

2.3.7 Oversight

2.4 Implementing the Act

3 Security AI in EU agencies

3.1 eu-LISA

3.1.1 Algorithmic profiling of travellers

3.1.2 AI in the shared Biometric Matching System

3.1.3 Digitalising the visa application process: visa chatbot

3.2 Europol

3.2.1 From challenge to opportunity

3.2.2 Machine learning

3.2.3 Facial recognition

3.2.4 Data protection and European policing

3.3 Frontex

3.3.1 AI in the maritime domain

3.4 EU Asylum Agency

3.4.1 Automated dialect recognition for asylum applicants

3.5 Eurojust

3.5.1 Joint Investigation Teams platform

4. Building the infrastructure

4.1 Institutional infrastructure

4.1.1 eu-LISA

4.1.2 The EU Innovation Hub for Internal Security

4.1.3 The European Clearing Board

4.1.4 Frontex

4.2 Technical infrastructure

4.2.1 Security Data Space for Innovation

4.2.2 Europol: sandboxes and pipelines

Annex I High-risk systems under the AI Act

Annex II Information to be registered in the EU database of high-risk AI systems

Annex III AI technologies and techniques of interest to EU policing, migration and criminal justice institutions and agencies

< Previous section
Publication information

Next section >
Summary analysis

Summary analysis

This section of the report is also available as a PDF.

Mainlining AI

Artificial intelligence (AI) is big business. Since the release of the generative AI chatbot, ChatGPT, in November 2022, the hype and hubbub surrounding AI technologies has reached fever pitch, and it seems unlikely to die down anytime soon.

Businesses are adopting AI to automate all manner of tasks. They are inserting the technology into everyday tools such as web searches, whether people like it or not. Governments hail AI’s supposed ability to improve public services, working conditions and education, amongst other things.

The UK government has promised – or, perhaps, threatened – to “mainline AI into the veins” of the country.[1] This phrase, perhaps unwittingly, represents the overall strategy of the companies behind the technology: insert AI into as much public and private infrastructure as possible, and thus guarantee an ongoing flow of profit.

Suprnational AI

The repressive agencies of the state also form part of this picture. Police and border forces across Europe and beyond are already investigating or using various forms of AI.

The German authorities are interested in “biometrics and face recognition.” The Czech Republic is aiming for “prediction of crime.” Norway hopes to enable “fraud detection in the immigration directorate.” Its neighbour, Sweden, has tested “facial recognition… at some borders.”[2] These technologies come on top of a wide range of other automated and algorithmic systems that bake in bias and discrimination.

These trends are reflected in EU institutions and agencies, which have long seen new technologies as “solutions” to a wide range of social issues. As Statewatch has previously analysed, the bloc’s immigration, border control and policing systems are being extensively digitalised.[3] Introducing “artificial intelligence” is the next step in this process.

Europol, the EU’s policing agency, stores vast amounts of data sourced from police forces, private companies, or retrieved from the web. It is adopting advanced technologies to process and analyse that data. These include machine learning (section 3.3.2) and upgraded facial recognition systems (section 3.3.3).

The border agency, Frontex, is deploying new surveillance technologies at the EU’s borders and beyond, and developing new systems for the collection, consolidation and analysis of data (section 3.4.1). Like Europol, it now also has a role in determining the EU’s priorities for its technological research and development programme, Horizon Europe. This means the agencies can influence the development of new technologies.

The EU agency for managing policing and migration databases, eu-LISA, is building tools to algorithmically profile and assess travellers (section 3.1.1). The EU Asylum Agency is developing a tool to help identify the nationality of asylum seekers based on the way they speak (section 3.4.1). Eurojust, the judicial cooperation agency, is also slowly incorporating AI tools into its systems and processes (section 3.5).

This is just the start. Many other potential uses of AI have been identified in lengthy studies carried out for EU institutions (see Annex III). Some of the potential uses would be incredibly invasive. They include:

the police using AI to detect “irregular travelling patterns,” through analysis of plane and other traffic;
using AI to monitor “the level of success in integration” achieved by individual migrants, and for migrant groups as a whole; and
using AI for “assigning individuals seeking asylum to detention centres.”

To the best of our knowledge, AI is not currently being used for these purposes. Concerted political and legal challenges and change will be needed for things to stay that way.

Security AI: exclusion and discrimination

These uses of AI – for policing, border, immigration, asylum and criminal justice purposes – are referred to in this report as “security AI.” Like other forms of AI technology, security AI systems receive certain data as inputs, and use it to produce various different “outputs.” These include predictions, profiles, risk assessment and suggestions.

These outputs can, in turn, be used to influence all manner of decisions: on criminal investigations and border interrogations, covert surveillance operations, visa decisions, and many more. This raises multiple questions for protecting the rights to liberty, security, non-discrimination, assembly, freedom of movement and effective remedies, amongst others.

However, the effects of these systems will not be evenly felt: marginalised and racialised people will bear the brunt of them.

The EU’s plans will mean millions of people travelling to the EU ‘legally’ are profiled by algorithms. Refugees, forced to travel ‘illegally’, already have to take dangerous and deadly journey to seek safety, due to border control and surveillance measures. Enhancing and increasing that surveillance with AI will only compound those risks.

Police forces and the criminal justice system are beset by racism and other forms of discrimination.[4] “Racist comments, more frequent stops and even violence – this is how people of different ethnic backgrounds experience policing in Europe,” says the EU’s Fundamental Rights Agency.[5] A 2018 study of the criminal justice system in 12 EU member states found that “disparities exist for people of various ethnic, racial, and national origins, at least at some stages of their criminal justice systems and in some form.”[6]

Reinforcing these systems with AI technologies may provide a veneer of technical objectivity and fairness. This is why there is such concern amongst officials about “debiasing” AI systems. However, this approach does nothing to address the structural dynamics of exclusion, subjugation and discrimination that shape the role and actions of police, immigration and other state agencies.

Security AI complex

Discussing these issues is difficult for many reasons. One of those is secrecy: the development of security AI remains largely hidden from public view and excluded from political debate. This report seeks to alter that situation, to encourage democratic discussion, and to support work towards accountability.

There are no smoking guns or big “reveals.” This is not that kind of story. Rather, it is the tale of an ongoing attempt by politicians and officials to develop new institutional, technical and legal infrastructure for the swift development, testing and use of security AI.

Over the past five years, these efforts have varied in their scale and ambition. Some appear to be slowly embedding themselves in the EU’s institutional landscape, such as the Innovation Hub for Internal Security (section 4.1.2). Others, like the plan to create a “centre of excellence” for AI at eu-LISA, the policing and migration database agency (section 4.1.1), have fizzled out – though of course, they may be revived.

Nevertheless, all these initiatives show an intended direction of travel, and it is one that merits far greater scrutiny than it has so far received. By exposing and analysing this emerging “EU security AI complex”, we aim to inform meaningful public and political debate and decision-making.

The development of supranational security policies and powers should not be left in the hands of agencies and institutions that remain invisible or unaccountable to the public or their elected representatives. This is particularly important in light of the path dependencies created by these policies and powers. Their existence makes certain future policy choices more likely than others, precluding possible alternatives.

The AI Act: exceptions, exemptions and loopholes

The first substantive section of the report analyses the AI Act. The Act provides the legal framework that will govern the use of AI – including security AI – in the EU for the years to come (section 2).

The Act achieves two key things. First, it establishes conditions for increased development and use of AI systems. Second, it ensures that security AI systems are subject to extremely limited accountability, oversight and transparency measures.

The Act includes:

the possibility to use mass biometric surveillance, AI-powered risk assessments and emotion recognition systems for immigration, asylum and border control purposes;
a total exemption from the law until 2030 for large-scale EU databases and information systems;
a self-assessment process that allows providers of high-risk AI systems to exclude their systems from the safeguards the Act imposes on high-risk systems;
widespread secrecy over the testing and use of security AI; and
the exclusion of people outside of the EU from the Act’s protections, despite a number of the EU’s own AI systems explicitly targeting such people.

The report examines provides a summary of each exemption (section 2.2), and then examines them in detail (section 2.3).

The law presents formidable challenges to understanding, scrutiny and accountability of security AI. The level of secrecy it permits is particularly problematic: turning the techniques used to detect and investigate crime, or to control migration, into state secrets simply increases impunity.

Legal experts have already taken a dim view of many of these exemptions and exceptions. The Act itself says it is “without prejudice” to a host of EU and national legal requirements. However, on the face of it, it clashes in a number of ways with the EU Charter of Fundamental Rights, jurisprudence from the Court of Justice of the EU, and existing laws.

These clashes cover topics ranging from the scope of a “national security” exemption, to the right for people to receive explanations about AI-informed decisions, and the powers of independent supervisory authorities. There will likely be a substantial amount of litigation in the years to come as authorities, companies and individuals seek to have aspects of the law clarified, and potentially strengthened in favour of protecting peoples’ rights.

The exception is the rule: the security AI imaginary

In a more ideological sense, the Act also contributes to a very particular ‘imaginary’ of AI. It invokes claims of urgency and emergency to justify restrictions on rights and safeguards. This is a familiar story, particularly for anyone who has lived through the growth, normalisation and bureaucratisation of the “war on terror.”[7]

What is more novel is the way these exceptions align with the visions put forward by some proponents of AI technologies.[8] Bypassing normal procedures in the name of urgency rests upon an idea that there already are, or will be, AI systems that are so effective or powerful that they will be able to protect society from various ‘threats.’

The law does not go into any details about what kind of AI system might be involved in such a situation. It is highly doubtful that any such AI system exists or will be built (section 2.3.5). This ideological role, however, fits neatly into a long, deeply-embedded history of techno-solutionism in EU policy-making.[9]

The police lobby: watering down safeguards

Part of the reason the law contains so many loopholes and exceptions is the result of lobbying by the police themselves. The police have also been working to develop tools for self-regulation, promoted as a way for security agencies to comply with the Act.[10] Weakening the law in secret, whilst publicly calling attention to your efforts to comply with it, is hardly a demonstration of trustworthiness.

In May 2022, the European Police Chiefs Convention issued a public statement on the AI Act, calling for specific exemptions for police forces.[11] This was the public face of a broader, secret effort to undermine any potential protections in the law.

EU governments worked hard to water down safeguards in the Act, with the French authorities playing a key role.[12] Internal security officials in the Council also kept a close eye on proceedings. This was part of a broader push to have “internal security needs” recognised in digital policies.[13]

Backing up this work was an obscure and secretive body called the European Clearing Board (EuCB, section 4.1.3). This is an informal group set up by senior EU member state police officers. The EuCB’s Strategic Group on AI worked extensively to weaken safeguards in the AI Act. One document obtained for this report says the EuCB’s lobbying:

…triggered important changes in the Council position on the AI Act, including on the definition, classification of systems, remote biometrics, use of dactyloscopy [fingerprinting] and exceptions for law enforcement (mandatory publishing of AI-systems in use or that are developed by law enforcement agencies).[14]

The EU treaties do not foresee a formal role for police agencies in negotiating new legislation, though it is hardly surprising that they engage in lobbying. It is certainly unfortunate, however, that the EU’s secretive and opaque law-making system makes it essentially impossible for the public to be aware of it.[15]

“Cutting-edge products for the security of citizens”

The EuCB is part of another new piece of institutional infrastructure: the Europol Innovation Lab. The Lab is based at Europol’s HQ in the Netherlands, and was set up to implement a December 2019 decision by EU interior ministers. The EuCB provides the Lab’s connection to national agencies and authorities.

The Lab, in turn, is a member and host of the EU Innovation Hub for Internal Security (section 4.1.2). The Hub brings together representatives of all the EU’s justice and home affairs agencies, covering “law enforcement, border management, criminal justice and the security aspects of migration and customs.”[16]

The Innovation Hub is supposed to “support the delivery of innovative cutting-edge products for the security of citizens in the EU,” through “the use and development of advanced and emerging technologies.”[17] It has yet to acquire the budget and staffing hoped for by officials, but has nevertheless coordinated a number of joint projects. In relation to AI, these include systems for profiling travellers to the EU, and research on biometric technologies for border and immigration control.[18]

More recently, the Hub has been reorganised and is now based around a number of “clusters”, including one dedicated to AI. This was launched in spring 2024. It remains to be seen how the work of the cluster and the EuCB will develop. Doing so will be difficult without ongoing monitoring and investigation. They do not publish the agendas and minutes of their meetings, and are formally accountable only to police and interior ministry officials.

A “centre of excellence” for security AI?

On the opposite side of Europe, in Estonia, another EU agency has also been trying to build up institutional infrastructure for security AI. The agency, eu-LISA,[19] is primarily responsible for the operation of the EU’s growing collection of large-scale policing, migration and criminal justice databases.[20]

In October 2021, eu-LISA produced a “roadmap” setting out all “planned & potential, near to medium/long term” AI initiatives. Amongst these was the development of a “Centre of Excellence for Artificial Intelligence in the Justice and Home Affairs Domain” (section 4.1.1).

Responsibility for developing this idea further was given to the multinational consultancy company Deloitte. It conducted a study that said the Centre of Excellence (CoE) would coordinate “the strategy for AI within the JHA domain.” The CoE would also set up “frameworks for future projects to speed up the adoption of AI.”[21] Deloitte proposed the “strategy, purpose, requirements and operating model” for the CoE.

The agency did not give direct responses to questions from Statewatch on this topic, though it seems that the CoE plan has been dropped, for now. If member states and the European Commission “consider that the creation of a Centre of Excellence for AI is necessary, the Agency will take the necessary steps to do so,” eu-LISA’s press office said. Currently, the agency is preparing an AI strategy to “serve as an umbrella for organising the internal governance on AI initiatives and ensuring compliance with the AI Act.”[22]

Despite its fate, the CoE initiative is noteworthy for two reasons. Firstly, it may be indicative of future proposals to facilitate the development and use of security AI. Secondly, it is a remarkably wide-ranging initiative that was undertaken with no democratic scrutiny or oversight.

Based on the paper trail examined for this report, the very idea to set up a Centre of Excellence was first mooted in a report by Deloitte, itself based on interviews with EU officials. The idea was then adopted by EU institutions and agencies. This close involvement of private companies with the EU’s emerging security AI complex should not come as a surprise: it is one of its defining features.

Public-private partnership

The drafting of EU policy studies is often outsourced by the Commission to consultancy companies.[23] In the field of security AI, more and less well-known companies such as E&Y, Unisys, PwC, RAND Europe and Deloitte have all been involved in this kind of work. The aim of these studies is usually to set out policy options and explore their political, financial and institutional implications.

In practice, they largely seem to be a way to provide a veneer of independence and impartiality to proposals that are already more-or-less settled. At a minimum, it can be said they largely reflect the views of Commission and EU member state officials, as these tend to be the people interviewed for the studies. In this regard, they reflect the undemocratic nature of law and policy-making in the EU.[24]

Public-private cooperation extends much further than this, however. The databases and information systems managed by eu-LISA are already a public-private endeavour. Billions of euros have been awarded to multinational technology and consulting companies to set up, operate and maintain them,[25] making public institutions structurally dependent on the private sector.

This dependency is likely to increase as AI, alongside other digital technologies, becomes further embedded in security policies and structures. This is actively invited by the EU’s justice and home affairs agencies, which regularly hold “industry days” where companies can market their products.

Then there is the EU’s security research programme, which since 2003 has provided billions of euros for security and surveillance technologies.[26] These research projects are used to develop new forms of security AI. Both Europol (section 4.2.2) and Frontex (4.1.4) now have a structural, agenda-setting role in the programme, giving them influence over technology research and development.[27]

The influence of the private sector on the public sector can also be considered from other angles. One internal Europol document says the agency’s work on AI aims for “value creation at speed” – a term which, at least in part, recalls the Silicon Valley motto of “move fast and break things.”[28]

From institutional to technical infrastructure

Alongside new institutional infrastructure, the security AI complex also requires technical infrastructure: hardware and software that can process vast amounts of data. Two separate, but parallel, initiatives are underway in this area.

The first of these is part of a broader EU plan to create an array of “common European data spaces” (section 4.2.1). These will be made up of interconnected, but separate, datasets held by different organisations and institutions. The data will be used to train AI systems. Around 20 data spaces have so far been announced in sectors such as health, agriculture, finance, mobility, energy, public administration, and security.

The “Security Data Space for Innovation” (SDSI) will initially target law enforcement agencies. Border, immigration, criminal justice and customs agencies will later have access. Technologies of interest include automated image recognition and video analysis.

However, the plan has had a rocky start. There have been problems finding contractors to develop the SDSI. After scaling back the ambition of its initial plans, the Commission provided €1 million for a project to carry out “preparatory work needed for the creation of high-quality large-scale shareable data sets for innovation.”[29]

The result was the TESSERA project. Amongst other things, it will map the types of datasets that could be shared through the SDSI, including:

photos;
videos;
voices samples;
unstructured text, such as that on web forums;
unstructured hybrid data, for example scraped from websites or emails;
structured data, such as telecommunications metadata.[30]^,[31]

Europol: AI sandbox

Europol is also working to develop technical infrastructure for developing and testing security AI (section 4.2.2). Specific topics of interest include voice print analysis, age and gender detection from audio recordings of voices, and the use of augmented and virtual reality for data analytics.

As part of this work, it is developing a “sandbox” – an isolated technical environment in which software can be developed and tested with no external effects. Under the AI Act, member state governments are obliged to set up at least one sandbox for use by AI companies, externalising the costs of private sector “innovation” onto the public.

Documents obtained for this report describe a plan to divide Europol’s sandbox into two separate areas: one in which personal data cannot be processed, and one in which it can. The latter would make it possible test new algorithms or techniques “against a live, operational dataset containing personal data.”[32] Of course, technologies that are not tested on personal data, or that do not make use of personal data, may still have very personal effects: arrest, questioning, or search and seizure of possessions.

The aim is to have the sandbox up and running as soon as possible. One Europol document describes it as having “paramount strategic significance as it will enable Europol to fulfil its role in leading Law Enforcement Innovation.” It is “a precondition and enabler” and an “infrastructural foundation” for “numerous depending initiatives.”[33]

Questioning: the security AI complex

There is a vast ongoing effort from technology companies, governments and other “stakeholders” to insert increasingly powerful technical systems into every aspect of life and society. This is a political issue and dealing with it requires more democracy, not less. Yet, in relation to security policies and security agencies, less democracy is exactly what the public is being given.

The AI Act provides an extremely limited framework for the oversight and accountability of security AI. That being said, the law is also confusing and unclear, and it is likely many aspects will be clarified through jurisprudence. Effective legal challenges would see that jurisprudence lead to increased oversight.

The new infrastructure being established to embed security AI in EU policy and practice is secretive, complex and confusing. Even basic transparency measures are lacking: the publication of agendas, minutes and other documentation. This may sound mundane, but it is crucial for democracy. It allows the public to see what is being done in their name, with their money, by whom. Transparency is a fundamental prerequisite for accountability – whatever form that accountability takes.

Other angles are also important: the rapid development of AI technologies and their underlying infrastructure are creating huge demands on the world’s natural resources, in particular water and energy. Security AI is by no means the largest part of this problem, but these questions must be taken into account. There is no sign so far of this happening.

The late British politician, Tony Benn, had five questions that he would ask “everywhere he went… on the chalkboards of classrooms and lecture halls… at rallies, protests and marches.” The questions are more relevant than ever – particularly in a context where supranational institutions and agencies continue to accrue new powers and competences:

“What power have you got?”

“Where did you get it from?”

“In whose interests do you use it?”

“To whom are you accountable?”

“How do we get rid of you?”[34]

With regard to the last question, Benn would say that anyone who cannot answer it “does not live in a democratic system.” In the interests of a democratic system, then, the least that could be done is for the public and their elected representatives to start asking more questions about the security AI complex. What follows from those questions remains to be seen.

< Previous section
Contents

Next section >
Acronyms and abbreviations

Notes

[1] ‘Prime Minister sets out blueprint to turbocharge AI’, 12 January 2025, https://www.gov.uk/government/news/prime-minister-sets-out-blueprint-to-turbocharge-ai

[2] eu-LISA Working Group on AI, 1st meeting minutes, 11 May 2021, https://statewatch.org/wp-content/uploads/2026/05/annex-10-minutes-1st-wgai-meeting-document-7-_redacted.pdf

[3] ‘Europe’s techno-borders’, https://www.statewatch.org/publications/reports-and-books/europe-s-tchno-borders/; ‘Frontex and interoperable databases: knowledge as power?’, https://www.statewatch.org/publications/reports-and-books/frontex-and-interoperable-databases-knowledge-as-power/; ‘Deportation Union: Rights, accountability and the EU’s push to increase forced removals’, https://www.statewatch.org/publications/reports-and-books/deportation-union-rights-accountability-and-the-eu-s-push-to-increase-forced-removals/; ‘Automated Suspicion: The EU’s new travel surveillance initiatives’, https://www.statewatch.org/publications/reports-and-books/automated-suspicion-the-eu-s-new-travel-surveillance-initiatives/

[4] In the UK, an official report found that London’s Metropolitan Police are “institutionally misogynist.” See: ‘

What is institutional misogyny in policing and why does it matter?’, University of Liverpool, 4 September 2023, https://news.liverpool.ac.uk/2023/09/04/what-is-institutional-misogyny-in-policing-and-why-does-it-matter/

[5] Fundamental Rights Agency, ‘Tackling racism in policing’, 10 April 2024, https://fra.europa.eu/en/news/2024/tackling-racism-policing

[6] JUSTICIA, ‘Disparities in Criminal Justice Systems for Individuals of Different Ethnic, Racial, and

National Background in the European Union’, November 2018, https://rightsinternationalspain.org/wp-content/uploads/2022/03/Disparities-in-Criminal-Justice-Systems-for-Individuals-of-Different-Ethnic-Racial-and-National-Background-in-the-European-Union.pdf

[7] Gene Ray, ‘On the targeting of activists in the “War on Terror”’, Statewatch, 1 July 2008, https://www.statewatch.org/statewatch-database/on-the-targeting-of-activists-in-the-war-on-terror-by-gene-ray/

[8] At the extreme end, this includes those who believe that humans will merge their consciousness into, or with, some sort of hyper-powered AI, and through that process become immortal. See: Darren Orf, ‘A Scientist Says Humans Will Reach the Singularity Within 21 Years’, Popular Mechanics, 8 August 2024, https://www.popularmechanics.com/science/a61777484/2045-singularity-ray-kurzweil/

[9] ‘NeoConOpticon: The EU Security-Industrial Complex’, Statewatch/Transnational Institute, 17 February 2009, https://www.statewatch.org/publications/reports-and-books/neoconopticon-the-eu-security-industrial-complex/

[10] ‘New Accountability Framework to use artificial intelligence in a transparent and accountable manner’, 10 March 2022, https://www.europol.europa.eu/media-press/newsroom/news/new-accountability-framework-to-use-artificial-intelligence-in-transparent-and-accountable-manner

[11] ‘Joint Declaration of the European Police Chiefs’, 24 May 2022, https://www.europol.europa.eu/cms/sites/default/files/documents/EPC%20Joint-Declaration%20on%20the%20AI%20Act.pdf

[12] ‘France spearheads member state campaign to dilute European AI regulation’, Investigate Europe, 22 January 2025, https://www.investigate-europe.eu/posts/france-spearheads-member-state-campaign-dilute-european-artificial-intelligence-regulation

[13] ‘Exceptions, loopholes and carve-outs: Presidency wants “internal security needs” recognized in EU digital policies’, Statewatch, 23 February 2023, https://www.statewatch.org/news/2023/february/exceptions-loopholes-and-carve-outs-presidency-wants-internal-security-needs-recognized-in-eu-digital-policies/

[14] Innovation Hub Team, ‘EU Innovation Hub for Internal Security – multi-annual planning of activities 2023-26’, Council doc. 5603/23, LIMITE, 16 February 2023, p.21, https://statewatch.org/wp-content/uploads/2026/05/1335957-v1-eu_innovation_hub_for_internal_security_multi-annual_planning_of_activities_2023-2026_st05603_en-public.pdf

[15] ‘Trilogues: the system that undermines EU democracy and transparency’, European Digital Rights, 20 April 2016, https://edri.org/our-work/trilogues-the-system-that-undermines-eu-democracy-and-transparency/; ‘EU: Civil society calls for rights to be prioritised in secret AI Act “trilogue” negotiations’, Statewatch, 12 July 2023, https://www.statewatch.org/news/2023/july/eu-civil-society-calls-for-rights-to-be-prioritised-in-secret-ai-act-trilogue-negotiations/

[16] ‘EU Innovation Hub for Internal Security’, 5757/20, 18 February 2020, https://data.consilium.europa.eu/doc/document/ST-5757-2020-INIT/en/pdf

[17] ‘EU Innovation Hub on Internal Security’, 5757/20, 18 February 2020, https://data.consilium.europa.eu/doc/document/ST-5757-2020-INIT/en/pdf

[18] ‘Europe’s techno-borders’, pp.36-39, https://statewatch.org/wp-content/uploads/2026/04/europe-techno-borders-sw-emr-7-23.pdf

[19] Its full name is the European Union Agency for the Operational Management of Large-Scale IT Systems in the Area of Freedom, Security and Justice.

[20] ‘EU agencies and interoperable databases’, https://www.statewatch.org/eu-agencies-and-interoperable-databases/

[21] Deloitte study for European Commission, ‘Deliverable 3.01: AI Centre of Excellence definition’, HOME/2020/ISFB/FW/VISA/0021, undated, p.6, https://statewatch.org/wp-content/uploads/2026/05/deliverable-d3-01_-ai-centre-of-excellence-definition.pdf

[22] Email, 8 February 2025.

[23] European Court of Auditors, ‘External consultants at the European Commission: Scope for reform’, 2022, https://www.eca.europa.eu/Lists/ECADocuments/SR22_17/SR_External_consultants_EN.pdf

[24] ‘Study proposes giving EU complete control over Schengen borders’, Statewatch, 10 December 2014, https://www.statewatch.org/news/2014/december/eu-study-proposes-giving-eu-complete-control-over-schengen-borders/

[25] ‘EU agencies and interoperable databases’, https://www.statewatch.org/eu-agencies-and-interoperable-databases/

[26] The programme currently has the formal title ‘Civil Security for Society’: https://rea.ec.europa.eu/funding-and-grants/horizon-europe-cluster-3-civil-security-society_en

[27] Forthcoming work by the Resist Europol network will analyse the role of the EU security research programme in developing security AI technologies.

[28] ‘Did Mark Zuckerberg Say, ‘Move Fast And Break Things’?’, Snopes, 29 July 2022, https://www.snopes.com/fact-check/move-fast-break-things-facebook-motto/

[29] European Commission, ‘Call for proposals on data sets for the European Data Space for innovation’, 21 March 2023,

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/isf/wp-call/2021-2022/call-fiche_isf-2022-tf1-ag-data_en.pdf

[30] For an explanation of metadata, see: ‘What is Metadata? An introduction’, GSMA, undated, https://www.gsma.com/solutions-and-impact/connectivity-for-good/public-policy/metadata/

[31] European Commission, ‘Call for proposals on data sets for the European Data Space for Innovation’, 21 March 2023, https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/isf/wp-call/2021-2022/call-fiche_isf-2022-tf1-ag-data_en.pdf

[32] Europol, ‘Building the Research and Innovation Pipeline: Update on the implementation of article 33a and the R&I Sandbox environment’, 17 April 2023, EDOC #1301551v2, document for meeting of the Information Management Working Group meeting on 16-17 May 2023, https://statewatch.org/wp-content/uploads/2026/05/europol-building-the-research-and-innovation-pipeline.pdf

[33] Europol Innovation Lab, ‘Progress Report and Strategic Priorities 2024-2026’, 22 September 2023, EDOC #1321956v13, p.5, https://statewatch.org/wp-content/uploads/2026/04/europol-innovation-lab-progress-report-and-plan-2023-25.pdf

[34] John Nichols, ‘Tony Benn and the Five Essential Questions of Democracy’, The Nation, 14 March 2014, https://www.thenation.com/article/archive/tony-benn-and-five-essential-questions-democracy/

Acronyms and abbreviations

Acronym	Full name
AI	Artificial intelligence
AMIF	Asylum, Migration and Integration Fund
AR/VR	Augmented reality/Virtual reality
BoMIC	Border Management Innovation Centre
BUC	Business Use Cases
CCTV	Closed Circuit Television
CELIA	Common European Language Indication and Analysis
CEN	The European Committee for Standardization
CENELEC	European Committee for Electrotechnical Standardisation
CIR	Common Identity Repository
CJEU	Court of Justice of the European Union
CoE	Centre of Excellence for AI in the justice and home affairs domain
COI	Country of Origin Information
COSI	Standing Committee on Operational Cooperation on Internal Security
CRRS	Central Repository for Reporting and Statistics
DG HOME	European Commission Directorate-General for Home Affairs and Migration
EASO	European Asylum Support Office
ECRIS-TCN	European Criminal Records Information System on third-country nationals and stateless persons
EDPB	European Data Protection Board
EDPS	European Data Protection Supervisor
EES	Entry/Exit System
EIBM	European integrated border management
EP	European Parliament
EPPO	European Public Prosecutor’s Office
ESP	European Search Portal
ETIAS	European Travel Information and Authorisation System
ETSI	European Telecommunications Standards Institute
EU	European Union
EU VAP	EU Visa Application Platform
EUAA	EU Agency for Asylum
EuCB	European Clearing Board – Tools, Methods and Innovations in the field of technical support of operations and investigations
eu-LISA	EU Agency for the Operational Management of Large-Scale IT Systems in the Area of Freedom, Security and Justice
Eurodac	European Dactyloscopy
Europol	European Union Agency for Law Enforcement Cooperation
FRIA	Fundamental Rights Impact Assessment
Frontex	European Border and Coast Guard Agency
HART	Homeland Advanced Recognition Technology
HENUs	Europol Heads of National Units
Interpol	International Criminal Police Organisation
IXIM	Working Party on information exchange for internal security
JIT	Joint Investigation Team
LEA	Law enforcement agency
LED	Law Enforcement Directive
LEWP	Law Enforcement Working Party
MID	Multiple Identity Detector
ML	Machine Learning
NERC	Named Entity Recognition and Classification
ODIN	Operational Data for Innovation
PNR	Passenger Name Record
sBMS	shared Biometric Matching System
SDSI	Security Data Space for Innovation
SIS	Schengen Information System
SPoC	Single Points of Contact
TCN	Third Country National
USA	United States of America
VIS	Visa Information System
WGAI	eu-LISA Working Group on AI

< Previous section
Summary analysis

Next section >
1. Introduction

1. Introduction

The EU has long sought to develop and deploy advanced technologies for policing, border and immigration control, and criminal justice: biometric border controls, border surveillance drones, and machine learning systems for analysing vast quantities of data, amongst other things. The latest part of this push for technological “solutions” to so-called security problems is the development and deployment of artificial intelligence (AI).

The EU has long sought to develop and deploy advanced technologies for policing, border and immigration control, and criminal justice. These efforts, representative of trends around the globe, are increasingly coming to fruition.

Biometric border controls will soon be enforced on all travellers to the EU, with fingerprints, photos and other personal data stored in vast databases. High-tech surveillance drones patrol the EU’s borders, supporting illegal pushbacks and violence. Police forces have increasing access to sensitive personal data through supranational information-sharing systems. Police officers are being equipped with mobile fingerprint and face scanners to use in the street.

The concrete effects of these technologies vary. For officials, they may well provide more efficient means of carrying out their tasks. However, it is not clear if this will be of significant social benefit. The EU’s border control model requires the routine use of violence and abuse against people seeking safety. Police forces embody and enforce systemic forms of racism and discrimination. AI is often heralded as introducing a technological ‘revolution’, but there is no sign of it creating any systemic political change to overcome this situation.

For individuals, the effects of these technologies range from privacy invasions (through increased data collection on migrants and refugees), to death (for some of those ‘pulled back’ to Libya with the aid of EU surveillance drones). One overall effect, particularly for migrants and refugees travelling to or present in the EU, is ever-more detailed inscription in digital state databases. This provides for new means of regulation and control by state authorities.

The latest part of this push for technological “solutions” to so-called security problems is the development and deployment of artificial intelligence (AI). The story begins in 2019, when security officials launched a number of initiatives to help develop and use AI for security purposes – that is, for policing, immigration and border control, and criminal justice.

Officials have nurtured new infrastructure, both institutional and technical, whilst overseeing other pre-existing AI projects. Lengthy studies have delved into the potential uses of AI for policing, immigration and criminal justice, leading to new projects and initiatives. These developments have received little, if any, public or democratic scrutiny.

During this period, the EU introduced a new law to regulate AI: the AI Act.[1] This aims to stimulate the development and use of AI technologies in the EU, through a complex regulatory regime. The Act classifies AI systems according to different risk levels. A whole host of systems and techniques are automatically considered high-risk.

Many of these high-risk systems concern law enforcement, immigration and criminal justice agencies. However, the law grants those agencies a number of exemptions and exceptions, in particular police and border agencies. It also does nothing to increase transparency over the development and use of security AI. Instead, it reinforces a long-standing logic of secrecy.[2]

Taken together, these institutional, technical and legal developments point to the development of a security AI complex: a confluence of political and economic interests that aim to make the development and use of security AI a structural feature of state power and practice in the EU. This, in turn, is intended to reinforce and extend the repressive powers of the state: to control peoples’ movement, to monitor their activities and habits, and to arrest or imprison people.

These developments will be seen by some as largely positive. They will be seen by others as largely negative. The authors of this report sit in the latter camp. Whatever one’s views, they should not go unreported, unexamined or unscrutinised. Understanding these developments and formulating responses designed to protect human rights, civil liberties and democratic standards is particularly urgent at a time of rising authoritarianism, xenophobia, and the resurgence of fascism.

This report follows previous work by Statewatch on the development of new security technologies and techniques. It is primarily based on information obtained from access to documents requests, open data, and material obtained through non-official routes. The story it tells is incomplete, but it illuminates a number of ongoing developments that merit close, critical scrutiny.

The report contains four main sections. The summary analysis brings together the findings and arguments, and is designed to provide a brief overview of the key points. Its content is drawn from the three sections that follow this introduction, which look at the issues covered by the summary analysis in detail.

The first of those sections examines the security, exemptions and exceptions in the EU’s AI Act. The second looks at current and forthcoming uses of security AI by EU agencies, in particular the EU’s justice and home affairs agencies, responsible for policing, immigration and border control, and criminal justice. The third examines the institutional and technical infrastructure officials are developing to accelerate the development and use of security AI. A list of acronyms and and abbreviations is also included.

There are three annexes to the report:

A note on terminology and definitions

AI and AI system: The EU AI Act, approved in June 2024, defines an AI system as a machine-based system that is designed to operate with varying levels of autonomy.” It must be able to infer, from “explicit or implicit objectives” based on the input it receives, “how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.” An AI system may also “exhibit adaptiveness after deployment.”[3] The report uses this definition, though we are aware there are many ways to define AI and AI systems.[4]

Security and security AI: As with the term AI, “security” can also be defined in many ways. It is used in this report as shorthand for the policy areas addressed by policing, border, immigration, asylum and criminal justice agencies. “Security AI” is used to collectively describe the AI technologies, techniques and tools being developed for, and used by, those agencies.

The term “security” should be read critically. Other interpretations are available that contrast with the dominant, state-centric idea. As the Ammerdown Group argued in a 2016 paper: “The proper goal of security should be grounded in the wellbeing of people in their social and ecological context, rather than the interests of a nation state as determined by its elite.”[5] This should be borne in mind when reading the word in this report: what kind of security does “security AI” offer?

Security AI complex: This is shorthand for the confluence of political and economic interests converging around security AI. It is an imperfect term, but is useful for giving conceptual form to the institutional, technical and legal initiatives launched in the EU in recent years.

< Previous section
Acronyms and abbreviations

Next section >
2. Cop out: security exemptions in the Artificial Intelligence Act

Notes

[1] Further legislation on AI is in the works. This deals with non-contractual civil liability for damage caused by AI systems, and are currently under negotiation. See: European Commission, ‘Liability Rules for Artificial Intelligence’, 28 September 2022, https://commission.europa.eu/business-economy-euro/doing-business-eu/contract-rules/digital-contracts/liability-rules-artificial-intelligence_en

[2] ‘Patrick Breyer v European Research Executive Agency’, 7 September 2023, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:62022CJ0135; Madalina Busuioc, Deirdre Curtin and Marco Almada, ‘Reclaiming Transparency: Contesting the Logics of Secrecy within the AI Act’, European Law Open, 23 December 2022, https://www.cambridge.org/core/journals/european-law-open/article/reclaiming-transparency-contesting-the-logics-of-secrecy-within-the-ai-act/01B90DB4D042204EED7C4EEF6EEBE7EA

[3] Article 3(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_3

[4] To give just one example, the definition in the Act is substantially different from the one included in the initial proposal. See: ‘Proposal for a Regulation laying down harmonised rules on artificial intelligence’, 21 April 2021, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206

[5] The Ammerdown Group, ‘Rethinking security: A discussion paper’, May 2016, https://statewatch.org/wp-content/uploads/2026/05/ammerdown-group-rethinking-security-5-16.pdf

2. Cop out: security exemptions in the Artificial Intelligence Act

The regulatory regime introduced by the Artificial Intelligence Act will frame the use of artificial intelligence in the EU, and perhaps elsewhere in the world, for many years to come. In the field of security, it achieves two key things. Firstly, it establishes conditions for increased development and use of security AI systems. Second, it ensures that those systems are subject to extremely limited accountability, oversight and transparency measures.

In this section

2.1 “A historic achievement”

2.2 Summary: exceptions and loopholes

2.3 In detail: the AI Act’s security exemptions

2.3.1 Scope and application of the law

2.3.2 (Un)prohibited practices

2.3.3 Risk and impact assessments

2.3.4 A “silicon curtain” of secrecy

2.3.5 Conformity assessment

2.3.6 Data protection

2.3.7 Oversight

2.4 Implementing the Act

2.1 “A historic achievement”

In December 2023, after two years of negotiations and debate, the EU’s Artificial Intelligence Act was approved by the Council and the Parliament. Carme Artigas, the Spanish state secretary for digitalisation and artificial intelligence, lauded it as “a historic achievement.” She described the Act as keeping “an extremely delicate balance: boosting innovation and uptake of artificial intelligence across Europe whilst fully respecting the fundamental rights of our citizens.”[1] With regard to her latter point, an examination of the text reveals quite the opposite with regards to law enforcement and migration authorities.

The overarching aim of the AI Act is to “improve the functioning of the internal market and promote the uptake of human-centric and trustworthy artificial intelligence.” At the same time, it is supposed to ensure “a high level of protection of health, safety, fundamental rights enshrined in the Charter, including democracy, the rule of law and environmental protection,” while supporting “innovation.”[2]

It seeks to achieve these ends by setting out:

rules governing when and how AI systems can be placed on the market or put into use;
prohibitions on certain uses of AI systems;
requirements for AI systems classified as high risk, and obligations for those operating them;
transparency rules for certain types of lower-risk AI systems;
rules on monitoring the market for AI systems, including governance of that market and enforcement of the rules; and
“measures to support innovation”.[3]

One key aspect of the law is its “risk-based approach.” More stringent rules apply to AI systems deemed to pose higher risks to health, safety, fundamental rights, ethical standards or the public interest.[4] Those that pose the highest risks are prohibited.

This includes AI systems that deploy “subliminal techniques” to impair people’s ability to make informed decisions, and systems for social scoring. Remote biometric recognition systems – for example, public deployments of facial recognition – also fall under this heading. However, as explained below, there are multiple exceptions to this supposed prohibition.

High-risk systems are permitted, as long as they meet certain requirements. The providers and/or deployers of such a system must establish a risk management system, ensure appropriate data governance, produce various technical documents, and ensure a certain level of transparency. There are multiple exemptions to these requirements for security authorities.

The Act includes a specific list of systems that must be treated as high-risk. Amongst them are a number relevant to security policies: remote biometric identification systems; systems for biometric categorisation; and emotion recognition systems. The Act also lists specific types of systems for law enforcement; migration, asylum and border control management; and the administration of justice and democratic processes. The full list is provided in Annex I to this report. It can be amended by the European Commission in certain circumstances.[5]

AI systems that are not prohibited or considered high-risk have to comply with certain transparency measures. For example, people must be informed that they are interacting with an AI system.[6] The Act also contains rules, drafted part-way through the negotiations, on “general-purpose AI models with systemic risk.”[7]

The Act raises multiple questions on how it relates to other EU laws, including the Charter of Fundamental Rights and jurisprudence from the Court of Justice of the EU. A number of exemptions appear to clash with existing legislation and case law, for example those relating to:

national security;
the right to explanations on law enforcement use of AI;
restrictions on supervisory authorities’ inspection powers; and
the exclusion of people outside the EU from its scope.

There will likely be a lot of expensive, lengthy and complex litigation in the next few years as attempts are made to resolve these problems.

It can also be observed that the Act contributes to a particular ‘imaginary’ of AI. Amongst the many exemptions in the Act are a number premised on the idea that it may be necessary to deploy AI systems in urgent situations. In the words of the law, deployment of an AI system may be required in “situation(s) of urgency for exceptional reasons of public security or in the case of specific, substantial and imminent threat to the life or physical safety of natural persons.”[8]

This rests on an assumption that there are, or will be, AI systems that can deal with such situations of urgency. The Act does not provide any examples of what such a situation might be, or what a system deployed to deal with it might do. It may well be that no such system exists. The inclusion of the exception almost guarantees that it will be used. At the same time, the exception also serves to further propel the hype that surrounds the development and deployment of AI technology.

In the field of security[9] more specifically, the Act achieves two key things. First, it establishes conditions for increased development and use of security AI systems. Second, it ensures that the development and use of those AI systems is subject to extremely limited accountability, oversight and transparency measures.

Governments played a key role in placing these exemptions, exceptions and loopholes into the law. A policing body, the European Clearing Board (EuCB) has also taken credit for a number of changes resulting from its “advocacy activities regarding law enforcement use of AI”[10] (section 4.1.3). According to a Europol document, lobbying of EU member state governments by the EuCB’s Strategic Group on AI:

… triggered important changes in the Council position on the AI Act, including on the definition, classification of systems, remote biometrics, use of dactyloscopy and exceptions for law enforcement (mandatory publishing of AI-systems in use or that are developed by law enforcement agencies).[11]

This can be seen by looking at the wide array of exemptions embedded in the Act, which can be divided into eight different categories:

scope and application of the law;
(un)prohibited practices;
risk assessment;
transparency;
conformity assessment;
data protection; and
oversight.

(return to top)

2.2 Summary: exceptions and loopholes

Despite promising to protect fundamental rights and ensure that AI systems deployed in the EU are “trustworthy,” the AI Act contains a multitude of exemptions, exceptions and shortcomings in relation to the use of AI systems for policing and migration purposes. These can be summarised as follows:

Scope and application of the law

law enforcement agencies in the EU can make use of AI systems operated by international organisations or non-EU state authorities, to which the AI Act does not apply;
a new concept of “sensitive operational data,” with an unclear definition, is used in the Act, introducing new possibilities for security agencies to avoid oversight and scrutiny;
the use of high-risk AI systems by EU and national authorities is excluded from the scope of the Act until at least 2030, and high-risk systems operated by private companies may be excluded permanently;
the Act does not apply to individuals located outside the EU, meaning that, for example, people applying for visas or authorisations to travel to the EU enjoy none of the protections the Act offers to individuals within the EU (though some redress should be possible through the European Charter of Fundamental Rights or EU data protection law);

(Un)prohibited practices

despite supposed bans on practices such as profiling, biometric categorization, and mass biometric surveillance, law enforcement and migration authorities enjoy numerous exemptions that may enable widespread deployment of these techniques;

Risk and impact assessments

providers of high-risk AI systems are given the possibility to classify those systems as not, in fact, falling into the high-risk category;

A “silicon curtain” of secrecy

a swathe of transparency measures do not apply when AI systems are used for law enforcement purposes, such as:
- the right for individuals to be informed when they are subject to decisions informed by an AI system (however, individuals will have rights under EU data protection law that means they should be informed of any such decisions[12]);
- the right for individuals to know they are interacting with an AI system or being subjected to an emotion recognition or biometric categorization system;
- the right for individuals to know that text, audio, video or images have been generated by an AI system;
- the right for individuals to have explanations about decisions made using AI systems;
- the right for individuals to know that they are being subjected to the testing of AI systems;
an EU-wide database will be created to store information on high-risk AI systems, with information accessible to the public – unless it relates to high-risk systems used by policing and migration agencies, which will be registered in a non-public section of the database, creating opacity over what types of systems are being used by which institutions and agencies;

Conformity assessment

policing and migration authorities are exempt from certain assessment and oversight procedures in cases of “urgency”;
in many cases, security agencies are exempt from the requirement for external oversight, designed to assess compliance with the standards and specifications included in the Act;

Data protection

the fundamental data protection principle of purpose limitation does not apply when personal data is used for testing AI systems in regulatory “sandboxes”;

Oversight

there are serious shortcomings to the procedure for reporting serious incidents caused by AI systems;
the use of AI systems by policing and migration authorities will be overseen by data protection authorities, who are under-funded and severely lacking in resources;
policing and migration authorities appear to be granted the ability to restrict or prevent supervisory authorities from exchanging information about their use of AI systems, and are granted control over supervisory authorities’ access to technical documentation.

In short, the Act will make meaningful supervision of and control over the use of AI systems for policing and migration authorities extremely difficult. In some cases, the rules are designed to prevent the public having any knowledge whatsoever about the use of intrusive and invasive AI systems.

The Act states that AI technologies should “serve as a tool for people, with the ultimate aim of increasing human well-being.”[13] When it comes to the use of AI by police and migration authorities, the aim is clearly to reduce oversight and scrutiny, whilst increasing their powers – something that all too often achieves quite the opposite of “increasing human well-being.”

However, the Act is also clear that it is “without prejudice” to a host of EU and national legal requirements that may help to protect peoples’ rights. The Act itself says it should not affect “existing Union law, in particular on data protection, consumer protection, fundamental rights, employment, and protection of workers, and product safety.”[14] On the face of it, there are some clear clashes with existing law and jurisprudence, particularly in relation to data protection.

What remains to be seen is how parts of the Act that appear to be in tension with other elements of EU law will be implemented in practice and, in particular, how they will be interpreted by the courts. There will likely be a substantial amount of litigation in the years to come as authorities, companies and individuals seek to have aspects of the law clarified.

(return to top)

2.3 In detail: the AI Act’s security exemptions

2.3.1 Scope and application of the law

Military and national security

The text is clear that the Act does not apply to any use of AI systems for “military, defence or national security purposes.” It emphasises that it is the purpose of a particular use of an AI system that matters, and not the agency or institution using it. For example, if the police were using an AI system for “national security” purposes, the AI Act would not apply.

The Act also emphasises that this exemption applies whether or not an AI system is available on the market commercially. For example, a system could be:

developed by a private company and made publicly available for purchase;
developed “in-house” by a state agency; or
developed via some form of public-private partnership, and never made available commercially.

In none of these cases would the Act apply, if the system were used for “military, defence or national security” purposes.

As one group of journalists has put it: “Climate demonstrations or political protests, for instance, could now be freely targeted with AI-powered surveillance if police have national security concerns.”[15] While this may be the hope of some EU governments, the practicalities may be more complicated.

There is a longstanding conflict between EU institutions and EU member states over whether or not matters of “national security” can be in any way regulated by EU law. The academic Plixavra Vogiatzoglou argues that the AI Act’s national security exemption runs counter to EU Court of Justice (CJEU) case law.[16] How the exemption works in practice is likely one of the many questions that will, at some point, be raised before the CJEU.

Data-laundering overseas

The Act applies to providers and deployers of AI systems within the EU and in non-EU states, “where the output produced by the AI system is used in the Union,”[17] but there is a major exemption for law enforcement and judicial cooperation purposes.

If EU or member states bodies make use of AI systems “in the framework of international cooperation or agreements for law enforcement and judicial cooperation” with the authorities of non-EU states or international organisations, the Act does not apply. The third country or international organisation must provide “adequate safeguards with respect to the protection of fundamental rights and freedoms of individuals.”[18]

However, there is no explanation of what those “adequate safeguards” are. Nor does the text explain who is to judge the adequacy of any safeguards. EU data protection law applies when law enforcement agencies transfer personal data to international organisations. However, in certain circumstances, law enforcement agencies can themselves determine whether or not “adequate safeguards” exist.[19] There are thus many questions that arise as to how transfers of data for use in non-EU AI systems will be assessed and supervised.

One international organisation developing AI technology is the International Police Organisation, better known as Interpol. At Interpol’s September 2023 annual conference, then-Secretary General, Jürgen Stock, told delegates about plans to “harness the power of artificial intelligence.” This is being done is through an “analytical platform” called INSIGHT, which is ultimately supposed to provide “visual, video, audio recognition, facial and bio-data matching,” for “advanced and predictive analytics.” European states, amongst others, are funding the development of the platform.[20]

The INSIGHT platform does not have to comply with the AI Act. Although Interpol has its headquarters in Lyon, France, it is an international organisation. Data shared with Interpol by EU member states, or agencies such as Europol, could be processed in the INSIGHT platform in breach of the AI Act’s rules. The outputs could then be used within the EU, without breaching the law. Another potential avenue for this kind of data-laundering is through data exchanges with the USA, where the Department of Homeland Security is building a vast database and analytical system known as Homeland Advanced Recognition Technology System (HART).[21]

Geographical discrimination

The Act “does not apply to affected persons outside the Union.” The protection it offers to individuals stops at the EU’s borders. Given that the EU is developing AI systems specifically designed to be used to profile and analyse vast numbers of people located outside its territory – primarily applicants for visas and travel authorisations – this exemption should be of major concern.

For example, the Act gives individuals the right to obtain “clear and meaningful explanations” of an AI system’s role in any decision which has “legal effects or similarly significantly affects that person in a way that they consider to have an adverse impact on their health, safety or fundamental rights.” If an individual is located outside the EU and affected by an AI system, they will have no right to any explanation. However, there is a clash here with other laws: the right to explanations may be available under data protection law[22] or from the right to effective remedy in the EU Charter of Fundamental Rights.[23]

There are also geographical exemptions within the EU. The Irish government has decided to opt out of certain provisions of the Act related to police and judicial cooperation. This includes prohibitions on AI-powered risk assessments of individuals, the use of mass biometric surveillance, and biometric categorisation systems. However, any measures taken by Ireland in these areas will still have to comply with, for example, EU data protection law and the Charter of Fundamental Rights.[24]

Research and testing

The Act “does not apply to any research, testing or development activity regarding AI systems or AI models prior to their being placed on the market or put into service,” although this does not apply to “testing in real world conditions” (examined further below). Furthermore: “Such activities shall be conducted in accordance with applicable Union law,”[25] for example, on data protection.

It may be questioned whether the provision excluding “research, testing and development activity” from the scope of the Act could be used to deliberately bypass other parts of the law. Authorities could claim that they are deploying high-risk AI technologies for research and development purposes. This would negate the need to meet the obligations the Act places on providers and deployers.

As Border Violence Monitoring Network have highlighted, a number of EU-funded security research projects have been tested in Greece. These have primarily focused on border surveillance and the integration and sharing of data generated by border surveillance.[26] Deployments have included drone flights in the Orestiada region, the testing of surveillance technologies designed to see through foliage in Evros, and the construction of pylons for the gathering and transmission of surveillance data, also in Evros.

As at other land borders in Europe, more border surveillance in Greece is likely to support the widespread (if not systematic) and flagrantly illegal practice of violently “pushing back” refugees. The question here, however, is whether such deployments would be covered by the AI Act’s provisions on “real world” testing, or would escape its provisions by being classified as “research, testing and development activity.”

Temporary general exemptions from the Act

The Act includes two temporary, general exemptions. The first concerns AI systems that are used as part of various large-scale EU information systems. The second concerns high-risk AI systems that are “placed on the market or put into service” before 2 August 2026, the date from which the Act comes into force. The exemptions provide carte blanche for all manner of practices until the end of the decade, at both EU and national level.

Large-scale EU information system put into use before 2 August 2027 do not have to comply with the Act until 31 December 2030.[27] The information systems in question are:

the Schengen Information System (SIS);
the Visa Information System (VIS);
Eurodac;
the Entry/Exit System (EES);
the European Travel Information and Authorisation System (ETIAS);
the European Criminal Records Information System on third-country nationals and stateless persons (ECRIS-TCN);
the information systems that establish the “interoperability” architecture, namely:
- the Common Identity Repository (CIR);
- the shared Biometric Matching System (sBMS);
- the Multiple Identity Detector (MID); and
- the European Search Portal (ESP).[28]

All these systems are intended to make use of some form of technology covered by the Act: for example, biometric identification. Perhaps the most invasive use of AI announced so far, however, will be in the ETIAS and the VIS. These systems will use algorithmic profiling to determine whether individuals are believed to pose a “security, illegal immigration or high epidemic risk.”[29]

A similar temporal exemption applies to operators of high-risk AI systems placed on the market or put into use before 2 August 2026.[30] If there are “significant changes” to the design of those systems after that date, they will be subject to the Act. Furthermore, all high-risk systems used, or intended to be used, by public authorities, must comply with the Act by 2 August 2030. However, private operators of high-risk AI systems are not mentioned here, and thus seem to be excluded from the requirement to comply with the Act by 2 August 2030.

“Sensitive operational data”

The Act introduces a new term into EU law: “sensitive operational data.” This is defined as: “operational data related to activities of prevention, detection, investigation or prosecution of criminal offences, the disclosure of which could jeopardise the integrity of criminal proceedings.”[31]

The phrase appears nowhere else in EU law, apart from a statement accompanying a proposed law on greenhouse gas emissions, where it has a different meaning.[32] The definition itself relies upon another term, “operational data,” that is not defined in the Act, but can evidently be taken to mean data relating to operations: for example, criminal investigations.[33]

It is unclear if “sensitive operational data” is somehow related to “sensitive data,” defined in EU data protection law as:

…data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership… genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person’s sex life or sexual orientation.”[34]

The lack of definition of “sensitive operational data” causes a problem: who gets to decide what is sensitive operational data, and what is not? Does it include “sensitive data” as defined in EU data protection law, or is it something separate? This is not merely a question of semantics: the AI Act includes various provisions that prevent supervisory authorities having access to sensitive operational data, examined further below.

It should be noted that the term “sensitive operational data” was not included in the original proposal for the Act.[35] It is unclear when or by whom it was inserted in the text, but it is certainly beneficial for law enforcement and other security agencies.

(return to top)

2.3.2 (Un)prohibited practices

The EU has loudly promoted the fact that certain uses of AI are banned under the Act. However, there are multiple exemptions to those bans. These exemptions cover the use of AI systems for profiling, biometric categorization, and mass biometric surveillance, known formally as “remote biometric identification.” The best-known example of such a technique, but far from the only one, is the use of facial recognition systems in public spaces. Furthermore, while there is a ban on certain uses of emotion recognition systems, it has important limits.

Profiling

The Act bans systems that try to calculate the likelihood that someone will commit a criminal offence, where that calculation is based solely on “the profiling of a natural person” or “assessing their personality traits and characteristics.” However, the ban is not absolute: these systems are legal if “used to support the human assessment of the involvement of a person in a criminal activity, which is already based on objective and verifiable facts directly linked to a criminal activity.”[36]

Reversing this statement makes it somewhat clearer. If there are “objective and verifiable facts directly linked to a criminal activity” that show a person is involved in criminal activity, a system can be used to profile them, or to assess that person’s “personality traits and characteristics.” That assessment can be used to support human decisions about the risk the person will commit a criminal offence. In short: these AI techniques cannot be applied to large datasets to single out individuals, but can be applied to individuals who have already been singled out.

It should be noted that the prohibition only extends to the use of profiling and personality assessment in relation to criminal activity – it does not apply to immigration, asylum or border control. There is thus no ban on using AI systems for profiling or personality assessments in those policy fields. The Protect Not Surveil coalition[37] sought to extend the ban to cover those areas, but was unsuccessful. The European Data Protection Supervisor has argued that the exclusion of administrative offences from the ban is unconvincing, in light of EU case law.[38] A set of EU guidelines designed to provide definitive explanations on this point is forthcoming. It remains to be seen what they say.[39]

Biometric categorization

Biometric categorization is defined by the Act as a process for automatically assigning people “to specific categories on the basis of their biometric data.” This might mean, for example, using a system to classify people by gender, eye colour, height, or some other measurable characteristic. Under the Act, it is illegal to use biometric categorization “to deduce or infer… race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation.”[40]

However, the ban “does not cover any labelling or filtering of lawfully acquired biometric datasets, such as images, based on biometric data or categorizing of biometric data in the area of law enforcement.”[41] The wording here is unclear, meaning the statement could be read in two ways:

it is legal to label or filter, for law enforcement purposes, lawfully acquired datasets that are based on biometric data or on the categorization of biometric data; or
the labelling or filtering of lawfully acquired biometric datasets, such as images, based on biometric data, is legal, as is the categorizing of biometric data in the area of law enforcement.

Like much of the labyrinthine wording of the Act, this lack of clarity may be due to the “3-day ‘marathon’ talks”[42] that led to its adoption. This might be better-described as a 72-hour negotiation binge. The Spanish version of the text, which uses the word ni (“nor”, rather than “or” in the English version) indicates that the second interpretation is correct.[43]

The upshot is that law enforcement agencies are excluded from complying with a ban on a practice that is otherwise deemed to be inherently dangerous for “health, safety and fundamental rights.”[44] However, the EDPS has warned that the provision “should not be interpreted as a general derogation,” but instead as clarification that “there are other types of biometrics categorisation for different legitimate purposes.”[45]

Nevertheless, the Act may provide EU legal backing for controversial practices that have been going on for some time, such as in the Czech Republic.[46] Europol makes extensive use of facial recognition software (examined in section 3.3.3). Governments have sought to introduce new powers so the police can cross-reference videos and photos with databases, as in Denmark[47] and Sweden.[48]

In Germany, the government introduced plans to legalise police searches of publicly-available data on the internet using facial recognition software.[49] The European Commission reportedly warned the German government that this would be illegal.[50] With elections forthcoming in Germany, the future of the proposals is unclear.

Mass biometric surveillance

Mass biometric surveillance – “remote biometric identification,” in the words of the law – is one of the most controversial topics covered by the AI Act. The Reclaim Your Face coalition sought a comprehensive ban on the practice.[51] The European Commission itself recognises that the use of systems to automatically identify individuals in public, at a distance and without their knowledge, undermines “personal data, privacy, autonomy and dignity,” as well as “freedom of expression, association and assembly… resulting in a chilling effect on democracy.”[52]

The Act prohibits the use of “remote biometric identification systems in publicly accessible spaces for the purposes of law enforcement.”[53] This of course allows the use of mass biometric surveillance for purposes other than law enforcement – for example, for border control or in the workplace. The law also contains three specific exemptions to the supposed prohibition:

targeted searches “for specific victims of abduction, trafficking in human beings or sexual exploitation of human beings”;
searches for missing persons;
preventing “a specific, substantial and imminent threat to the life or physical safety of natural persons”;
preventing “a genuine and present or genuine and foreseeable threat of a terrorist attack”; and
“the localisation or identification of a person suspected of having committed a criminal offence,” with regard to a specific set of offences.[54]

If a member state wishes to make use of these exemptions, it must pass national legislation authorising that use. The legislation must include detailed rules on “the request, issuance and exercise of, as well as supervision and reporting related to the authorizations,” and specify for which of the offences in the Act mass biometric surveillance can be deployed. The Act also states that member states can, if they wish, introduce “more restrictive laws on the use of remote biometric identification systems.”[55]

Two EU member states planning new legislation to introduce mass biometric surveillance include Sweden[56] and the Czech Republic, where a proposal to allow facial recognition in airports seeks to sidestep even the limited protections of the Act.[57]

Member states must notify the Commission of any national rules on this topic, though there is no obligation for the Commission to publish that information. This means individuals travelling from one member state to another may not be able to find out if and how they may be subjected to mass biometric surveillance.

The Act also sets out a number of other requirements that must be considered by the deploying authorities. Usage of a mass biometric surveillance system must take into account the seriousness, probability and scale of harm if the system is not used; and the consequences that usage of the system will have for the individual rights and freedoms. [58]

There must also be “necessary and proportionate safeguards and conditions” in place. As well as national legislation, there must be limitations on the length of time the system can be used for, the locations in which it can be used, and the persons to whom it may be applied.[59]

According to the Act, the deploying authority must carry out a fundamental rights impact assessment. European Court of Human Rights jurisprudence on public facial recognition systems requires a high level of justification for mass biometric surveillance to be considered “necessary in a democratic society.”[60]

The European Data Protection Supervisor (EDPS) has also remarked that member states will need to assess whether “the purpose pursued cannot be achieved without using biometric data,” and that the “fundamental values of democratic systems” need to be taken into account.[61] It is doubtful whether mass biometric surveillance in any form is compatible with democratic norms, though this view does not seem to be shared by most politicians and officials.

Deploying authorities should also register the system in the EU-wide database for high-risk AI systems. This database is governed by the AI Act and, as noted below, there are numerous exemptions for law enforcement authorities. One exemption allows them to deploy mass biometric surveillance systems without registering them in the database in “duly justified cases of urgency,” provided that registration is completed “without delay.”[62]

A similar law enforcement exemption applies to judicial or administrative authorization for deploying a mass biometric surveillance system. Deployments are supposed to have prior authorization by such an authority, but this requirement can be ignored in a “duly justified situation of urgency.” Authorization must however be requested within 24 hours of deployment. If it is denied then use of the system must be halted, and any outputs resulting from that use deleted.[63]

“Each use” must also be notified to the relevant supervisory authority, although it is not clear how a single “use” is to be defined. The national supervisory authority for law enforcement, migration, asylum, border control and judicial authorities will be the national data protection authority.[64]

That supervisory authority must send an annual report to the European Commission detailing any authorizations or refusals for the use of mass biometric surveillance, and the Commission has to publish an annual report based on that information. This is likely to mean that the public will get a sanitised, partial version of the information compiled by national supervisory authorities. Indeed, the Act states explicitly that the Commission’s annual reports cannot include “sensitive operational data” from national authorities.

Emotion recognition

The Act bans AI systems used to “infer emotions of a natural person in the areas of workplace and education institutions.” Outside of these two areas, therefore, emotion recognition systems can be deployed – for example, for policing or border control purposes. They are classified as high-risk when used for any other purpose, though law enforcement use of emotion recognition systems may be kept secret, thanks to an exemption in the law (section 3.3.4).

The EDPS has highlighted that the reason for banning emotion recognition in workplaces and educational institutions is “the imbalance of power.” This, the authority noted, “is even more applicable and relevant in the law enforcement context. The same consideration applies also in the field of border control, migration and asylum.”[65] This view was evidently not shared by EU legislators.

(return to top)

2.3.3 Risk and impact assessments

Self-assessment

As noted above, the AI Act is premised on a system of risk assessment, with differing levels of control and oversight applying depending on the level of risk posed by a particular AI system. As this analysis makes clear, that level control and oversight can be substantially reduced when the authority in control is responsible for law enforcement, immigration, border control or asylum. The Act also allows providers of nominally high-risk systems to assess them as not, in fact, being high-risk. That means, in turn, that certain safeguards do not apply.

As the Act puts it, systems that are included in the high-risk list[66] are not be considered high-risk if they do not pose “a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making.” This is the case when one any of four conditions is met:

the AI system is for performing a narrow procedural task;
the AI system is for improving the result of a previously completed human activity;
the AI system is for detecting patterns and deviations in decision-making, but not replacing or influencing any previously-completed human assessment, without “proper human review”; or
the AI system is for performing a preparatory task in an assessment that is “relevant” for any of the purposes of a system defined as high-risk.

The Act then goes on to introduce an exemption to these exemptions: any AI system that performs profiling of natural persons must be considered high-risk, regardless of whether or not it meets one of the four conditions listed above.

If a provider of a high-risk system determines that their system is not in fact high-risk, they must document that assessment before they put the system on the market or put it into use. The assessment must be made available to national supervisory authorities upon request, and the information must also be registered in the EU database for high-risk systems. Guidelines for the “practical implementation” of these provisions must be published by the European Commission by 2 February 2026.

Impact assessments

Deployers of high-risk AI systems are obliged to carry out a fundamental rights impact assessment (FRIA) prior to deployment. That assessment has to include:

“a description of the deployer’s processes in which the high-risk AI system will be used in line with its intended purpose”;
the time period for use of the system, and the frequency with which it will be used;
“the categories of natural persons and groups likely to be affected by its use in the specific context”;
the “specific risks of harm” likely to affect those categories of people and groups;
how human oversight measures specified in the instructions for the system will be implemented;
a description of action to be taken in case any of the identified risks materialise, including arrangements for internal governance and complaint mechanisms.[67]

A FRIA has to be conducted for the first deployment of a high-risk AI system. The deployer can use the same FRIA for “similar cases,” if they wish. The FRIA must be updated by the deployer if any of the requirements listed above “has changed or is no longer up to date.”[68]

The completed FRIA must then be submitted to the relevant oversight authority – presumably for review and assessment, though this is not specified in the Act. It is worth noting that one analysis of 10 different FRIAs for AI systems found sharp divergences in their “length and completeness,” with serious questions for their utility and meaningfulness.[69]

Such an assessment should also, presumably, include an assessment of the necessity and proportionality of using the AI system. These are general principles of EU law,[70] and are mentioned multiple times in the preamble to the AI Act, particularly in connection with law enforcement use of AI systems.

If personal data is processed in an AI system, deployers will also have to conduct a data protection impact assessment, in accordance with EU data protection law. Under the AI Act, the FRIA “shall complement the data protection impact assessment,” if the latter assessment already meets any of the obligations set out in the Act.[71]

The deploying authority may be exempt from the requirement to submit the FRIA to the oversight authority. If the deployer has invoked the derogation from the conformity assessment procedure,[72] they are also exempt from the obligation to send the FRIA to that authority. The derogation from the conformity assessment procedure is intended to be temporary, “for exceptional reasons of public security or the protection of life and health of persons, environmental protection or the protection of key industrial and infrastructural assets.” Presumably the FRIA must be submitted once the derogation ends, though the Act does not specify if this is the case or not.

(return to top)

2.3.4 A “silicon curtain” of secrecy

A swathe of the Act’s exemptions and exceptions for law enforcement, immigration, border control and asylum authorities relate to transparency. While most deployers of high-risk AI systems are required to provide certain types of information to the public when they are subjected to those systems, security authorities do not.

During the Cold War, the phrase “iron curtain” was used to describe the physical and political barriers between eastern and western Europe. The AI Act could be described as introducing a “silicon curtain” of secrecy, designed to prevent the public and elected officials from knowing when, where or how security AI is being used.

EU database of high-risk AI systems

The Act mandates the establishment of an EU-wide database for high-risk AI systems. The majority of information in this database is to be made public, to “increase the transparency towards the public… allowing the general public to find relevant information [on] the registration of high-risk AI systems and on the use case of high-risk AI systems.”[73]

However, biometric systems, emotion recognition systems, and other high-risk AI systems “in the areas of law enforcement, migration, asylum and border control management” are exempt from a number of the requirements related to this database.

Providers of two categories of AI systems, or their authorised representatives, must register information on themselves and their high-risk systems in this database. They must do so when:

an AI system is classified as high-risk; or
the provider has determined the AI system is not high-risk according to Article 6(3).[74]

When a national authority, an EU institution, office, body or agency, or persons acting on behalf of national or EU authorities are planning to use a high-risk AI system, they must take two steps. Firstly, they have to register themselves in the database as users of a high-risk system. Then they have to select the system they plan to use. In this way, a record of the providers, deployers, and the systems they have created or used is put in place.

However, different rules apply to providers of such systems for law enforcement, migration, asylum and border control purposes. They are entitled to register the systems in a “secure non-public” section of the EU database, and only have to provide a limited amount of information compared to other registrants (see Annex II for more detail).

Notably, they do not need to provide information on “the information used by the system (data, inputs) and its operating logic,” amongst other things. The “secure non-public” section of the EU database will only be accessible by the European Commission, the European Data Protection Supervisor, and national supervisory authorities.

There will thus be no increase in “transparency towards the public” regarding high-risk AI systems used for law enforcement and migration purposes – in fact, quite the opposite. There are certainly arguments for keeping details of particular police investigations secret. The means and methods available to the police for detecting and investigating crime, however, should be publicly-available information, so that it can be the subject of democratic scrutiny and public debate.

As noted above, this secrecy appears to be the result of lobbying by police officials, through the European Clearing Board. The body has claimed credit for changing the Council of the EU’s negotiating position on “mandatory publishing of AI-systems in use or that are developed by law enforcement agencies.”[75]

It is noteworthy that in the USA, police are deliberately obscuring their use of facial recognition technology, “which means defendants are being deprived of their constitutional right to challenge the veracity of the evidence being used against them.”[76] In the EU, the AI Act offers various new means police and immigration authorities to keep the use of AI technology secret, including by exempting them from public registration in the database.

The right to information

Where high-risk AI systems are used to make decisions that affect individuals, those individuals must be informed by the system’s deployer.[77] However, national law enforcement agencies using high-risk systems are subject to a different legal framework, dealing with data protection in national law enforcement agencies.[78] This allows them to delay, restrict or omit providing information to affected individuals, for example to “avoid obstructing official or legal inquiries, investigations or procedures.”[79]

People may also directly interact with AI systems. The most obvious example of this would be chatbots deployed for customer service on websites. Whatever the situation, providers must ensure that people are made aware of this, unless it is “obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect, taking into account the circumstances and the context of use.”[80] Quite how the concept of a “well-informed, observant and circumspect” person might be interpreted is open to question. There is an extensive literature on the interpretation of the similar legal fiction of a “reasonable person.”[81]

However, the obligation to inform people they are interacting directly with an AI system does not apply to “AI systems authorised by law to detect, prevent, investigate or prosecute criminal offences, subject to appropriate safeguards for the rights and freedoms of third parties.”[82] The only exception to this is when an AI system is available to members of the public to report crime. The Act includes an equivalent exemption for emotion recognition and biometric categorisation systems.[83]

As with many similar provisions examined below, it can be inferred from this that the police wish to use various types of AI systems without the public’s knowledge. At the very least, it leaves the possibility open for them to do so. There is no further detail in the Act on what “appropriate safeguards” might be.

Watermarking of AI-generated media

Providers of AI systems used to generate “synthetic audio, image, video or text content” must ensure the outputs are “marked in a machine-readable format and detectable as artificially generated or manipulated.”[84] However, this does not apply when those systems are “authorised by law to detect, prevent, investigate or prosecute criminal offences.” There is no requirement set out for “appropriate safeguards.” Like the previous exemption, the article indicates that the police intend to make operational use of AI-generated media.

Notification that text has been generated by AI

Deployers of AI systems that generate or manipulate text used for “informing the public on matters of public interest” – presumably, news and other media articles – have to disclose if the text has been artificially-generated and/or manipulated. However, this is not required if the text has been subject to “human review or editorial control and where a natural or legal person holds editorial responsibility for the publication of the content.” Nor does the rule apply “where the use is authorised by law to detect, prevent, investigate or prosecute criminal offences.”[85]

The article indicates that police intend to make operational use of AI-generated text, and there is no explicit requirement for appropriate safeguards or for use by law enforcement to be in accordance with EU law – though, again, this does not mean that EU law does not apply.

Right to explanations

The Act gives individuals the right to obtain from the deployer of a high-risk AI system “clear and meaningful explanations of the role in the AI system” in any decisions that have produced “legal effects or similarly significantly affects that person.” Importantly, this is based on the individual’s opinion, not any external assessment: there is a right to an explanation when someone considers an AI system has had “an adverse impact on their health, safety or fundamental rights.” This, of course, only applies to people located within the EU, as with all the other such safeguards in the Act.[86]

However, if EU or national law contains “exceptions from, or restrictions to” the provision of the AI Act setting out the right to explanations, it does not apply. It is unclear whether any such restrictions or exemptions already exist, or whether this provision is laying the foundation for future exceptions. In either case, the AI Act makes it possible for governments to exempt law enforcement, asylum, immigration, border and judicial authorities from the right to provide explanations to individuals about the use of AI systems.

The CJEU has already warned of the potential of AI to deprive individuals of their right to an effective judicial remedy. In a judgment on the EU’s travel surveillance law, it found that the “opacity” of AI technology “might make it impossible to understand the reasons why a given program led to… a result.”[87] People affected by technological procedures must be able to understand “how those criteria and those programs work” to allow them to decide “with full knowledge of the relevant facts” whether or not to challenge the unlawful and indiscriminatory nature of these criteria.”[88]

The Act also appears to clash with article 41 of the Charter of Fundamental Rights. This provides for a right to good administration, and places an obligation on the administration “to give reasons for its decisions.” How this will work in the context of technically-complex AI systems remains to be seen.[89]

Information on and consent to being subjected to “real-world testing”

Most people would probably not want to be the subject of experiments by state authorities. Even fewer would want to be the subject of experiments that take place without their knowledge. Yet the AI Act permits precisely that: it gives law enforcement authorities the permission to test experimental AI systems on individuals without their knowledge, provided certain conditions are met.

The Act contains rules on how testing in “real world conditions” should take place.[90] Such testing can be undertaken by providers or prospective providers of systems “at any time before the placing on the market or the putting into service of the AI system.” Testing can be done by providers “on their own or in partnership with one or more deployers or prospective deployers.” For example, a company producing AI systems could team up with a public authority, such as a police force, to test an AI system.

To do so, the provider or prospective provider must be legally registered in the EU or have an authorised representative in the EU. They must put in place “appropriate and applicable” safeguards for any personal data processed during the testing that will be transferred to a third state. For example, a provider based in the US might have an authorised representative in the EU. Personal data processed during testing could then be transferred to the US, provided adequate safeguards exist. The provider or prospective provider must also be registered in the EU-wide database of AI systems and providers. They have to draw up a plan for the test, and that plan has to be approved by the relevant supervisory authority.

Individuals subjected to the tests who are “vulnerable” due to “age or disability” must be “appropriately protected” (a provision that excludes other forms of vulnerability), and test subjects must have provided informed consent to the testing. The testing cannot last longer than six months, and it must be possible for any outputs from the AI system being tested to be “effectively reversed and disregarded.” Furthermore, “the testing itself and the outcome of the testing in the real world conditions shall not have any negative effect on the subjects.” All personal data processed must be deleted “after the test is performed.”

Supervisory authorities must ensure that real world testing is carried out in accordance with the law via inspections and checks, though it is not mandated that they do so for every instance of testing. Supervisory authorities must also be informed when testing has finished, and of the results. Providers or prospective providers of the AI system being tested are liable for any damage caused during the tests.

All instances of real-world testing must be registered in the EU-wide database. However, there is a presumption of secrecy with regard to the public. Information will only be accessible to supervisory authorities and the Commission, “unless the prospective provider or provider has given consent for also making the information accessible the [sic] public.”[91]

Information on real world testing for these systems and purposes will likely never be made public. This is due to the transparency exemptions in place for the registration in the EU database of systems for (1) remote biometric identification, biometric categorisation and emotion recognition; and of (2) high-risk AI systems for law enforcement, migration, asylum and border control.

Furthermore, the information that has to be registered by providers or prospective providers regarding (1) and (2) is more limited than for other types of system. There is an exemption from the requirement to register “a summary of the main characteristics of the plan for testing in real-world conditions.”[92] This makes supervisory authorities’ work more complicated, as they will have to go through extra steps to obtain the relevant information (see below). Testing for the purposes of law enforcement is also exempt from the requirement to seek and obtain informed consent from test subjects, if doing so “would prevent the AI system from being tested.”

Even if it is the case that an AI system being tested in “real world conditions” is prohibited from having any negative effects on individuals, and that personal data processed for testing must be deleted as soon as the test has ended, these provisions add a new level of secrecy and opacity to the law enforcement use of AI systems. There is no reasonable justification for allowing law enforcement agencies to test AI systems on the public without their knowledge. Turning the techniques used to detect and investigate crime into state secrets simply increases police impunity.

(return to top)

2.3.5 Conformity assessment

Conformity assessment is the process by which an authority assesses whether a product, material, service or process complies with particular standards. Standards are described by the International Standardisation Organisation as “a formula that describes the best way of doing something.”[93] They govern all manner of things – components for machinery or vehicles, cybersecurity requirements, and food safety, amongst others.

The AI Act introduces obligations for the development of certain new standards that AI systems must comply with, “prior to their placing on the market or putting into service.”[94] The extensive reliance upon standards has been criticised by Corporate Europe Observatory. Companies are lobbying hard to influence the AI Act standards, to try to have them meet corporate preferences. There are also broader questions to be asked about relying on standards to define and guide issues concerning fundamental rights.[95]

Under the Act, the European Commission must issue “standardisation requests” for all the requirements related to high-risk AI systems.[96] Such requests would be sent to the European Committee for Standardisation (CEN), the European Committee for Electrotechnical Standardisation (CENELEC), or the European Telecommunications Standards Institute (ETSI).

These bodies would then be responsible for developing the standards in question. Developers or providers of high-risk AI systems would demonstrate conformity with those standards to demonstrate compliance with the Act. National conformity assessment bodies would, in some cases, be responsible for deciding whether a system is compliant or not. In the case of policing, migration or judicial agencies, however, data protection authorities would undertake the assessment procedure,[97] at both national and EU level.[98]

A standardisation request made by the European Commission may not be met. The Act notes four situations in which this could happen:

the request is not accepted by a standardisation organisation;
the standards are not provided within the required deadline;
the standards do not comply with fundamental rights requirements; or
the standards do not comply with the request.[99]

In such cases, the Commission is able to draw up implementing acts.[100] These are used to support the uniform implementation of EU law by member states.[101] These acts would contain “common specifications” serving the same purpose as standards. However, even if a high-risk AI system does not comply with these specifications, the provider must adopt “technical solutions” that are “at least equivalent” to them.[102]

External oversight

There is a choice of conformity assessment procedures for providers of remote biometric identification, biometric categorization, or emotion recognition systems that rely on either harmonised standards[103] or common specifications.[104] These are:

internal control, set out in Annex VI of the Act; or
assessment by a conformity assessment body, set out in Annex VII of the Act.

The internal control procedure is a self-assessment procedure. It means providers must establish a quality management system (meeting at least 13 different requirements set out in the Act[105]) and produce a range of technical documentation demonstrating compliance with the requirements for high-risk systems. They also have to verify that the system’s design and development, and its “post-market monitoring,”[106] are consistent with that documentation.[107]

Alternatively, providers can request that a conformity assessment body examine whether they comply with the relevant standards or specifications. In certain cases, they are obliged to do so: if standards have only been partially applied by the provider or do not exist at all; if common specifications have not been applied or are not available; or if a standard contains restricted information, and their conformity with the restricted part of the standard requires assessment.

However, providers of high-risk AI systems for police, migration or judicial purposes are obliged to follow the self-assessment procedure,[108] meaning there is no external oversight of whether or how they conform with relevant standards and requirements. The Commission can change these rules, if the procedure is not deemed effective at preventing or minimising risks to health, safety or fundamental rights.[109]

Ongoing supervision

A high-risk system must undergo a new conformity assessment procedure (even if it has undergone one before) where there is a “substantial modification” to the system. This new assessment must take place even if the system is not to be further distributed, or will continue to be used by the current deployer.

However, there is an exemption to this for high-risk AI systems that “continue to learn after being placed on the market.” That learning shall not be considered as a “substantial modification,” provided that the system’s original conformity assessment and technical documentation set out how it would learn and change.

Urgent authorisation and deployment without authorisation

There are numerous exemptions to the conformity assessment procedures for high-risk AI systems. Market surveillance authorities – data protection authorities, in the case of policing, migration or judicial agencies – can authorise, for a limited period, the deployment of “specific high-risk AI systems… for exceptional reasons of public security or the protection of life and health of persons, environmental protection or the protection of key industrial and infrastructural assets.”[110] However, a full authorisation can only be issued if the authority determines that the system in question meets the Act’s requirements for high-risk AI systems.

As explained above, high-risk AI systems used for policing, migration or judicial purposes are exempt from conformity assessment procedures involving external oversight (with the exception of remote biometric identification, biometric categorisation or emotion recognition systems). It is unclear if this means they can also self-authorise “urgent” deployments. However, as exceptions to the law must be interpreted strictly,[111] this seems unlikely. The Act also seems to indicate that urgent deployments of whatever type of system require a request to and authorisation by the oversight authority.[112]

When there is this type of “urgent” deployment of a high-risk AI system, conformity assessment procedures must be carried out “without undue delay.” An authorisation must be issued if the system complies with the relevant requirements.

A further exemption allows “law-enforcement authorities or civil protection authorities” to deploy “a specific high-risk AI system” without any kind of authorisation. If they do so, authorisation must be requested “during or after the use without undue delay.” If it is refused, “the use of the high-risk AI system shall be stopped with immediate effect and all the results and outputs of such use shall be immediately discarded.”[113] There is no equivalent provision for the procedure that allows deployment via urgent authorisation.

In either case, the Commission and EU member states must be informed of any authorisations issued in this way, and can object to it. However, they must do so within 15 days of being notified of the authorisation. If they do not object, the authorisation is to be considered justified. In the case of objections, the Commission must enter consultations with the member state in which the system has been deployed, after which it must issue a decision on whether the authorisation is justified or not. If deemed unjustified, the market surveillance authority has to withdraw the authorisation.

It is noteworthy that individuals or organisations cannot object to the “urgent” deployment of high-risk systems – this is a privilege reserved for state authorities or the European Commission.[114] A further observation can also be made. These provisions clearly benefit security agencies in a very direct way, by exempting them from requirements (however limited) to ensure the safety and fundamental rights compliance of high-risk AI systems. There is, however, more to it than this.

Like many other provisions in the Act, these articles contribute to a particular ‘imaginary’ of AI. The very fact that such exceptions exist implies that AI systems will be developed that are capable of dealing with “situation(s) of urgency for exceptional reasons of public security or in the case of specific, substantial and imminent threat to the life or physical safety of natural persons.”[115] From this angle, it would appear that the law itself seeks to further propel the hype that surrounds the development and deployment of AI.

(return to top)

2.3.6 Data protection

Purpose limitation

Purpose limitation is a fundamental data protection principle. It requires that personal data be “collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes.” There may be exceptions to this: for example, an individual can give their consent for their personal data to be processed for another purpose, and EU data protection law provides exemptions for “archiving purposes in the public interest, scientific or historical research purposes or statistical purposes.”[116]

The AI Act introduces further exceptions: personal data collected for one purpose can be used for developing, training and testing AI systems in a “regulatory sandbox,” if certain conditions are met:

the AI system being tested must be for “safeguarding substantial public interest by a public authority or other natural or legal person” in one or more of five areas, including “public safety” and the “efficiency and quality of public administration and public services”;
the personal data used to train the system must be needed to comply with one or more of the requirements for high-risk AI systems, and can only be used if “anonymised, synthetic or other non-personal data” will not suffice;
there must be monitoring mechanisms in place to identify and mitigate any risks to the data subjects;
it must be possible to halt the data processing, if necessary;
personal data must be stored separately to other data and only accessible to authorised persons;
data can only be shared in accordance with EU law and any personal data generated in the sandbox cannot be shared further;
the processing of personal data in the sandbox cannot lead to measures or decisions affecting the data subjects;
there must be data security measures in place, and personal data must be deleted once use in the “sandbox” is over, or the data retention period comes to an end;
logs must be kept for duration of the participation in the sandbox, unless EU or national law permits otherwise;
the rationale and results must be kept, alongside other technical documentation relating to the testing process; and
a short summary of the project must be published on the website of the competent authorities.

These requirements also apply to law enforcement authorities undertaking tests in a “sandbox,” with the exception of the requirement to publish a short summary of the project online. This reinforces the general secrecy surrounding law enforcement use of AI allowed by the Act. Just as the public is not to be told if the police are testing an AI system on them, neither are they permitted to know when a system is being tested in a “regulatory sandbox.”

(return to top)

2.3.7 Oversight

Reporting of serious incidents

The Act includes provisions dealing with the reporting of serious incidents. A serious incident is defined as “an incident or malfunctioning of an AI system that directly or indirectly” causes:

the death of a person, or serious harm to a person’s health;
a serious and irreversible disruption of the management or operation of critical infrastructure;
the infringement of obligations under Union law intended to protect fundamental rights; or
serious harm to property or the environment.[117]

When a serious incident occurs, the Act requires that “providers of high-risk AI systems placed on the Union market” must report it to the supervisory authority in the member state(s) where the incident occurred. A report must also be filed in case of “a widespread infringement,” though this term is not defined. After a report is filed, the provider must conduct an investigation and take corrective action to address the problem.

These requirements are limited to “providers of high-risk AI systems placed on the Union market.” There are separate, and more limited, requirements regarding high-risk systems that are “placed on the market or put into service by providers that are subject to Union legislative instruments laying down reporting obligations equivalent to those set out in this Regulation.” What those legislative instruments are is not explained, and this type of report is limited to serious incidents concerning “the infringement of obligations under Union law intended to protect fundamental rights.”[118]

On the face of it, the use of the phrase “incident or malfunctioning” would appear to exclude any serious incidents caused by the normal functioning of an AI system, though this depends on how the word “incident” is interpreted. It remains to be seen how these reporting systems work in practice. However, as there is no requirement for the Commission or any other supervisory authority to publish any information regarding serious incidents, the public may not get to hear much about it.

Specific supervisory authorities

In EU member states, most supervision of AI systems and their use will be carried out by market surveillance authorities. However, national data protection authorities will take on that role for high-risk systems for law enforcement, migration, border control, asylum, the administration of justice or democratic processes; and remote biometric identification, biometric categorisation and emotion recognition systems used for those purposes.[119] The European Data Protection Supervisor will take on this role in the case of EU institutions, bodies, offices and agencies.[120]

Given that data protection authorities at a national and EU level are already responsible for monitoring how law enforcement and other such authorities process personal data, including by using advanced technologies, this may well make sense. However, data protection authorities are also notoriously under-funded and lacking in resources. An August 2021 study by the European Data Protection Board (EDPB) found that 80% of national data protection authorities had insufficient funding to carry out their statutory tasks. The EDPB itself warned that it was “at risk of no longer being able to fulfil its legal duties.”[121]

It is obvious that whichever authority is tasked with supervising the use of AI systems will need sufficient resources to carry out those tasks. Failing to provide them only further adds to the possibilities for mistaken or malicious uses of dangerous technologies, and limits the opportunities for protecting rights and ensuring redress for affected groups and individuals. It may also be observed that politicians and officials who claim that the AI Act fully respects fundamental rights, whilst failing to provide resources to supervisory authorities, are essentially gaslighting the public.

Ability to prevent exchanges of information between supervisory authorities

The Act requires that the Commission, supervisory authorities and notified bodies exchange information to carry out their supervisory and monitoring tasks. When they do so, they must ensure that information remains confidential, to ensure the protection of:

intellectual property rights, confidential business information or trade secrets;
the implementation of the Act, in particular with regard to investigations, inspections and audits;
public and national security interests;
criminal or administrative proceedings; and
information that is classified under EU or national law.

Authorities granted a supervisory role by the Act must only request the information “strictly necessary” for undertaking their work and must put in place “effective cybersecurity measures” to protect that information.[122] A requirement to ensure confidentiality of “public and national security” already provides wide grounds to invoke exemptions to transparency, for example in response to freedom of information requests.

When it comes to biometric and emotion recognition systems, and high-risk AI systems used for law enforcement, migration, border control or asylum purposes, the Act contains more restrictive measures. In these cases, information cannot be exchanged between national authorities, or between national authorities and the Commission, “without prior consultation of the originating national competent authority and the deployer” of the system in question. Furthermore, any such exchange of information “shall not cover sensitive operational data”.[123] This provides policing and migration authorities the ability to restrict supervisory authorities’ access to information.

There are no further procedural requirements set out in the Act. It is not clear if, when or how competent authorities can refuse permission for monitoring authorities to exchange information. It is noteworthy that these restrictions apply even though it is made explicit that such exchanges of information cannot include “sensitive operational data.” As remarked above, this term and its lack of clear definition provides further leeway to the authorities to restrict supervisory authorities obtaining information on the development and deployment of AI systems.

It also appears to contradict the legal powers of data protection authorities to undertake independent investigations. Where data subject rights are restricted, Article 17 of the LED provides that these rights may be exercised through the competent data protection authority.[124]

Control over access to technical documentation

Following on from the above, if a law enforcement, immigration or asylum authority is the provider of a high-risk AI system for biometric identification or categorisation, emotion recognition, or for law enforcement, migration, border control or asylum purposes, they are given control over who has access to the technical documentation related to those systems.

The Act requires that technical documentation must stay on the premises of the authority that is the provider. Supervisory authorities must be able to request and “immediately” obtain access to that documentation, or a copy of it, provided their staff have the relevant security clearance.[125] However, it is not clear what procedure or penalties might apply if “immediate” access to requested documentation is not provided. This may allow the erection of further barriers to effective monitoring and oversight of high-risk AI systems deployed by police and migration authorities.

(return to top)

2.4 Implementing the Act

The exceptions and exemptions outlined above all exist on paper. What remains to be seen is how they will be implemented in practice. There is an array of guidelines, implementing legislation, delegated acts,[126] standards and other requirements intended to clarify certain elements of the Act – for example, on prohibitions and definitions. Civil society organisations, including Statewatch, have issued calls to ensure those guidelines are centred on protecting fundamental rights.[127]

The European Commission has a central role in the implementation process, providing an obvious route for corporate and state lobbyists to push for their interests. Such lobbying is already taking place in international standardisation organisations, where “standard-setting is being used to implement [AI Act’s] requirements related to fundamental rights, fairness, trustworthiness and bias.”[128] The European Parliament has also set up its own working group to monitor implementation of the Act.[129]

The law has led to questions from EU agencies themselves on how it should be interpreted. During a meeting of the EU Innovation Hub (section 4.1.2), Frontex asked how the law applies when “the Agency deploys AI technical equipment/capabilities that belong to a Member State?”

The border agency has also sought clarity on:

whether the Act applies if a member state passes it data generated by an AI system that is exempt from the law on national security grounds;
the use of “remote fingerprint acquisition” technology, one of many advanced biometric technologies that Frontex hopes to deploy in coming years;[130] and
how responsibility would be determined if an AI system jointly deployed with a member state breaches the law.[131]

The EU Asylum Agency has also raised questions:

Would EUAA be considered a deployer when an AI tool that is developed by a third party is shared with EU+ countries through a central EU platform?

Is the case of using AI for researching COI [country-of-origin information] considered high-risk? COI reports do assist the examination of asylum cases indirectly by informing the case officers about the situation in a country of origin. They are also quoted in asylum decisions.

If… speech-to-text tools [for transcribing asylum interviews] would be procured, would these be considered high-risk or an AI model with a systemic risk?[132]

In a report on AI and policing, Europol noted that the Act means AI systems already in use may need to be re-evaluated, to determine if and how they comply with the law.[133] The exemptions and exceptions in the law may well keep some of those systems out of reach of the more stringent rules in the AI Act.

As noted in section 4.1.3, EU police forces appear to have set up a specific working group on the implementation of the AI Act. If nothing else, the Act will create a substantial amount of work for lawyers, across the EU and beyond.

Indeed, the Act will undoubtedly be the subject of multiple court cases in the years to come, as governments, state agencies, businesses and individuals seek to challenge or clarify aspects of it. The extensive discrepancies between the Act and the EU’s data protection laws, highlighted in the sections above, is likely to be one topic under examination.

This will be a piecemeal and gradual process, especially as the Act will come into force in a step-by-step manner.[134] Many civil society organisations and lawyers across the EU will seek to ensure the Act is used and interpreted in a way that upholds fundamental rights, to the extent this is possible. Governments and industry may well seek the opposite.

Whatever the outcome of these efforts, the regulatory regime introduced by the Act will frame the use of artificial intelligence in the EU, and perhaps elsewhere in the world, for many years to come. In this context, it is vital to understand ongoing and emerging projects that seek to embed AI in policing, border, migration and criminal justice agencies. These developments are examined in the sections that follow.

(return to top)

< Previous section
1. Introduction

Next section >
3. Security AI in EU agencies

Notes

[1] Council of the EU, ‘Artificial intelligence act: Council and Parliament strike a deal on the first rules for AI in the world’, 9 December 2023, https://www.consilium.europa.eu/en/press/press-releases/2023/12/09/artificial-intelligence-act-council-and-parliament-strike-a-deal-on-the-first-worldwide-rules-for-ai/

[2] Article 1(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e1915-1-1

[3] Article 1(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e1915-1-1

[4] Recital(5), Recital (26), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[5] Article 7, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_7

[6] Article 50, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[7] Chapter V, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#cpt_V

[8] Article 46(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_46

[9] Used here as shorthand for policing, border control, immigration, asylum and criminal justice.

[10] Innovation Hub Team, ‘EU Innovation Hub for Internal Security – multi-annual planning of activities 2023-26’, Council doc. 5603/23, LIMITE, 16 February 2023, p.23, https://statewatch.org/wp-content/uploads/2026/05/1335957-v1-eu_innovation_hub_for_internal_security_multi-annual_planning_of_activities_2023-2026_st05603_en-public.pdf

[11] Ibid.

[12] Article 20, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_20

[13] Recital (6), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[14] Recital (9), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[15] Maria Maggiore, Leila Minano and Harold Schumann, ‘France spearheaded successful effort to dilute EU AI regulation’, EUobserver, 22 January 2025, https://euobserver.com/digital/ardc3193c4

[16] Plixavra Vogiatzoglou, ‘The AI Act National Security Exception’, Verfassungsblog, 9 December 2024, https://verfassungsblog.de/the-ai-act-national-security-exception/

[17] Article 2(1)(c), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e1975-1-1

[18] Article 2(4), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e1975-1-1

[19] Articles 37-38, Law Enforcement Directive, https://eur-lex.europa.eu/eli/dir/2016/680/oj/eng#cpt_V; Articles 48 and 50, EU data protection Regulation, https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32018R1725#cpt_V

[20] ‘Interpol: multi-million dollar “predictive analytics” system under construction’, Statewatch, 21 September 2023, https://www.statewatch.org/news/2023/september/interpol-multi-million-dollar-predictive-analytics-system-under-construction/

[21] ‘Hart Attack: How DHS’s massive biometrics database will supercharge surveillance and threaten rights’, Surveillance Resistance Lab, January 2023, https://surveillanceresistancelab.org/resources/hart-attack-how-dhss-massive-biometrics-database-will-supercharge-surveillance-and-threaten-rights/

[22] For example, through the General Data Protection Regulation or the Regulation on data protection in EU institutions, agencies, offices and bodies.

[23] Giovanni De Gregorio and Simona Demkova, ‘The Constitutional Right to an Effective Remedy in the Digital Age: A Perspective from Europe’ Ch. Van Oirsouw et al., (eds.), European Yearbook of Constitutional Law, 31 January 2024, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4712096

[24] ‘Analysis: Ireland’s AI Act exemptions’, Irish Legal News, 15 January 2025, https://www.irishlegal.com/articles/analysis-irelands-ai-act-exemptions

[25] Article 2(8), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_2

[26] ‘Surveillance Technologies at European Borders: Evros’, Border Violence Monitoring Network, 1 October 2024, https://borderviolence.eu/app/uploads/Border-surveillance-in-Evros.pdf

[27] Article 111(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_111

[28] More information on those systems is available in Statewatch’s interactive map: ‘EU agencies and interoperable databases’, https://www.statewatch.org/eu-agencies-and-interoperable-databases/

[29] ‘Automated Suspicion: the EU’s new travel surveillance initiatives’, Statewatch, 13 July 2020, https://www.statewatch.org/automated-suspicion-the-eu-s-new-travel-surveillance-initiatives/

[30] Article 111(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_111

[31] Article 3(38), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_3

[32] ‘Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on the accounting of greenhouse gas emissions of transport services’, 11 July 2023, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52023PC0441

[33] In a law establishing an online platform to facilitate cooperation in judicial investigations, “operational data” is defined as “information and evidence processed… during the operational phase of a [joint investigation] to support cross-border investigations and to support prosecutions.” See: Regulation (EU) 2023/969 of the European Parliament and of the Council of 10 May 2023 establishing a collaboration platform to support the functioning of joint investigation teams and amending Regulation (EU) 2018/1726, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32023R0969

[34] Recital 51 of the GDPR equates the term “sensitive personal data” with special categories of personal data, which are defined in Article 9 of the GDPR, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679

[35] Proposal for a Regulation laying down harmonised rules on artificial intelligence, 21 April 2021, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206

[36] Article 5(d), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[37] Protect Not Surveil, https://protectnotsurveil.eu/

[38] European Data Protection Supervisor, ‘EDPS comments to the AI Office’s consultation on the application of the definition of an AI system and the prohibited AI practices established in the AI Act launched by the European AI Office’, 19 December 2024, https://www.edps.europa.eu/data-protection/our-work/publications/formal-comments/2024-12-19-edps-ai-offices-consultation-application-definition-ai-system-and-prohibited-ai-practices-established-ai-act-launched-european-ai_en

[39] This is a new body established by the Act. See: ‘European AI Office’, https://digital-strategy.ec.europa.eu/en/policies/ai-office

[40] Article 5(g), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[41] Article 5(g), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[42] Council of the EU, ‘Artificial intelligence act: Council and Parliament strike a deal on the first rules for AI in the world’, 9 December 2023, https://www.consilium.europa.eu/en/press/press-releases/2023/12/09/artificial-intelligence-act-council-and-parliament-strike-a-deal-on-the-first-worldwide-rules-for-ai/

[43] “…esta prohibición no incluye el etiquetado o filtrado de conjuntos de datos biométricos adquiridos lícitamente, como imágenes, basado en datos biométricos ni la categorización de datos biométricos en el ámbito de la garantía del cumplimiento del Derecho.” This translates to: “…this prohibition does not include the labelling or filtering of sets of legally acquired biometric data, such as images, based on biometric data, nor the categorisation of biometric data within the area of law enforcement.”

[44] Recital (7), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[45] European Data Protection Supervisor, ‘EDPS comments to the AI Office’s consultation on the application of the definition of an AI system and the prohibited AI practices established in the AI Act launched by the European AI Office’, 19 December 2024, https://www.edps.europa.eu/data-protection/our-work/publications/formal-comments/2024-12-19-edps-ai-offices-consultation-application-definition-ai-system-and-prohibited-ai-practices-established-ai-act-launched-european-ai_en

[46] ‘TZ: Policie již téměř rok využívá analytický nástroj na rozpoznávání tváří. Podrobnosti jeho fungování tají’, Iuridicum Remedium, 12 July 2023, https://digitalnisvobody.cz/blog/2023/07/12/tz-policie-jiz-temer-rok-vyuziva-analyticky-nastroj-na-rozpoznavani-tvari-podrobnosti-jeho-fungovani-ale-pred-verejnosti-taji/

[47] ‘Danish Police Trials Facial Recognition Technology for Crime Investigation Amid Controversy’, en.365Nyt, 31 August 2024, https://en.365nyt.dk/2024/08/31/danish-police-trials-facial-recognition-technology-for-crime-investigation-amid-controversy/; ‘ Police in Denmark to implement facial recognition technology to combat violent crimes’, euronews, 14 August 2024, https://www.euronews.com/next/2024/08/14/police-in-denmark-to-implement-facial-recognition-technology-to-combat-violent-crimes

[48] ‘Sweden: Government Bill to Allow Police Use of Facial Recognition and DNA Genealogy Cleared for Parliament’s Consideration’, Library of Congress, 9 October 2024, https://www.loc.gov/item/global-legal-monitor/2024-10-09/sweden-government-bill-to-allow-police-use-of-facial-recognition-and-dna-genealogy-cleared-for-parliaments-consideration/

[49] Svea Windwehr, ‘Germany Rushes to Expand Biometric Surveillance’, Electronic Frontier Foundation, 7 October 2024, https://www.eff.org/deeplinks/2024/10/germany-rushes-expand-biometric-surveillance

[50] According to a report of a meeting between MEPs and the European Commission on the implementation of the AI Act.

[51] ‘Reclaim Your Face’, https://reclaimyourface.eu

[52] European Commission, ‘Impact assessment accompanying the proposal for a Regulation laying down harmonised rules for artificial intelligence’, 21 April 2021, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=SWD:2021:0084:FIN

[53] Article 5(1)(h), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[54] Annex II, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#anx_II

[55] Article 5(5), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[56] Abigail Opiah, ‘Swedish proposal tests AI Act’s live public facial recognition limits’, Biometric Update, 4 June 2024, https://www.biometricupdate.com/202406/swedish-proposal-tests-ai-acts-live-public-facial-recognition-limits

[57] ‘Biometric surveillance in the Czech Republic: the Ministry of the Interior is trying to circumvent the Artificial Intelligence Act’, EDRi, 9 October 2024, https://edri.org/our-work/biometric-surveillance-in-the-czech-republic-the-ministry-of-the-interior-is-trying-to-circumvent-the-artificial-intelligence-act/

[58] Article 5(2)(a) and (b), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[59] Article 5(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[60] European Court of Human Rights, ‘Case of Glukhin v. Russia’, 4 October 2023, https://hudoc.echr.coe.int/?i=001-225655

[61] European Data Protection Supervisor, ‘EDPS comments to the AI Office’s consultation on the application of the definition of an AI system and the prohibited AI practices established in the AI Act launched by the European AI Office’, 19 December 2024, https://www.edps.europa.eu/data-protection/our-work/publications/formal-comments/2024-12-19-edps-ai-offices-consultation-application-definition-ai-system-and-prohibited-ai-practices-established-ai-act-launched-european-ai_en

[62] Article 5(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[63] Article 5(3), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2812-1-1

[64] Article 74(8), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_74

[65] European Data Protection Supervisor, ‘EDPS comments to the AI Office’s consultation on the application of the definition of an AI system and the prohibited AI practices established in the AI Act launched by the European AI Office’, 19 December 2024, https://www.edps.europa.eu/data-protection/our-work/publications/formal-comments/2024-12-19-edps-ai-offices-consultation-application-definition-ai-system-and-prohibited-ai-practices-established-ai-act-launched-european-ai_en

[66] Annex III, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e38-127-1

[67] Article 27(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_27

[68] Article 27(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_27

[69] ‘Comparative review of 10 FRIAs’, Algorithm Audit, September 2024, https://algorithmaudit.eu/knowledge-platform/knowledge-base/comparative_review_10_frias

[70] European Data Protection Supervisor, ‘Necessity & Proportionality, undated, https://www.edps.europa.eu/data-protection/our-work/subjects/necessity-proportionality_en

[71] Article 27(4), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_27

[72] Article 27(3), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_27

[73] Recital (131), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[74] Article 6(4), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_6

[75] Innovation Hub Team, ‘EU Innovation Hub for Internal Security – multi-annual planning of activities 2023-26’, Council doc. 5603/23, LIMITE, 16 February 2023, p.23, https://statewatch.org/wp-content/uploads/2026/05/1335957-v1-eu_innovation_hub_for_internal_security_multi-annual_planning_of_activities_2023-2026_st05603_en-public.pdf

[76] Tim Cushing, ‘Public Records Show Cops Are Obscuring Their Use Of Facial Recognition Tech In Criminal Cases’, Techdirt, 21 October 2024, https://www.techdirt.com/2024/10/21/public-records-show-cops-are-obscuring-their-use-of-facial-recognition-tech-in-criminal-cases/

[77] Article 26(11), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_26

[78] Directive (EU) 2016/680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, https://eur-lex.europa.eu/eli/dir/2016/680/oj

[79] Article 13(3), Directive (EU) 2016/680, https://eur-lex.europa.eu/eli/dir/2016/680/oj

[80] Article 50(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[81] Marvin L. Astrada and Scott B. Astrada, ‘Law, continuity and change: Revisiting the reasonable person within the demographic, sociocultural and political realities of the twenty-first century’, undated, https://rutgerspolicyjournal.org/jlpp/wp-content/uploads/sites/26/2017/05/Astrada.pdf

[82] Article 50(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[83] Article 50(3), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[84] Article 50(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[85] Article 50(4), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[86] Article 2(1)(g), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[87] Case C-817/19, 21 June 2022, paras. 194-195, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:62019CJ0817

[88] Case C-817/19, 21 June 2022, para. 210. See also : Evelien Brouwer, ‘Challenging Bias and Discrimination in Automated Border Decisions’, Verfassungsblog, 11 May 2023, https://verfassungsblog.de/pnr-border/

[89] Melanie Fink, ‘The Hidden Reach of the EU AI Act: Expanding the Scope of EU Public Power’, Verfassungsblog, 20 January 2025, https://verfassungsblog.de/the-hidden-reach-of-the-eu-ai-act/

[90] Article 60, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_60

[91] Article 71(4), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_71

[92] Annex IX, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#anx_IX

[93] International Standards Organisation, ‘Standards’, undated, https://www.iso.org/standards.html

[94] Recital (123), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[95] ‘Bias baked in: How Big Tech sets its own AI standards’, Corporate Europe Observatory, 9 January 2025, https://corporateeurope.org/en/2025/01/bias-baked

[96] Article 40(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_40

[97] Article 43(1), AI Act: “…where the high-risk AI system is intended to be put into service by law enforcement, immigration or asylum authorities or by Union institutions, bodies, offices or agencies, the market surveillance authority referred to in Article 74(8) or (9), as applicable, shall act as a notified body.” Article 74(8) of the Act designates data protection, rather than market surveillance, authorities as responsible for overseeing use of high-risk systems for “law enforcement purposes, border management and justice and democracy.” In those cases: “Member States shall designate as market surveillance authorities for the purposes of this Regulation either the competent data protection supervisory authorities under Regulation (EU) 2016/679 [General Data Protection Regulation] or Directive (EU) 2016/680 [Directive on data protection in law enforcement], or any other authority designated pursuant to the same conditions laid down in Articles 41 to 44 of Directive (EU) 2016/680.”

[98] “…‘national competent authority’ means a notifying authority or a market surveillance authority; as regards AI systems put into service or used by Union institutions, agencies, offices and bodies, references to national competent authorities or market surveillance authorities in this Regulation shall be construed as references to the European Data Protection Supervisor.” See: Article 3(48), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_3

[99] Article 41(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_41

[100] Article 41(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_41

[101] European Commission, ‘Implementing and delegated acts’, undated, https://commission.europa.eu/law/law-making-process/adopting-eu-law/implementing-and-delegated-acts_en

[102] Article 41(5), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_41

[103] Article 40, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_40

[104] Article 41, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_41

[105] Article 17(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_17

[106] Annex VI, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#anx_VI

[107] Annex VI, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#anx_VI

[108] Article 43(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_43

[109] Article 43(6), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_43

[110] Article 46(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_46

[111] Court of Justice of the EU, ‘Ordre des barreaux francophones et germanophone and Others v Conseil des ministers’, Case-718/19, 22 June 2021, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A62019CJ0718

[112] Article 46(1), AI Act: “By way of derogation from Article 43 and upon a duly justified request, any market surveillance authority may authorise the placing on the market or the putting into service of specific high-risk AI systems within the territory of the Member State concerned”.

[113] Article 46(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_46

[114] Article 46, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_46

[115] Article 46(2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_46

[116] Article 89, GDPR, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679#d1e6494-1-1

[117] Article 3(49), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e2090-1-1

[118] Article 73(9), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e7140-1-1

[119] Article 74(8), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_74

[120] Article 74(9), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_74

[121] Data protection: 80% of national authorities underfunded, EU bodies “unable to fulfil legal duties”, Statewatch, 30 September 2022, https://www.statewatch.org/news/2022/september/data-protection-80-of-national-authorities-underfunded-eu-bodies-unable-to-fulfil-legal-duties/

[122] Article 78(1), (2), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e7427-1-1

[123] Article 78(3), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e7427-1-1

[124] Article 17, Law Enforcement Directive. See also: Case C-333/22, 16 November 2023, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:62022CA0333

[125] Article 78(3), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#d1e7427-1-1

[126] Article 97, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_97

[127] ‘EU: Human rights must be “central guiding basis” for new AI guidelines’, Statewatch, 16 January 2025, https://www.statewatch.org/news/2025/january/eu-human-rights-must-be-central-guiding-basis-for-new-ai-guidelines/

[128] ‘Setting the rules of their own game: how Big Tech is shaping AI standards’, Corporate Europe Observatory, 9 January 2025, https://corporateeurope.org/en/2025/01/setting-rules-their-own-game-how-big-tech-shaping-ai-standards

[129] European Parliament, ‘Working Group on the implementation and enforcement of the AI Act’, undated, https://www.europarl.europa.eu/committees/en/working-group-on-the-implementation-and-/product-details/20241113CDT13823

[130] ‘Europe’s techno-borders’, Statewatch/EuroMed Rights, 10 July 2023, https://www.statewatch.org/publications/reports-and-books/europe-s-techno-borders/

[131] Frontex, ‘Questions – the AI Act and the EBCG’, undated, https://statewatch.org/wp-content/uploads/2026/05/doc-14_edoc-1409502-v1-frontex_ai_act_-_questions_jun_2024_full-access.pdf

[132] EU Asylum Agency, ‘Key Questions on Implementing the EU AI Act – Draft contribution to DG HOME’, 25 June 2024, https://statewatch.org/wp-content/uploads/2026/05/doc-15_edoc-1401527-v1-eu_innovation_hub_ai_cluster_-_euaa_questions_ai_act_implementation_full-access.pdf

[133] Europol, ‘AI and policing’, pp.43-44, https://www.europol.europa.eu/cms/sites/default/files/documents/AI-and-policing.pdf

[134] European AI Office, ‘Implementation Timeline’, updated 1 August 2024, https://artificialintelligenceact.eu/implementation-timeline/

3. Security AI in EU agencies

EU agencies are already developing and using various types of AI technology. This section looks in particular at projects and activities launched by eu-LISA and Europol, as well as Frontex, Eurojust and the EU Asylum Agency. There are a wide variety of AI technologies – from facial recognition to machine learning and ‘predictive’ technologies – that have been examined or are actively deployed.

In this section

3.1 eu-LISA

3.1.1 Algorithmic profiling of travellers

3.1.2 AI in the shared Biometric Matching System

3.1.3 Digitalising the visa application process: visa chatbot

3.2 Europol

3.2.1 From challenge to opportunity

3.2.2 Machine learning

3.2.3 Facial recognition

3.2.4 Data protection and European policing

3.3 Frontex

3.3.1 AI in the maritime domain

3.4 EU Asylum Agency

3.4.1 Automated dialect recognition for asylum applicants

3.5 Eurojust

3.5.1 Joint Investigation Teams platform

EU agencies are already developing and using various types of AI technology. This includes:

systems for the algorithmic profiling of travellers (eu-LISA);
machine learning for the analysis of massive quantities of data (Europol); and
systems for automatic dialect recognition, to be used to determine the nationality of refugees (EUAA).

The implementation of the Act will require new bureaucratic processes and procedures to justify and manage the use of these systems. However, it is unlikely to fundamentally change how they operate. Whatever impact the Act has will not be felt for some time. A carve-out for the EU’s large-scale IT systems (section 2.3.1) means the rules may not apply to them until 2030.

Even where the Act does apply, it remains to be seen whether EU agencies’ systems for internal accountability, combined with the external supervision of the European Data Protection Supervisor, will ensure the law is applied as intended.

3.1 eu-LISA

The EU Agency for the Operational Management of Large-Scale IT Systems in the Area of Freedom, Security and Justice is usually referred to as eu-LISA. It has its headquarters in Tallinn, Estonia, with technical infrastructure hosted in Austria and France. It “manages large-scale IT systems that support the implementation of asylum, border management and migration policies in the EU.” These are:

the European Travel Information and Authorisation System (ETIAS)
the Schengen Information System (SIS);
the Visa Information System (VIS).

It is also developing a number of other systems that are due to be introduced:

the Central Repository for Reporting and Statistics (CRRS);
the Common Identity Repository (CIR);
the Entry/Exit System (EES);
Eurodac;
the European Criminal Records Information System on Third-Country Nationals (ECRIS-TCN);
the European Search Portal (ESP); and
the shared Biometric Matching System (sBMS).

The purposes of these systems include the sharing of information between police, border and judicial authorities (SIS); the storage and processing of information on visa applicants (VIS); the storage of biometric and biographic “identity data” on non-EU citizens (CIR); and the registration of border crossings by all non-EU citizens (EES).[1]

The agency is incorporating various forms of AI into the systems it manages. In 2021, these initiatives – and others – were incorporated into a “roadmap” on AI initiatives. One initiative was the establishment of a Centre of Excellence for AI for justice and home affairs policies, examined in section 4.1.1 as a form of institutional infrastructure. With regard to the use of AI technologies for the EU’s large-scale IT systems, three initiatives from the roadmap are examined below:

the algorithmic profiling of travellers;
the use of AI in the shared Biometric Matching System; and
the development of a “chatbot” to support the visa application process.

(return to top)

3.1.1 Algorithmic profiling of travellers

Of the EU’s current AI projects, the one likely to have the most widespread impact concerns the use of automated risk profiling against travellers. In the years to come, anyone travelling to the Schengen area who requires a short-stay visa or a travel authorisation will be subjected to an array of automated screening and profiling techniques. As remarked in a previous Statewatch report, “people visiting the EU from all over the world are being placed under a veil of suspicion in the name of enhancing security.”[2]

Visas and travel authorisations

Citizens of more than 100 countries must acquire a visa to enter the Schengen area legally. Those countries are largely poor (or, at least, poorer than EU states), with majority non-white populations.[3] Those who wish to travel to the Schengen area must go through a lengthy, invasive and often expensive application process.

The EU is also in the process of creating a separate travel authorisation process, for citizens of countries who do not need a visa to enter the Schengen area.[4] While the application process is neither as expensive, intrusive or inconvenient as that for a visa, it amounts to the same thing: you require government permission to travel, and must pay and hand over information for the privilege.

Both categories of traveller will be subject to AI decision-making techniques, once the EU’s “interoperability” architecture has been set up. The interoperability architecture will interconnect six large-scale policing, migration and criminal justice databases that hold biometric and biographic data on tens of millions of people, and create three new large-scale databases.

The aim is to make it easier for officials to access data on non-EU citizens, and to make it possible to use large amounts of personal data in new ways. For example, large amounts of data must be mined to develop the various “risk profiles” and “screening rules” that will be used to single out supposedly risky travellers.

All Schengen visa applicants have their personal data a stored in a database called the Visa Information System (VIS). For individuals who have to apply for a travel authorisation system, the equivalent is the European Travel Information and Authorisation System (ETIAS). Through the interoperability architecture, visa and travel authorisation applications will be cross-checked against other connected European and international databases. Statistical data will also be used to determine whether an individual belongs to a group considered to pose a potential risk.

Statistical power

As part of the interoperability project, the EU is constructing a database called the Central Repository for Reporting and Statistics (CRRS). This will extract data from the VIS, the ETIAS and four other large-scale databases, to allow the generation of data and statistics on all manner of events: border crossings; refusals of entry; non-EU citizens overstaying their allotted time in the Schengen area; the number of asylum applicants in the EU; and much more.[5]

When it was first proposed by the group of officials that set out the interoperability plans, it was referred to as a “data warehouse.” The aim, in their words, was to:

…help Member States to make better use of the systems, including by taking informed decisions on EU policies in the area of migration and security. It would also provide valuable statistics for relevant agencies in these areas, to perform analytical reviews.”[6]

The “valuable statistics” include those that will be used by the ETIAS Central Unit, hosted at Frontex, to generate “specific risk indicators.” Those indicators will be used to flag travelling individuals as risky or not. They include age range, sex and nationality; country and city of residence; and the type of employment the visa or travel authorisation applicant holds.

Basis for the specific risk indicators
ETIAS	VIS
age range, sex, nationality	age range, sex, nationality
country and city of residence	country and city of residence
current occupation (job group)	current occupation (job group)
the Member States of destination
	the Member State of first entry
	purpose of travel
level of education (primary, secondary, higher or none)

Building an automated profiling system

Work has been ongoing for some time to develop the technology needed to implement this plan. One document that provides further insight was finalised in November 2022. It was produced under a contract with a consortium of companies (Unisys Belgium, Unisys Luxembourg and Wavestone) for Frontex and eu-LISA, as part of a larger contract for constructing the interoperability architecture.[7]

The report is clear that it is “a research document and the outcome does not indicate that the AI Solutions will be utilized in the manner suggested and described in the Study.” Nevertheless, it provides important insight into ideas that have been developed on this topic.

The study outlines a number of requirements for the CRRS. Firstly, it should include “automated mechanisms to predict and discover potential risks.” It should also be expansive, in the sense that the techniques developed should be used in realms other than visa and travel authorisation applications. The report says that the system’s ability “to learn based on data inputs and/or outputs” should “extend beyond the specific objectives of this project.”

The AI techniques deployed in the CRRS should “allow to analyse and draw inferences from data.” At the same time, the CRRS should be only “one of the sources to be utilised by AI technology to analyse current and past information through machine learning capabilities.” The report does not specify what other sources may be used, though this would appear to go beyond the scope of the current legislation.[8]

Finally, the technology used should “enable to identify patterns within the data stored in the CRRS,” but should “not be used to predict or forecast the future.” However, despite saying this, the report includes a diagram for a “predictive analytics” function, and says that it should be possible “to generate new predictions and produce the corresponding results/outcomes.” Indeed, a component of the system called “Produce Predictions” is proposed, which would indicate that there is at least an interest in predictive technologies.

The study also examined seven possible “Business Use Cases” (BUCs) for the application of AI on visa and travel authorisation applicants.

Identification of risk for specific group of travellers

Here, “the proposed Al technology will identify patterns or a set of common characteristics from the analysis of historical data available in the CRRS.” It would be used to identify “security, illegal immigration or high epidemic risk for a specific group of travellers,” referred to as “clusters.” These could be based on “similar age/education/occupation, etc. or any combination of the input attributes.”[9]

Review and validate identified groups of travellers of risk

This would involve “an Al Solution that seeks for deviations from the already identified groups of travellers of risk for ETIAS.” The use case is described as “a characteristic scenario of Machine Learning (ML)-based Anomaly Detection.” A series of indicators “commonly agreed with eu-LISA’s stakeholders” would be developed to illustrate “the normal behaviour of the risk profiles.” Any individual deviating from this “‘proper/normal’ behaviour will be marked as ‘suspicious’.”[10]

Analyse risk profile correlations

This technique would “identify and highlight correlations amongst the risk profiles from its historical data… allowing a more precise definition.” The aim would be “to comprehend the patterns between risk profiles, and further understand any underlying behaviours behind them or improve the classification process to lead to more tangible, robust and reliable results.” Algorithms for pattern recognition would be deployed “to identify any hidden trends and regularities.” The “recognition of trends” could be done by officials, or by using “state-of-the-art Machine Learning techniques.”[11]

Automatic processing of textual information from various sources

AI technology would be deployed in this scenario to “identify text patterns or a set of common textual characteristics” in the relevant CRRS data. This would be analysed to identify clusters “that meets the criteria for the validation of a proposed risk profile.”[12] The “key concept,” according to the report, is to use CRRS data to “produce valuable insights, such as the required keywords of interest for the ETIAS database.” This would allow travel authorisation applications to be checked for those keywords in order to detect potentially risky individuals.[13]

Ex-post assessment process

The risk indicators are supposed to be reviewed regularly by Frontex and the other agencies participating in the ETIAS Central Unit. AI technology could also be deployed to “enhance” this “ex-post assessment process.” This would be done “by analysing and detecting deviations and proposing the review of the risk indicators.”[14]

Virtual assistance to provide guidance on certain activities

This would provide travel authorisation and visa applicants with an automated chatbot “to be provided with answers to certain questions about ETIAS.” Such a service could also be offered to officials making use of the system in their work. The report notes that “similar methodology and design may also be extended and applied to other sub-systems as well.”[15] As noted in section 3.1.3, a separate project is seeking to develop an automated chatbot for the visa application process, which evidently has parallels here.

Verification when a hit takes place

In this use case, machine learning-based “Anomaly/Fraud Detection” would be used “to identify all relevant patterns within the provided data that will eventually identify ‘hits’.”[16] The report gives the example of when a new ETIAS risk profile is agreed. The technology could assigned a label “based on the previously trained model, indicating whether the specific risk profile is considered a ‘hit candidate’ or ‘no-hit-candidate’.” It thus appears to be a way to use AI to further refine the profiling process.

(return to top)

The “Business Use Cases” examined in the report on AI in the CRRS produced for eu-LISA

3.1.2 AI in the shared Biometric Matching System

The shared Biometric Matching System (sBMS) is used to “support all biometric operations required by the systems that eu-LISA runs.” This would include, for example, matching fingerprint scans taken from individuals with samples stored in the Schengen Information System or Visa Information System; or matching facial image scans with samples stored in Eurodac or the Entry/Exit System.

According to eu-LISA’s AI roadmap, the sBMS uses “Convolutional Neural Networks” for face and fingerprint matching and to generate templates from face and fingerprint scans:

Currently, the sBMS supports two types of biometric operations, both of which use AI:

Verification, which is a comparison of two images in order to determine if they are the same. In this context, AI is used in order to analyse the image and build the templates (a mathematical representation of the image), while the comparison itself is done without using AI.
Identification, where the AI computes the template of a specific image, associated with a specific context. The template is then stored in the database and used for later comparisons.[17]

This includes the use of AI “to enhance the ability to extract more accurate templates, specifically in cases of low quality, thus reducing the risk of false negative and false positive matches.”[18]

As the shared Biometric Matching System is a component of the large-scale information systems operated by eu-LISA, it is excluded from the scope of the AI Act until 1 January 2031. From this point onwards, it must meet the Act’s requirements. Before then, the law ostensibly designed to ensure “human centric and trustworthy artificial intelligence”[19] will not apply to it.

(return to top)

3.1.3 Digitalising the visa application process: visa chatbot

In November 2023, new rules for a digital Schengen visa application process were approved by EU interior ministers.[20] This will simplify the process of applying for a short-stay Schengen visa, at least for those with access to and the ability to use an online system. However, the new rules make no changes to the way in which applications are assessed or the substantive requirements for acquiring a visa.

The rules establish an EU Visa Application Platform (EU VAP), through which visa applicants will be able to enter all the information necessary to apply for a visa, with the exception of fingerprints. This will digitise substantial amounts of information that were previously provided by applicants on paper forms.

Despite this, the new rules do not introduce requirements to store this data in the Visa Information System, the EU’s large-scale centralised database on short-stay Schengen visa and residence permit applications. The newly digitised data will be stored by the state with which the visa application is filed. Given the likely perceived utility of that data for profiling and “screening” visa applicants, it may be that the rules are changed further in the future, to facilitate centralised collection of the data.

One of the novelties that will be introduced with the new digital platform is a chatbot. This is defined as “software that simulates human conversation through interaction by text or voice.”[21] The visa chatbot is supposed to provide answers “on the visa application procedure, the rights and obligations of applicants and visa holders, the entry conditions for third-country nationals, contact details, and data protection rules.”[22]

The law on digitalising the visa process does not state that the chatbot is a form of AI. Nevertheless, under the AI Act, people must be made aware when they are interacting directly with an AI system, “unless this is obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect, taking into account the circumstances and the context of use.”[23] The chatbot will also not be “the only means by which the applicant could get information on the visa procedure,” according to the law.[24]

Given that eu-LISA will manage the chatbot and that it will be related to – if not part of – the Visa Information System, it may be excluded from the scope of the AI Act until the end of the decade.[25] However, it is hard to see what benefit this would bring to either authorities or visa applicants.

Perhaps the most interesting aspect of the visa chatbot is that its development was intended to serve as the first project for the AI Centre of Excellence, to be based at eu-Lisa, though it has now seemingly been shelved (section 4.1.1). Both the development of the CoE and the development of the chatbot itself were farmed out to Deloitte under a 2020 contract with the European Commission. One of the reports produced under that contract states:

Approaching the CoE with a ‘dream big, start small’ vision, implies that whilst defining the end state of the CoE is the main goal, starting the implementation of the CoE from a practical project, i.e. the Visa Chatbot, is desired.[26]

By February 2022, five EU member states were involved in the chatbot project, and were working on a “proof of concept.” The aim was to put the chatbot into service from 2023 onwards.[27] However, the rules on digitalising the visa application process were not approved until November of that year, and an array of implementing decisions have to be approved by the Commission before the system can be put into use.

(return to top)

3.2 Europol

The EU Agency for Law Enforcement Cooperation, better known as Europol, is tasked with supporting EU member states in “in preventing and combating all forms of serious international and organised crime, cybercrime and terrorism.”[28] It primarily does this by gathering and receiving large quantities of data on individuals and objects (for example, vehicles or firearms). It then analyses that data to inform and advise police operations and investigations.

In May 2021, the agency reported that it was using six types of AI technology:

biometrics;
“facial recognition modules”;
“automatic identity extraction modules”;
automatic translation software;
tools for the automated analysis of images and videos; and
an “AI tool for malware analysis.”[29]

An article by an unnamed Europol offers an insight into the use of some of these tools:

Europol uses AI to support high-profile investigations to extract and classify information from an increasingly large number of data sources. As such, analysts supported by the data science team use a set of AI models to classify images by automatically assigning tags to millions of pictures or to extract named entities from text, including the names of people, locations, phone numbers, or bank accounts. Other AI models allow analysts to search for images of cocaine bricks with a specific logo or detect useful information in pictures, like the number on the door of a shipment container or the name and date of birth from a picture of a badge.[30]

However, as this report demonstrates, the agency is also seeking to use other forms of AI, such as machine learning (see section 3.2.2)

As with all policing agencies, the work of Europol has high potential risks for individual rights and freedoms. Indeed, the agency has a structural role in upholding and enforcing political and social systems that cause substantial harm. For example, “facilitation of irregular immigration” is one of the EU’s top crime priorities, and Europol plays a key role in aiding investigations.[31]

The “war on smugglers” that has been ongoing for the last decade has done little to alleviate “illegal” immigration. Instead, it has pushed people to use more dangerous routes, putting their lives at risk. As a result of these enforcement efforts, thousands of people who did nothing more than drive boats have been imprisoned in Europe since 2014 on charges of facilitating illegal entry, primarily in Greece and Italy. Apparently impervious to the fact that a relentless focus on law enforcement has changed little (if anything) for the better, politicians remain wedded to it.[32]

The upshot for law enforcement agencies has been increases in their budgets and powers. In 2023, the European Commission published a proposal that would extend Europol’s powers even further, specifically in relation to the offences of human smuggling and trafficking. It was announced at the launch of the Commission’s “Global Alliance to Counter Migrant Smuggling,” and another legislative proposal, together intended to ensure a “new legal, operational and international cooperation framework against migrant smuggling.”[33] It remains to be seen whether the proposal will result in any substantial changes,[34] but its key aim is to increase the amount of data sent to Europol.

The 2023 proposal follows legal reforms that came into force in 2022. These were specifically designed to make it simpler for the policing agency to process mass quantities of data, to ease cooperation with non-EU states, and to get involved in “the development and use of artificial intelligence for analysis and operational support,”[35] in large part to simplify the processing of massive datasets.

Police and state officials frequently claim that new technologies such as AI are needed to “keep pace” with criminals. In the words of an unnamed Europol official writing in Police Chief magazine, AI can “make policing more efficient and effective.”[36] This may well be true. Whether it is ultimately a desirable outcome, however, remains open to question.

No matter how efficient and effective law enforcement agencies may be, turning complex social, political and economic issues (such as the smuggling of people across state borders) into matters to be primarily dealt with through police power sidesteps more fundamental and potentially effective responses. Attempting to increase police powers at a time of growing support for far-right political parties – already in power in a number of European states – is also giving would-be authoritarian politicians precisely what they want.

(return to top)

3.2.1 From challenge to opportunity

In April 2019, Catherine de Bolle, a Dutch police officer who had been appointed as head of Europol just under a year earlier, wrote to the European Data Protection Supervisor with concerns over “major compliance issues with the Europol Regulation.” She referred to those issues as the agency’s “big data challenge.”[37]

The rules that governed the agency at the time set out relatively strict rules on how it may process data on various categories of persons.[38] For example, the agency could process far more types of data on suspects than it could on victims or witnesses. However, the EDPS’ inquiry found that “it is not possible for Europol, from the outset, when receiving large data sets to ascertain that all the information contained in these large datasets comply with these limitations.”[39]

The agency had evidently been receiving vast quantities of personal data for some time. Following the terrorist attacks in Paris and Brussels in 2015, it received over 16.7 terabytes of data from national law enforcement agencies.[40] It would therefore appear that the agency was breaking the law for several years, until de Bolle was appointed and, just under a year after coming into the job, alerted the EDPS to the situation.

The EDPS’ inspection report said:

The processing of data about individuals in an EU law enforcement database can have deep consequences on those involved. Without a proper implementation of the data minimisation principle and the specific safeguards contained in the Europol Regulation, data subjects run the risk of wrongfully being linked to a criminal activity across the EU, with all of the potential damage for their personal and family life, freedom of movement and occupation that this entails.[41]

In September 2020, the EDPS formally admonished the agency and set out a number of changes the agency needed to make to comply with the law. Europol failed to meet these requirements. In response, the EDPS issued a deletion order at the beginning of 2022. The supervisory authority noted that while Europol had put in place “some measures” since September 2020, the agency:

…had not complied with the EDPS’ requests to define an appropriate data retention period to filter and to extract the personal data permitted for analysis under the Europol Regulation. This means that Europol was keeping this data for longer than necessary, contrary to the principles of data minimisation and storage limitation, enshrined in the Europol Regulation.[42]

The response to this decision came from the European Parliament and the Council of the EU. Rather than seeking to ensure compliance with the law as it stood, they changed it to legalise Europol’s practices. In June 2022, changes to the law governing the agency came into force. These allowed the agency to continue processing the personal data of individuals with no established link to criminal activity. What was once a big data challenge had become a big data opportunity.

Europol’s Management Board then landed itself in hot water with the EDPS in relation to the changes. The Management Board had to adopt decisions setting out how the agency would implement the legal changes. At the end of June 2022, it did so – but did not consult the EDPS, which it was legally required to do. After the EDPS raised the issue with Europol, the Parliament, the Council and the Commission, an agreement was reached between the EDPS and the Management Board.[43]

Despite all these twists and turns, the saga is not over yet. In September 2022, the EDPS filed a legal case against the Parliament and Council that sought to annul the relevant parts of the legislation, noting that “the co-legislators have decided to retroactively make this type of data processing legal,” contrary to the EDPS order to delete the data in question. The EDPS said that the changes to the law undermined “the independent exercise of powers by supervisory authorities,” and that the legal action sought to ensure that legislators could not “unduly ‘move the goalposts’ in the area of privacy and data protection.”[44] However, the Court of Justice of the EU found the complaint inadmissible. An appeal is pending.[45]

(return to top)

3.2.2 Machine learning

While the EDPS investigation into Europol’s “big data challenge” was ongoing, it also had a number of other cases to deal with. In May 2019, a month after de Bolle wrote to the EDPS about Europol’s big data challenge, the EDPS launched an inquiry on a different topic. The supervisory authority sought information on “the use of operational data for data science purposes,” namely “the training, testing and validation of machine learning models.”

According to the multinational computing corporation IBM:

Machine learning (ML) is a branch of artificial intelligence (AI) focused on enabling computers and machines to imitate the way that humans learn, to perform tasks autonomously, and to improve their performance and accuracy through experience and exposure to more data.[46]

The aim of the EDPS inquiry was to understand “the lawfulness of data processing activities taking place in this context and of the safeguards put in place to address the data protection risks linked to the use of machine learning tools.”[47] It appears that it did not reach the level of understanding hoped for:

We requested, on several occasions, detailed information about the policies in place; the appropriate legal basis for the processing operations; the safeguards put in practice to protect individuals’ personal data; and the specific projects that were at this point carried out by Europol. The replies and information sent to us were not considered satisfactory.[48]

This inquiry was subsequently merged with another one. In October 2020, perhaps wary of the admonishment on the “big data challenge” issued by the EDPS the previous month, Europol officials had requested an informal consultation on the use of machine learning techniques.

Europol wanted to use machine learning techniques for the “operational analysis” of large datasets shared with the agency by national law enforcement agencies. It seems likely that the data in question related to the Encrochat[49] or SkyECC[50] cases.[51] In those cases, law enforcement authorities hacked and/or bugged two encrypted communications services to gain access to large quantities of messages (in the SkyECC case, police reportedly obtained around one billion messages[52]).

Through the use of machine learning, Europol aimed to facilitate “the pre-selection, for review and assessment, of communications or users of these illegal communications.” The development of machine learning models would “help the selection of messages that could be of higher relevance” for investigators.

While Europol officials had sought an informal consultation with the EDPS, exchanges between the two sides led to the EDPS opening a formal consultation procedure in February 2021. The following month, the EDPS issued an opinion on the topic.[53]

The opinion described Europol’s explanation of the proposed data processing as “often incomplete,” and “completely lack[ing] any description of the processing operations. This is problematic as this is the foundation for the rest of the prior consultation process.” It notes that the information shared by Europol:

…does not allow the EDPS to sufficiently understand the full effects of the new processing operations in detail, from the selection of models, to the use of the operational data, including how all the processes are monitored.[54]

The opinion also raised potential issues with the necessity and proportionality of the use of machine learning; with the data minimisation principle; and risks related to bias, statistical accuracy, errors and security. It concluded by saying that the EDPS was unable “to assess whether the notified processing operations comply with the provisions of the Europol Regulation.” Europol was requested to implement an array of measures set out in an action plan and a letter, though these remain unavailable to the public.

One action that did follow the EDPS opinion was the introduction of a specific policy on machine learning for operational analysis. This was signed off by the agency’s Management Board in June 2021. It defines operational analysis as a process by which:

…personal data is used specifically to determine operational action against (a group of) individuals in relation to one or more criminal offences, which may include the seizure of goods, the arrest of suspects and the deployment of investigative techniques to collect evidence.[55]

The policy states that it does not cover techniques such as predictive policing or automated decision-making, nor the use of machine learning used in or developed for research and innovation projects.

The policy requires that any “use and development of machine learning tools for the purpose of operational analysis shall be necessary and proportionate,” and goes on to explain how these two requirements can be assessed and met. It goes on to do the same with the principles of data minimisation, data bias, data accuracy, human intervention, data retention and data security. It also includes provisions on auditing and includes a “by no means exhaustive” list of unacceptable uses of machine learning tools.

Large parts of the policy, which was released to Statewatch in response to an access to documents request, are censored, making it difficult to fully assess its comprehensiveness. However, the fact that it was only released in response to a formal request is noteworthy.

“The development and use of machine learning tools is considered as a form of application of Artificial Intelligence (AI) entailing a number of risks to the fundamental rights and freedoms of individuals,” says the policy. Keeping it locked away until a request was made for its release, and then censoring substantial parts of it, does nothing to aid scrutiny of it.

(return to top)

3.2.3 Facial recognition

More recently, Europol has sought data protection advice on a new facial recognition system it planned to use. In mid-October 2023, the EDPS received a request from Europol for a prior consultation on a “Face Recognition Solution” based on the corporation NEC’s NeoFace Watch system.[56]

As with the “big data challenge” and the introduction of machine learning techniques, the need for a new facial recognition system was premised on the growing amount of data informing Europol’s work. While the agency had been using a facial recognition system known as “FACE” since 2016, new technology was acquired due to the rising volume of requests for assistance from “Europol’s stakeholders,” says the EDPS opinion.

NEC markets the NeoFace Watch system on the basis of its ability to be used in real time. “Faces of individuals are captured and extracted from the video feed and quality matched in real-time. NeoFace Watch software is able to process multiple camera feeds extracting and matching thousands of faces per minute,” say the company.[57] However, Europol uses the system on images and video transmitted to it post facto, and not on live video feeds. The system can also be purchased with a machine learning component, though Europol did not acquire this.

Although large parts of the EDPS opinion released to Statewatch are censored, it appears evident that the data protection impact assessments drawn up by Europol were insufficient. For example, while the agency said it aimed to use facial recognition for three different purposes, it only included an assessment for one of these, operational analysis.

In relation to this issue, the EDPS concluded it was “necessary for Europol to specify the categories of individuals for whom facial recognition will be used” in Europol’s portfolio of Analysis Projects. These sit within the Europol Analysis System and “focus on certain crime areas from commodity-based, thematic or regional angles, e.g. drugs trafficking, Islamist terrorism, Italian organised crime.”[58] Failing to specify on whom facial recognition would be used “creates risks of non-compliance with the principle of purpose limitation,” the EDPS noted.

Europol also intended to use facial recognition for the purposes of cross-checking and for determining whether data it received was relevant to its work. The former involves searching through Europol’s repositories to see if information received is connected to other information held. The latter concerns the type of data processing that underpinned the “big data challenge,” examined above.

In both cases, the EDPS was damning of Europol’s failure to describe how the process would work. This left the EDPS unable to provide an opinion “on whether the intended processing is strictly necessary and proportionate for this purpose.” Failing to assess the necessity and proportionality of the proposed processing would risk “incompliance with the conditions of strict necessity and proportionality” set out in the Europol Regulation.[59]

The EDPS thus recommended that Europol undertake the relevant assessments of necessity and proportionality. The opinion also called on the agency to set out the categories of individuals in Analysis Projects to whom facial recognition would be applied. A “pilot project” approach to the NeoFace Watch system was recommended, “to allow evidence-based decision-making”. The results of that pilot project should be submitted to the EDPS.

The EDPS also sought further information on the accuracy of the NeoFace Watch algorithm for children under 12; to set out a plan for migrating images from the FACE system to the new system; and make sure that when images were deleted from NeoFace Watch, they could not be recovered.

There are several other issues mentioned in the EDPS opinion that were not examined in any more detail by the supervisory authority. The opinion notes that documents provided by Europol mention “the possibility to query external systems (such as those hosted by eu-LISA) with facial images,” but its data protection impact assessment did not examine this matter.

Despite noting that “the possibility to query external systems or to open Europol’s database to external queries considerably increases the impact of the processing on data subject’s rights and freedoms,” the EDPS did not investigate further.[60]

(return to top)

3.2.4 Data protection and European policing

Europol’s frequent requests for assistance from the EDPS to comply with the law will no doubt be seen as welcome by many. Indeed, the EDPS was appointed as supervisor of Europol’s personal data processing “to ensure strengthened and effective supervision.”[61] However, the documents analysed here indicate multiple shortcomings in Europol’s own data protection assessments. Whether or not the agency complied with the recommendations issued by the EDPS remains unknown, as this information is not made public.

Beyond this, however, there are structural questions that arise following the changes to the Europol Regulation that came into force in June 2022. As the EDPS itself has noted:

The amendments to the Europol Regulation… have shifted the balance between data protection and Europol’s operational needs, as it expands considerably the Agency’s mandate regarding exchanges of personal data with private parties, the use of artificial intelligence, and the processing of large datasets. We believe that these changes heighten the risks to individuals’ personal data.[62]

This highlights the political problem underpinning the changes to the law: governments and MEPs felt it appropriate to retroactively legalise prohibited practices.

The resource constraints on both EU and national data protection authorities leave many of them unable to fulfil their statutory duties.[63] In this context, it is hard to see how the promise of “strengthened and effective supervision” of an increasingly powerful agency can be met. This problem will become increasingly acute as Europol continues to acquire ever more data, and as it moves into the business of developing its own artificial intelligence tools and technologies, an issue examined further in section 4.

It should also be noted that even if the data protection regime for the agency were to function as intended, it would do nothing to alleviate the structural role of the agency in policies that cause widespread harm to groups and individuals – a political question that goes far beyond issues of data protection, and which requires a more fundamental reassessment of whether, where and how social issues should be treated as questions of criminality.

(return to top)

3.3 Frontex

Frontex is responsible for ensuring the development and implementation of the EU’s model of “European integrated border management” (EIBM). Amongst other things, this requires “the use of state-of-the-art technology including large-scale information systems.”[64]

As has been well-documented, this includes the use of technologies such as surveillance drones. In the Mediterranean, drones are used to spot boats departing from southern Mediterranean countries – such as Libya and Tunisia – so that the so-called Libyan Coast Guard can be directed to intercept them.

Many of the people intercepted will face “systematic and widespread abuse when forcibly returned to Libya.” Frontex is directly complicit in these human rights violations through its surveillance operations.[65] New technologies thus play a key role in further cementing the EU’s violent and harmful system of border management.

The AI Act requires that AI technologies should not “infringe on the principle of non-refoulement,” or be used “to deny safe and effective legal avenues into the territory of the Union, including the right to international protection.”[66] It remains to be seen whether this legal provision will lead to any change in current practices.

Every year, the agency’s management board decides the amount of technical equipment that will be needed for the agency’s operations, which can take place at EU and non-EU borders.[67] Equipment ranges from “lethal and non-lethal weapons” to binoculars, heartbeat detectors, cars and vans, drug and explosives detectors, and a host of systems for air, land and sea surveillance.[68]

Forms of AI will become increasingly integral to many of these tools and technologies, and the agency explicitly aims to increase the use of AI in its work. Indeed, tools for verifying travel documents are regulated as a form of AI under the Act, though they are not considered high-risk.[69] Frontex has also explored, with other EU agencies, ways to use AI for more intensive screening and monitoring of travellers.[70] In EU-level fora for advancing the use of AI, the agency has referred to one specific use case.

(return to top)

3.3.1 AI in the maritime domain

At the first meeting of eu-LISA’s Working Group on AI in May 2021, a Frontex representative was noted as saying that “one of the key areas is the maritime domain. Proof of concept has been conducted and capabilities have been procured and implemented.”[71] There is no further information provided, but this may be a reference to the agency’s procurement of the services of Windward, a company promising “predictive risk insights” for “proactively identifying behavioral [sic] patterns and uncovering hidden threats.”[72]

Windward was founded by two former officers in the Israeli navy. One of them, Ami Daniel, has said that the company’s software “can take all the shipping information on vessels, cargos, and companies, and put that together with one dynamic view of risk, and a profile of activity.”[73] Since December 2021, it has been listed on the London Stock Exchange.[74]

At the end of 2020 Frontex awarded the company a contract worth €2.6 million, with the possibility to renew it up for up to a further three years.[75] Windward also signed a one-year contract worth €3.2 million with the Greek government towards the end of 2023.[76] Even if the contract with Frontex were extended to the maximum length possible, it should by now have expired. It is unknown if the company is still providing services to Frontex. The EU has been seeking to develop its own AI tools to analyse and predict maritime traffic and vessel behaviour, through the PROMENADE project,[77] in which Frontex has taken an active interest.[78]

(return to top)

3.4 EU Asylum Agency

The EU Asylum Agency (EUAA) was established in 2021, and is the successor to the European Asylum Support Office (EASO). It is tasked with supporting member states in their implementation of EU asylum law and policy, including through “effective operational and technical assistance.”[79]

This can include the deployment of “asylum support teams” to aid in interviewing of asylum-seekers, or to provide interpretation services. These teams may be deployed alongside Frontex and Europol officials in areas designated as “hotspots” by the EU, thus enabling deployments of additional border guards, police officers and asylum officials.

EUAA officials deployed in Greece have been accused of “routinely fail[ing] survivors of pushbacks and survivors of human trafficking,” accusations which are the subject of an ongoing European Ombudsman inquiry.[80] EASO, the predecessor agency, was the subject of similar accusations.[81]

The agency has declared its interest in “the potential and challenges of artificial generative intelligence and applied machine learning,” and plans to work with other EU agencies “to leverage these and other technological innovations.” A role is seen for using AI in “collecting and analysing data and offering support functions for asylum procedures and reception services.”[82] One specific project is currently pursuing these ambitions.

(return to top)

3.4.1 Automated dialect recognition for asylum applicants

In September 2023, the EU Agency for Asylum (EUAA) announced its intention to develop a “Common European platform to identify the country of origin of [asylum] applicants through language assessment.” It has been given name CELIA: Common European Language Indication and Analysis.

Using AI technologies to determine people’s nationality would be given primacy over human assessment. Language and dialect analysis by human specialists would be relegated to the “second-line.” A pilot project “demonstrated that the model is feasible and can be implemented in practice,” and EU member states “expect the EUAA to play a central role in the establishment of a European system.”[83]

The first phase of the project runs from 2024-27 and will be undertaken by the Dutch authorities, with financial backing from the EU’s Asylum, Migration and Integration Fund (AMIF). This phase will focus on “Arabic dialects only.” It aims to:

validate “(semi-) automatic language analysis” techniques;
develop training and selection procedures for human analysts; and
produce a report on the legal and technical requirements for implementing the system.[84]

The academic Cecilia Manzotti has offered a preliminary assessment of some of the issues raised by such a tool. In her view, in the context of the new asylum and migration laws being introduced by the EU, there are substantial risks for individual rights.

All asylum applicants will be subject to a “screening” process under the new rules. This aims to identify applications that are likely to be considered inadmissible or unfounded, and to refer them to accelerated procedures with fewer safeguards.

Manzotti concludes that:

…the use of an AI language recognition tool to establish asylum seekers’ country of origin during the pre-entry screening would not be without consequences on the credibility assessment of asylum seekers’ nationality claims…

…combined with the systematic channelling of applicants from certain countries of origin into the border procedure [an accelerated assessment procedure], the use of automatic language indication would compromise the applicants’ right to seek asylum. With their asylum applications being examined at the border and under limited procedural guarantees, applicants identified as originating from certain countries would face significant obstacles asserting their claim and challenging the first instance authorities’ decision.[85]

(return to top)

3.5 Eurojust

The use of AI in the field of criminal justice is increasing at both national and EU level. The EU action plan for e-Justice includes a specific action on “Artificial Intelligence for Justice.”[86] A number of EU reports have explored the potential uses of AI technology in the criminal justice area. These are outlined in Annex II to this report. Amongst the potential use cases are:

automated document processing, for example by extracting and sorting relevant information;
automated production of case law summaries;
biometric recognition technologies for tracking individuals across video, photo or audio evidence; and
automated anonymisation of personal data in evidence.

In December 2024, the EU Justice and Home Affairs Council, made up of member states’ justice and interior ministers, agreed a set of conclusions on AI and justice systems. The conclusions emphasise that “final decision-making must remain a human-driven activity.” However, they also note with approval that judicial decision-making can be supported by AI.

In order to propel the development and uptake of AI technology in the justice sector, the conclusions call on the European Commission to create a “Justice AI Toolbox”:

…a repository of AI use cases (in particular, actors, scope, target, purpose, functionality, scenarios, expected benefits) and tools in the justice sector. The AI tools to be included in the toolbox, whether developed with or without EU funding, could be made available to all Member States.[87]

Even if final decisions in the judicial sector are made by humans, the use of AI technologies for justice – whether criminal or civil – poses enormous procedural and substantive risks. The AI Act prohibits judicial authorities using AI systems for “researching and interpreting facts and the law and in applying the law to a concrete set of facts, or to be used in a similar way in alternative dispute resolution.”[88] However, there are multiple other ways AI systems can be used within judicial procedures, and the EU is evidently keen on exploring them.

While many such systems are being developed and deployed in member states, the EU’s judicial cooperation agency, Eurojust, will also host and use AI technologies. Eurojust’s main task is to “support and strengthen coordination and cooperation between national investigating and prosecuting authorities” in relation to various criminal activities[89] affecting two or more EU member states.[90] It can act in response to requests from member states, on its own initiative, or following a request from the European Public Prosecutor’s Office.

Unlike other EU justice and home affairs agencies, Eurojust’s current work programme makes no specific mention of AI. However, it does make a commitment to “continue to improve our digital infrastructure.” There is at least one specific initiative at the agency making use of AI technology.

(return to top)

3.5.1 Joint Investigation Teams platform

Legislation approved in 2023 will see the EU’s judicial cooperation agency, Eurojust establish a new IT platform for “Joint Investigation Teams” (JITs). JITs are made up of “the competent authorities of two or more Member States” who wish to “carry out criminal investigations in one or more of the Member States setting up the team.”[91]

The platform is supposed to improve cooperation and coordination between members of JITs. It will incorporate “functionalities required for the coordination and management of a JIT,” including AI technology. At a minimum, it will provide a system for “machine translation of non-operational data” – that is, data concerned with financial and administrative issues.[92]

This is the only form of AI mentioned specifically in the legislation establishing the platform. At least on the surface, it seems to pose limited issues in relation to fundamental rights, particularly in relation to other forms of AI that could be used for criminal justice purpose.

There were initial suggestions that a revamped JIT platform could use AI for “analysis of unstructured data (e.g. named entity identification), analysis of different types of audio-visual media for the identification of crime victims or perpetrators, etc.”[93] It remains to be seen whether these will be developed and incorporated into the platform. A 2021 meeting report indicates that JITs were to be given access to a system developed by Europol for the same purposes.[94]

(return to top)

< Previous section
2. Cop out: security exemptions in the Artificial Intelligence Act

Next section >
4. Building the infrastructure

Notes

[1] More detailed information on each system is available in an interactive map produced by Statewatch: EU agencies and interoperable databases, https://www.statewatch.org/eu-agencies-and-interoperable-databases/

[2] ‘Automated Suspicion: the EU’s new travel surveillance initiatives’, Statewatch, July 2020, https://www.statewatch.org/automated-suspicion-the-eu-s-new-travel-surveillance-initiatives/

[3] List of third countries whose nationals are required to be in possession of a visa when crossing the external borders of the member states, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32018R1806#anx_I

[4] List of third countries whose nationals are exempt from the requirement to be in a possession of a visa when crossing the external borders of the member states for stays of no more than 90 days in any 180-day period, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32018R1806#anx_II

[5] For a detailed overview, see: ‘Content of and access to the CRRS’ in ‘Frontex and interoperable databases: knowledge as power?’, Statewatch, February 2023, https://www.statewatch.org/frontex-and-interoperable-databases-knowledge-as-power/

[6] Ibid.

[7] The report was produced as part of Lot 1 of the Framework Contract on the Transversal Engineering Framework (TEF), procured by Frontex and eu-Lisa in October 2020. See: ‘LISA/2019/OP/01’, https://ted.europa.eu/en/notice/-/detail/626975-2020

[8] A previous collaboration between Frontex and Europol also examined developments on this topic: ‘Future Group on Travel Intelligence and Border Management’, Statewatch, 7 September 2021, https://www.statewatch.org/observatories/frontex/access-to-documents-requests/future-group-on-travel-intelligence-and-border-management/

[9] eu-Lisa, ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, 14 November 2022, p.8, https://statewatch.org/wp-content/uploads/2026/05/eu-ai-in-crrs-profiling-travellers-final-report-2022.pdf

[10] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.88

[11] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.92

[12] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.95

[13] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.97

[14] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.98

[15] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.101

[16] ‘AI in CRRS in the Context of ETIAS and Revised VIS Final Report’, p.106

[17] eu-LISA, ‘Roadmap for AI initiatives at eu-LISA’, October 2021, https://statewatch.org/wp-content/uploads/2026/05/roadmap-for-ai-initiatives-at-eu-lisa.pdf

[18] Ibid.

[19] Recital (1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[20] ‘Council gives green light to the digitalisation of the visa procedure’, 13 November 2023, https://www.consilium.europa.eu/en/press/press-releases/2023/11/13/council-gives-green-light-to-the-digitalisation-of-the-visa-procedure/

[21] Article 2(2)(b), Regulation (EU) 2023/2667 of the European Parliament and of the Council of 22 November 2023 as regards the digitalisation of the visa procedure, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32023R2667

[22] Article 2(3), Regulation (EU) 2023/2667 of the European Parliament and of the Council of 22 November 2023 as regards the digitalisation of the visa procedure, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32023R2667

[23] Article 50(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_50

[24] Recital (8), Regulation (EU) 2023/2667 of the European Parliament and of the Council of 22 November 2023 as regards the digitalisation of the visa procedure, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32023R2667

[25] Article 111(1), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#art_111

[26] Deloitte study for European Commission, ‘Deliverable 3.01: AI Centre of Excellence definition’, HOME/2020/ISFB/FW/VISA/0021, undated, https://statewatch.org/wp-content/uploads/2026/05/deliverable-d3-01_-ai-centre-of-excellence-definition.pdf

[27] Deloitte study for European Commission, ‘Deliverable D2.01: Target architecture and design’, HOME/2020/ISFB/FW/VISA/0021, undated, https://statewatch.org/wp-content/uploads/2026/05/deliverable-d2-01-target-architecture-and-design-report.pdf

[28] Europol, ‘About Europol’, undated, https://www.europol.europa.eu/about-europol

[29] eu-Lisa Working Group on AI, 1st meeting minutes, 11 May 2021, https://statewatch.org/wp-content/uploads/2026/05/annex-10-minutes-1st-wgai-meeting-document-7-_redacted.pdf

[30] ‘Policing in an AI-Driven World’, Police Chief Online, 24 April 2024, https://www.policechiefmagazine.org/policing-ai-driven-world-europol/

[31] Europol, ‘Facilitation of illegal immigration’, undated, https://www.europol.europa.eu/crime-areas/facilitation-of-illegal-immigration

[32] Julie Bourdin et al., ‘The Human Toll of Europe’s ‘War on Smuggling’’, New Lines, 13 December 2022,
https://newlinesmag.com/reportage/the-human-toll-of-europes-war-on-smuggling/

[33] European Commission, ‘Commission reinforces EU rules and launches a Global Alliance to Counter Migrant Smuggling’, 28 November 2023, https://home-affairs.ec.europa.eu/news/commission-reinforces-eu-rules-and-launches-global-alliance-counter-migrant-smuggling-2023-11-28_en

[34] ‘Europol migrant smuggling proposal torn to shreds by the Council’, Statewatch, 10 May 2024, https://www.statewatch.org/news/2024/may/europol-migrant-smuggling-proposal-torn-to-shreds-by-the-council/

[35] ‘EU: Council’s plans for Europol: mass processing of personal data, simplified cooperation with non-EU states, artificial intelligence’, Statewatch, 24 November 2020, https://www.statewatch.org/news/2020/november/eu-council-s-plans-for-europol-mass-processing-of-personal-data-simplified-cooperation-with-non-eu-states-artificial-intelligence/

[36] ‘Policing in an AI-Driven World’, Police Chief Online, 24 April 2024, https://www.policechiefmagazine.org/policing-ai-driven-world-europol/

[37] EDPS, ‘EDPS Decision on the own initiative inquiry on Europol’s big data challenge’, 18 September 2020, https://statewatch.org/wp-content/uploads/2026/04/eu-edps-decision-redacted-inquiry-europol-big-data-challenge-10-20.pdf

[38] Annex II, Europol Regulation, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32016R0794#d1e32-109-1

[39] EDPS, ‘EDPS Decision on the own initiative inquiry on Europol’s big data challenge’, 18 September 2020, https://statewatch.org/wp-content/uploads/2026/04/eu-edps-decision-redacted-inquiry-europol-big-data-challenge-10-20.pdf

[40] Council of the EU, ‘Information sharing in the counter-terrorism context: Use of Europol and Eurojust’, 9201/16, 31 May 2016, https://statewatch.org/wp-content/uploads/2026/04/eu-council-c-t-info-sharing-9201-16.pdf

[41] ‘Europol unlawfully processing personal data of vast numbers of innocent people, says report’, Statewatch, 8 October 2020, https://www.statewatch.org/news/2020/october/europol-unlawfully-processing-personal-data-of-vast-numbers-of-innocent-people-says-report/

[42] EDPS, ‘Annual report 2021’, April 2022, https://www.edps.europa.eu/system/files/2022-04/2022-04-20-edps_annual_report_2021_en.pdf

[43] ‘Europol management board in breach of new rules as soon as they came into force’, Statewatch, 3 November 2022, https://www.statewatch.org/news/2022/november/europol-management-board-in-breach-of-new-rules-as-soon-as-they-came-into-force/

[44] EDPS, ‘EDPS takes legal action as new Europol Regulation puts rule of law and EDPS independence

under threat’, 22 September 2022, https://www.edps.europa.eu/system/files/2022-09/EDPS-2022-23-EDPS-request%20to%20annul%20two%20new%20Europol%20provisions_EN.pdf

[45] Appeal in Case T-578/22, 16 November 2023, https://curia.europa.eu/juris/document/document.jsf?text=&docid=281002&pageIndex=0&doclang=EN&mode=lst&dir=&occ=first&part=1&cid=1499515

[46] IBM, ‘Machine learning’, undated, https://www.ibm.com/think/topics/machine-learning

[47] EDPS, ‘Annual report 2021’, April 2022, https://www.edps.europa.eu/system/files/2022-04/2022-04-20-edps_annual_report_2021_en.pdf

[48] Ibid.

[49] ‘Euro police forces infiltrated encrypted phone biz – and now ‘criminal’ EncroChat users are being rounded up’, The Register¸ 2 July 2020, https://www.theregister.com/2020/07/02/encrochat_op_venetic_encrypted_phone_arrests

[50] ‘Dutch cops take out encrypted chat service SkyECC; Thirty arrests’, NL Times, 9 March 2021; https://nltimes.nl/2021/03/09/dutch-cops-take-encrypted-chat-service-skyecc-thirty-arrests

[51] There are references in the EDPS opinion to “messages”, “decrypted communications” and a “platform.” It has also been widely-reported that a Europe-wide Joint Investigation Team (JIT) was set up for the Encrochat investigation. The request sent by Europol to the EDPS refers to the use of machine learning “in the context of a specific Joint Investigation Team.”

[52] Daniel Boffey, ‘Colombia’s cartels target Europe with cocaine, corruption and torture’, The Observer, 11 April 2021, https://www.theguardian.com/world/2021/apr/11/colombias-cartels-target-europe-with-cocaine-corruption-and-torture

[53] EDPS, ‘Opinion on a prior consultation requested by Europol on the development and use of machine learning models for operational analysis (Case 2021-0130)’, 5 March 2021, p.1, https://www.edps.europa.eu/system/files/2022-06/22-03-05_europol-prior-consultation-machine-learning-opinion_redacted_en.pdf

[54] EDPS, ‘Opinion on a prior consultation requested by Europol on the development and use of machine learning models for operational analysis (Case 2021-0130)’, 5 March 2021, p.8, https://www.edps.europa.eu/system/files/2022-06/22-03-05_europol-prior-consultation-machine-learning-opinion_redacted_en.pdf

[55] Europol policy, ‘Development and Use of Machine Learning Tools for the Purpose of Supporting Operational Analysis at Europol’, 11 June 2021, EDOC # 1162317v5, https://statewatch.org/wp-content/uploads/2026/05/1162317-policy-on-the-development-and-use-of-machine-learning-tools-at-europol.pdf

[56] EDPS, ‘Supervisory opinion on a prior consultation requested by the European Union Agency for Law Enforcement Cooperation (Europol) on a face recognition solution’, Case 2023-1104, 20 December 2023, https://statewatch.org/wp-content/uploads/2026/05/edps23-12-20_edps_prior_consultation_opinion_en.pdf

[57] NEC, ‘NeoFace Watch’, undated, https://www.nec.com/en/global/solutions/biometrics/face/neofacewatch.html

[58] Europol, ‘Europol Analysis Projects’, undated, https://www.europol.europa.eu/operations-services-innovation/europol-analysis-projects

[59] DPS, ‘Supervisory opinion on a prior consultation requested by the European Union Agency for Law Enforcement Cooperation (Europol) on a face recognition solution’, Case 2023-1104, 20 December 2023, p.14, https://statewatch.org/wp-content/uploads/2026/05/edps23-12-20_edps_prior_consultation_opinion_en.pdf

[60] ‘Supervisory opinion on a prior consultation requested by the European Union Agency for Law Enforcement Cooperation (Europol) on a face recognition solution’, p.8

[61] Recital (51), Europol Regulation, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32016R0794

[62] EDPS, ‘Annual report 2022’, April 2023, https://www.edps.europa.eu/system/files/2023-04/23-04-26_edps_ar_2022_annual-report_en.pdf

[63] ‘Data protection: 80% of national authorities underfunded, EU bodies “unable to fulfil legal duties”’, Statewatch, 30 September 2022, https://www.statewatch.org/news/2022/september/data-protection-80-of-national-authorities-underfunded-eu-bodies-unable-to-fulfil-legal-duties/

[64] Article 3(1)(j), Frontex Regulation, https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32019R1896

[65] ‘EU: Frontex Complicit in Abuse in Libya’, Human Rights Watch, 8 December 2022, https://www.hrw.org/news/2022/12/12/eu-frontex-complicit-abuse-libya

[66] Recital 60, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[67] ‘EU: Frontex: equipment requirements for 2023 include “lethal and non-lethal weapons”’, Statewatch, 21 April 2022, https://www.statewatch.org/news/2022/april/eu-frontex-equipment-requirements-for-2023-include-lethal-and-non-lethal-weapons/

[68] ‘Management Board Decision 12/2023 of 22 March 2023’, https://prd.frontex.europa.eu/wp-content/uploads/mb-decision-12_2023-adopting-the-rules-relating-to-te-to-be-deployed-in-frontex-coordinated-activities-in-2024.pdf

[69] Annex III(7)(d), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#anx_III

[70] ‘EU: Police plans for the “future of travel” are for “a future with even more surveillance”’, Statewatch, 30 August 2022, https://www.statewatch.org/news/2022/august/eu-police-plans-for-the-future-of-travel-are-for-a-future-with-even-more-surveillance/

[71] eu-Lisa Working Group on AI, 1st meeting minutes, 11 May 2021, https://statewatch.org/wp-content/uploads/2026/05/annex-10-minutes-1st-wgai-meeting-document-7-_redacted.pdf

[72] Windward, https://windward.ai/industries/gov/

[73] LaToya Harding, ‘Maritime company Windward floats on LSE in first Israeli listing in five years’, Yahoo! Finance, 6 December 2021, https://uk.finance.yahoo.com/news/maritime-company-windward-floats-on-lse-in-first-israeli-listing-in-five-years-145603962.html

[74] ‘London Stock Exchange welcomes Windward Ltd. to AIM’, London Stock Exchange, 6 December 2021, https://www.londonstockexchange.com/discover/news-and-insights/london-stock-exchange-welcomes-windward-ltd-aim

[75] ‘Poland-Warsaw: Maritime Analysis Tools’, Tenders Electronic Daily, 12 January 2021, https://ted.europa.eu/en/notice/-/detail/10370-2021

[76] ‘Greece-Piraeus: License management software package’, 17 November 2023, https://ted.europa.eu/en/notice/-/detail/699946-2023

[77] ‘imPROved Maritime awarENess by means of AI and BD mEthods’, CORDIS, last updated 2 April 2024, https://cordis.europa.eu/project/id/101021673

[78] Frontex, ‘PROMENADE: Artificial intelligence and big data for improved maritime awareness, 29 March 2023, https://www.frontex.europa.eu/innovation/eu-research/news-and-events/promenade-artificial-intelligence-and-big-data-for-improved-maritime-awareness-NoxagQ

[79] Article 2, Regulation (EU) 2021/2303 of the European Parliament and of the Council of 15 December 2021 on the European Union Agency for Asylum, https://eur-lex.europa.eu/eli/reg/2021/2303/oj/eng

[80] ‘European Ombudsman opens an inquiry into how the European Union Agency for Asylum addresses allegations of fundamental rights violations in its activities in Greece’, I Have Rights, 17 July 2024, https://ihaverights.eu/european-ombudsman-opens-an-inquiry-into-how-the-european-union-agency-for-asylum-addresses-allegations-of-fundamental-rights-violations-in-its-activities-in-greece/

[81] ‘Greek hotspots: Complaint against European Asylum Support Office to the EU Ombudsperson’, ECCHR, undated, https://www.ecchr.eu/en/case/greek-hotspots-complaint-against-european-asylum-support-office-to-the-eu-ombudsperson/

[82] EUAA, ‘Single Programming Document’, 25 September 2024, https://euaa.europa.eu/sites/default/files/publications/2024-10/SPD_2025-2027_adopted_Sept_2024.pdf

[83] EUAA, ‘Strategy on Digital Innovation’, September 2023, pp.28-29, https://euaa.europa.eu/sites/default/files/publications/2023-10/2023_EUAA-Strategy-on-Digital-Innovation-in-Asylum-Procedures-and-Reception-Systems_EN.pdf

[84] Immigration and Naturalisation Service, ‘CELIA: Common European Language Indication and Analysis – phase 1’, December 2024, https://ind.nl/en/documents/12-2024/flyer-celia.pdf

[85] Cecilia Manzotti, ‘A European language detection software to determine asylum seekers’ country of origin: Questioning the assumptions and implications of the EUAA’s project’, ADiM Blog, November 2024, https://www.adimblog.com/wp-content/uploads/2024/12/Manzotti_DEF.pdf

[86] ‘European e-Justice Strategy and Action Plan 2019-2023’, Eur-LEX, last updated 12 June 2019, https://eur-lex.europa.eu/EN/legal-content/summary/european-e-justice-strategy-and-action-plan-2019-2023.html

[87] ‘Council Conclusions on the use of Artificial Intelligence in the field of justice’, 16 December 2024, https://data.consilium.europa.eu/doc/document/ST-16933-2024-INIT/en/pdf

[88] Annex III, Article 8(b), AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689#anx_III

[89] Annex I, Regulation (EU) 2018/1727, https://eur-lex.europa.eu/eli/reg/2018/1727/oj/eng#anx_%C2%A0I

[90] Regulation (EU) 2018/1727, https://eur-lex.europa.eu/eli/reg/2018/1727/oj/eng

[91] Council Framework Decision of 13 June 2002 on joint investigation teams, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32002F0465

[92] Article 6(a), Regulation (EU) 2023/969 of the European Parliament and of the Council of 10 May 2023 establishing a collaboration platform to support the functioning of joint investigation teams, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32023R0969#art_6

[93] eu-LISA, ‘Roadmap for AI initiatives at eu-LISA’, October 2021, https://statewatch.org/wp-content/uploads/2026/05/roadmap-for-ai-initiatives-at-eu-lisa.pdf

[94] “Another tool is Entity Extraction, which allows the electronic analysis of written text in order to extract identified entities and links by way of the use of Artificial Intelligence. Whilst currently only available internally within Europol, during 2021 the plan is that it will be deployed to external users in the EU Member States and third parties.” See: ‘Conclusions of 16th Annual Meeting of the National Experts on Joint Investigation Teams’, 10 November 2020, https://www.eurojust.europa.eu/sites/default/files/2021-02/conclusions_of_the_16th_annual_meeting_of_the_national_experts_on_jits.pdf

4. Building the infrastructure

This report considers two types of “infrastructure” required for the development of the EU security AI complex: institutional and technical. The former is made up various processes, working groups and other ‘spaces’ (whether formal or informal) that have been brought into use in recent years, principally since 2019. The latter consists of the hardware and software needed for the development of new security AI tools and techniques.

In this section

4.1 Institutional infrastructure

4.1.1 eu-LISA

4.1.2 The EU Innovation Hub for Internal Security

4.1.3 The European Clearing Board

4.1.4 Frontex

4.2 Technical infrastructure

4.2.1 Security Data Space for Innovation

4.2.2 Europol: sandboxes and pipelines

This report considers two types of “infrastructure”[1] required for the development of the EU security AI complex. The first is institutional infrastructure. This is taken to mean the different processes, working groups and other ‘spaces’ (whether formal or informal) that have been brought into use in recent years, principally since 2019.

Understanding this emerging institutional infrastructure is important for understanding the decision-making processes and forms of accountability that currently exist (or not) in relation to security AI. Examining it makes it possible to see which decisions have been taken, when and by whom. The analysis provided in this report is intended to spur further investigation and inquiry, with the aim of finding ways to make decision-making subject to democratic scrutiny and accountability. This is something that, so far, has largely been absent.

Technical infrastructure is taken to mean the hardware and software needed for the development of new security AI tools and techniques. In practice, the distinction between technical and institutional infrastructure may be blurred. Technical infrastructure is hosted and managed by different institutions – for example, as with Europol’s development of a “research and innovation pipeline” (section 4.2.2).

The distinction between institutional and technical infrastructure attempts to provide clarity over different elements of the security AI complex, though no such distinction can be clear-cut. What is evident, however, is that significant time, money and effort is going towards the development of new security AI tools, technologies and techniques. This is a topic that merits political and public interest, inquiry and scrutiny.

4.1 Institutional infrastructure

To embed AI development and deployment within internal security agencies in the EU, new forms of institutional cooperation and connection are being set up. Key to these efforts are the EU Innovation Hub for Internal Security, and a quiet plan to turn eu-LISA into an AI “centre of excellence” in the justice and home affairs policy field.

4.1.1 eu-LISA

Roadmap on AI initiatives

In October 2021, eu-LISA produced a “roadmap for AI initiatives”, which sought to provide “an overview of all existing and future (planned & potential, near to medium/long term) activities of the Agency in the area of AI in the JHA [justice and home affairs] domain.”

The roadmap was always a “draft document,” according to eu-LISA’s press office, as it was never formally adopted by the agency’s management board. However, the agency anticipates updating it once it has adopted a strategy on AI, according to the press office.

The roadmap included 10 initiatives:

Centre of Excellence for Artificial Intelligence in the JHA domain
VisaChat proof of concept (PoC) project (see section 3.1.3)
AI in ETIAS/CRRS
AI in the shared Biometric Matching System (sBMS)
European Security Data Space for Innovation
Supporting the development of AI in the justice domain
WGAI and transversal activities
Internal AI Proof-of-Concept (PoC) projects
AI Testing Lab
AI Training Activities

Here, we examine the proposal to create a Centre of Excellence on AI. That process began with the establishment of eu-LISA’s Working Group on AI.

Working Group on AI

In February 2021, the then-director of eu-LISA set up a Working Group on Artificial Intelligence (WGAI). This “informal advisory body” was assigned a number of tasks:

Providing a space for “the exchange of best practices and the discussion on opportunities and challenges arising from the implementation of AI-based solutions within the Agency’s mandate”;
Identifying “use cases for the implementation of AI solutions in the systems entrusted to eu-LISA”;
Helping to develop “a common approach for the use of AI-based solutions in the context of the operational management of large-scale IT systems in the JHA domain”;
Seeking ways to ensure “standardised solutions” were used by “stakeholders” when deploying AI technologies.

Although given a three-year mandate, the group was only in operation for 15 months, during which it met six times. The minutes of its meetings suggest that the WGAI was primarily a space for sharing information. The press office of eu-LISA also put forward this view, saying that the main achievements of the WGAI were “the creation of a forum for exchange of information, leveraged synergies and discussion on the implementation of AI-based solutions”.[2]

The minutes contain extensive notes on different EU and member state initiatives in the field of security AI (the agendas, minutes, and presentations that were used and discussed at the meetings will be published online alongside this report[3]).

However, the minutes also indicate the WGAI also provided a way for member states, EU agencies and EU institutions to coordinate certain policy initiatives, and to consider future possibilities.

For example, the second meeting, held in September 2021, saw a discussion on the results of a questionnaire circulated by eu-LISA to member state authorities. There were responses from 20 member states, mostly “from the Ministry of Interior and Police/Law Enforcement, however, there are some responses also from Immigration authorities and one Ministry of Foreign Affairs.”[4]

The responses indicated that while most member states had national AI strategies in place or under development, there were very few “specific strategies or roadmaps for AI in the area of JHA.” Nevertheless, member states were developing and deploying numerous AI technologies such as automated translation, entity recognition, chatbots, biometric recognition, and video analysis tools. Another “prominently featured topic for AI is big data analytics, like passenger name (PNR) data, passenger profiling and money laundering investigations.”

The survey sought to find out what types of AI technology member states considered it feasible to deploy. “More feasible” were technologies “under human supervision, analysis of internal/seized data, systems that don’t use personal data, biometric recognition in investigations and AI for data analytics for investigations.” On the other hand, unsupervised real-time biometric recognition or automated decision-making tools were considered “less feasible.”

The survey also sought member state views on what types of AI could be used in the large-scale IT systems managed by eu-LISA. Top of the list was the use of AI for “passenger profiling” in systems such as the European Travel Information and Authorisation System (ETIAS), on which efforts are ongoing (see section 3.2.1).

Other topics of interest included tools for predicting busy times at border crossings, “tools for victim identification using various media (voice/image/video),” ways to automatically analyse large datasets, such as Schengen Information System alerts, and detecting fraud in visa applications.

The sixth meeting of the group, in September 2022, discussed the results of a separate survey to member states. This sought to obtain opinions on the different potential uses for AI technology that were identified in a report written for the European Commission by consulting firm Deloitte, entitled ‘Opportunities and Challenges for the Use of Artificial Intelligence in Border Control, Migration and Security’.[5] In particular, it sought to identify “use-cases MS would like to see implemented in the near future.” One of those was the “visa chatbot” (section 3.1.3). All the use cases identified in that report are listed in Annex II to this report.

As well as providing a space to exchange information and coordinate activities between EU and member state agencies, eu-LISA’s Working Group on AI was also supposed to serve as the basis for the AI Centre of Excellence at the agency. Indeed, the creation of the WG was purportedly “the first step” towards the centre.

Centre of Excellence for AI in the justice and home affairs domain

First amongst the initiatives listed in eu-LISA’s AI roadmap was the proposal to create a “Centre of Excellence for AI (AI CoE) in the JHA domain.”[6] This would be “an overarching organisation” to coordinate AI initiatives “in the area of freedom, security and justice.”[7]

The eu-LISA roadmap on AI says the European Commission proposed establishing the AI CoE. However, the roadmap cites a report by international consultancy firm Deloitte as the source for this claim. That report, in turn, includes a legal notice, stating that while it was prepared for the European Commission, “the Commission cannot be held responsible for any use which may be made of the information contained therein.”[8] Whatever the exact origin of the idea, there was no democratic scrutiny or debate involved.

The roadmap goes on to say that to back up its proposal, the Commission’s Directorate-General for Home Affairs and Migration (DG HOME) acquired a more detailed study of what the Centre of Excellence might entail. That study, also carried out by Deloitte, was buried within the VisaChat project (see section 5.1.3).

Rather than set up the CoE in one fell swoop, the aim was to use the VisaChat project as a first step in its establishment:

…whilst defining the end state of the CoE is the main goal, starting the implementation of the CoE from a practical project, i.e. the Visa Chatbot, is desired. In other words, eu-LISA requested Deloitte to propose steps that could be gradually implemented.[9]

Like much of the documentation used in the preparation of this report, the Deloitte study could only be obtained by filing an access to documents request. Its objective was “to provide the reader with an AI Centre of Excellence (CoE) strategy and purpose, establish the CoE’s construct and operating model, and finally define its requirements.”[10]

This was done through “three workshops with eu-LISA and the European Commission (DG HOME) to define the AI CoE’s strategy and purpose, establish the operating model and define the technology requirements.”[11] The study underscores that the aim of the CoE would be to support “the EU JHA community in the development of AI tools and capabilities.”[12] It was proposed to do this by, amongst other things, coordinating “the strategy for AI within the JHA domain, to ensure a cohesive plan of implementation across all stakeholders,” and putting in place “frameworks for future projects to speed up the adoption of AI.”

These objectives have significant, wide-reaching implications, given the potential impact of security AI for the rights and liberties of individuals. It is remarkable that it was considered appropriate to set out the purpose and structure of the CoE on the back of three workshops with just two EU bodies, and no transparency or democratic scrutiny of any kind.

In any event, the initiative appears to have led to nothing. The press office of eu-LISA did not give direct answers to questions from Statewatch on the proposed CoE, but did say: “…should the Member State authorities and the European Commission consider that the creation of a Centre of Excellence for AI is necessary, the Agency will take the necessary steps to do so.”[13]

(return to top)

4.1.2 The EU Innovation Hub for Internal Security

“Cutting-edge products for the security of citizens”

In December 2019, EU member state interior ministers approved a plan for “a joint innovation lab within Europol to harness technological developments and trends, innovation and research.”[14] The aim was for the EU to take “a proactive role” to new technologies, increasing their uptake by police forces.

This led to the establishment of the EU Innovation Hub for Internal Security. The Hub is hosted by Europol’s Innovation Lab, within the agency’s Governance Directorate. It brings together representatives of all the EU’s justice and home affairs agencies, covering “law enforcement, border management, criminal justice and the security aspects of migration and customs.” It also receives input from member state authorities.

The Hub’s mandate was agreed by the Council’s internal security committee (COSI) in early 2020. It should:

…provide a joint EU platform to support the delivery of innovative cutting-edge products for the security of citizens in the EU, with a view to better assess the risks and foster the use and development of advanced and emerging technologies. The Hub will promote a culture of innovation and knowledge sharing across internal security actors in the EU and its Member States.[15]

So far, this has involved agencies involved in the Hub bringing different projects under its auspices, to coordinate support and involvement from other agencies. For example, amongst the “pilot projects” listed in its annual report for 2021 was “EU-coordinated Darknet monitoring to counter criminal activities.” This was led by the EU’s Joint Research Centre, with Europol and the European drugs agency as partners. As outlined below, AI-related projects have consistently been high on the Hub’s agenda.

The Hub held an initial meeting in 17 December 2020, with its first official meeting on 17 January 2021. It hosts an annual event, alongside more frequent internal meetings, and has recently established internal sub-groups (“clusters,” see below) dedicated to particular topics.

Formal membership of the EU Innovation Hub of Internal Security

EU Agency for Fundamental Rights (FRA)

EU Agency for Law Enforcement Training (CEPOL)

EU Asylum Agency (EUAA)

EU Counter-Terrorism Coordinator

EU Drugs Agency (EUDA, formerly European Monitoring Centre for Drugs and Drug Addiction)

eu-LISA

Eurojust

European Commission

European Institute for Gender Equality (EIGE)

Europol

Frontex

General Secretariat of the Council of the EU

The Hub secretariat has made continuous requests for an increased budget and for EU agencies to provide staff, but these do not seem to have been forthcoming. This is despite COSI setting a “requirement to Agencies participating in the Hub to second [provide] representatives,” according to a 2023 report by the Hub itself.[16]

There is clearly a political intention to use the Hub as a central coordination point for the development and use of new technologies in EU justice and home affairs policy, including security AI. That intention seems to have been hampered by financial and staffing limitations. How it develops in the future remains to be seen.

AI projects

According to the Hub’s annual report for 2023, “one of the most discussed topics in the Hub Team meetings was Artificial Intelligence.” In fact, AI projects have been on the agenda of the Hub since it was first launched. Since 2021, it has worked on:

Accountability Principles for Artificial Intelligence used in the Area of Freedom, Security and Justice (AP4AI);
Artificial Intelligence in the European Travel Information and Authorisation System (ETIAS) and Visa Information System (VIS);
Land border pilot project for the Entry/Exit System (requiring the testing of biometric capture and recognition technologies, classified as a form of AI under the AI Act); and
Technology Foresight on Biometrics for the Future of Travel (a report led by Frontex, published in 2022[17] and examined in more detail in a previous Statewatch report[18]).

The Hub has been reorganised for the 2024-26 period. It has shifted from being organised around ad-hoc projects, to a structure based on a series of “clusters”.

AI “cluster”

One of those “clusters” is dedicated to AI. It was launched in spring 2024. It sits alongside clusters on:

foresight and key enabling technologies;
biometric recognition systems: data quality, evaluation and standardisation;
encryption; and
knowledge management and innovation in training”

The report outlining the new structure also lists other topics of interest to the Hub: secure communication systems, drones, “virtual/extended reality,” the metaverse, and “privacy-enhancing technologies.”[19]

Following a brief kick-off meeting in March,[20] the first full meeting of the AI cluster took place in April 2024 at Europol’s headquarters in The Hague. It included a session to gain feedback from national members of the cluster on how their governments were implementing the AI Act; presentations on AI tools from the EU’s Joint Research Centre, Europol and the EU Agency for Law Enforcement Training (CEPOL), amongst others; and updates on the CC4AI (Compliance Checker 4 AI) project.[21]

A number of presentations made at the event were released to Statewatch in response to access to documents requests and are published alongside this report.[22] Several of these presentations demonstrate a concern with bias in AI systems. This is a major question for companies, institutions and agencies seeking to use AI technology. However, it has also been argued that this approach misses a more fundamental point.

The academics Agathe Balayn and Seda Gürses have offered an in-depth critique of “debiasing” approaches towards AI.[23] While these approaches will no doubt improve over time, it is hard to see how they can ever fully mitigate biases caused by structural, socio-political forms of discrimination. As Balayn and Gürses note:

…the technocentric solution of debiasing algorithms and datasets… squeezes complex socio-technical problems into the domain of design and thus into the hands of technology companies.[24]

It may of course be police forces, border agencies or other state institutions seeking to do the “debiasing,” rather than technology companies, but the problem remains the same. Balayn and Gürses were writing in 2021, and referred to debiasing as “a technocentric approach in the making.”[25] Four years later, it has arguably become even more institutionally-embedded, yet it remains an inadequate solution to AI systems produced by and for an unequal, unjust world.

A second meeting of the AI cluster took place in June 2024, and a third in September. It remains to be seen whether the cluster will introduce any form of transparency – for example, by publishing the agendas and minutes of its meetings – or whether this will fall upon researchers and journalists filing requests.

(return to top)

4.1.3 The European Clearing Board

Every EU member state has to designate a national policing body (its “national unit”) to serve as “the liaison body between Europol and the competent authorities of the Member State.”[26] An official must be appointed as the head of that unit. Collectively, those officials are known as the Heads of National Units (HENUs).

In 2020, the HENUs, on the initiative of the German Council Presidency and the German Federal Criminal Police Office (Bundeskriminalamt), agreed to establish a new body called the European Clearing Board (EuCB).[27] This was to provide a way for national police agencies to “directly steer the work of the Europol Innovation Lab,”[28] according to a Europol report.

The terms of reference of the EuCB, obtained by Statewatch via an access to documents request, shed more light on its role and remit. The terms of reference describe its mission: to “connect subject matter experts and investigators/analysts at working level with the aim of translating research results into practice and conveying requirements to the technical and strategic level.” It focuses on “tools, methods and innovative technologies used by police practitioners and supporting experts for data retrieval and analysis in the context of criminal police investigations.”[29]

The overall objective of the EuCB is to provide a way for EU and Schengen states to engage with the Europol Innovation Lab. Specifically, the terms of reference sets out seven aims:

Channel law enforcement agency needs from the operational level to the strategic level, and vice-versa;
Act as a central point for exchange of information and ideas on “innovative solutions” already in existence and/or use, and that are relevant to law enforcement agencies;
Discuss the creation of new Core Groups within the Europol Innovation Lab;
Disseminate the results of Innovation Lab and Core Group work;
Serve as relay between the Innovation Lab and EU and Schengen member states, and issue non-binding recommendations to the Innovation Lab on its work;
To establish “a process/methodology for innovation assessment and need evaluation”;
Identify projects with “cross-sectorial” aspects (that is, not solely related to law enforcement) that could be able at Innovation Hub

The EuCB is supposed to meet at least twice a year, in spring and autumn. These meetings are attended by one representative from each EU and Schengen member state – specifically, those states’ “Single Points of Contact” (SPoC) for the Innovation Lab. These officials can be supported by a deputy.

Meetings of the EuCB can also be attended by a representative of the Innovation Hub’s Steering Committee, and “the portfolio holder ‘Information Management’” of the Heads of Europol National Units (HENUs). Finally, representatives of any third country or international organisation with which Europol has an agreement to exchange personal data[30] can be invited to attend meetings as observers.

The EuCB has also adopted a specific set of rules regarding the establishment and management of Core Groups and Strategic Groups. An annex to the terms of reference states that a Core Group is:

…a working group focusing on a specific tool, project or technology relevant for law enforcement operational work in the EU Member States and the Schengen-associated countries. The result of a Core Group process should be the development of an innovative tool or method for the benefit of the practitioners and investigators of the European law enforcement community.[31]

The EuCB takes decisions on the opening or closing of core groups. Groups should be led by an EU member state or a Europol expert, and require commitment from several other member states. Financing to develop an “innovative tool or method” should “ideally” come from the participating member states. The document also encourages Core Groups to make use of EU budgets such as the Internal Security Fund.[32]

Strategic Groups are focused, unsurprisingly, on strategic topics rather than particular tools or methods. The document gives the examples of “ethics of AI use by LE, communication to the public on AI, or facial recognition, contribution to EU policy and legislation on data spaces etc.”[33] It goes on to say that Strategic Groups “will be used as a forum to express a law enforcement position on specific topics” – for example, the AI Act.[34]

Strategic Groups can be set up on the back of a suggestion from the Europol Innovation Lab or a member state, working through the EuCB. It is left to Strategic Groups themselves to define the timeline and objectives for their work.

The Innovation Lab itself is required to join all Core Groups and Strategic Groups, and to offer them various services. This includes coordination and logistics, a secretariat function, project management support, and communication and information-sharing. Furthermore, the Innovation Lab is supposed to maintain “dedicated thematic networks, comprising law enforcement and academia, research institutes, SMEs and industry,” that should be available to support the groups:

This should lead to co-creation where law enforcement agencies, academia and private sector develop meaningful solutions that enable law enforcement agencies to maximize the benefit of emerging technologies.[35]

Co-creation, however, is more relevant for the Core Groups. The role of the Strategic Groups is to advance the political views of law enforcement agencies – as with the Strategic Group on AI.

(return to top)

Strategic Group on AI

As of 2023, the Strategic Group on AI was being co-led by France and the Netherlands. Amongst other activities, it was lobbying governments to adopt police-friendly positions on the AI Act. According to one Innovation Hub document, these endeavours were successful:

The Group’s contribution triggered important changes in the Council position on the AI Act, including on the definition, classification of systems, remote biometrics, use of dactyloscopy and exceptions for law enforcement (mandatory publishing of AI-systems in use or that are developed by law enforcement agencies).[36]

Documents released to Statewatch indicate that the Strategic Group on AI held at least 19 meetings between the proposal of the AI Act and its adoption. The AI Act was an agenda item for all but one of those meetings. It is evident that the group followed negotiations in the Council, between EU member states, closely.

One agenda, from March 2022, notes that a meeting of the Council’s TELECOM Working Party would be “dedicated to LEA [law enforcement agency] related articles.” At the time, the group was working on a “two pager with our position as LEA’s [sic].” On 19 April, the group’s meeting included an update on the outcome of the TELECOM meeting, and in July the group was discussed: “Update on document drafted for WP TELECOM.” The agenda for the group’s March 2023 meeting is the most detailed of those released to Statewatch. It included:

Most important and impactful amendments on the AIA for LEA’s [censored]

Note that there are many changes proposed. Focus on the proposed prohibitions on

Biometric categorisation in (art. 5.1 ba)
Removing the exceptions for Law Enforcement in the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces (art 5.1 d, 5.2, 5.3. 5.4)
Al systems for individual risk assessment of natural persons or groups, making recidivism estimates, predicting offence occurrence (art. 5.1 da)
Al systems that create or expand facial recognition databases through the untargeted scraping of facial images (a1t. 5.1 da)
Categorising emotion recognition systems as High-Risk (Annex III under 1c) and changes in Annex III under 6. related to new prohibitions

And also have a look at recent developments with respect to regulation of generative Al (like use of LLM) and national position in EU countries on these proposed amendments[37]

In June 2023, the group discussed “Impact of the EP [European Parliament] Draft Compromise Amendments on our Position Statements paper,” and in February 2024 there was an agenda item on “Implementing the AI Act.” This appears to have led to the establishment of a separate working group on the implementation of the Act, which was adopted in June 2024. The agenda of the September 2024 meeting, the last one obtained by Statewatch, indicates ongoing engagement with implementation of the Act:

4. Updates Al Act (please also check previously sent information)

a. Guidelines

b. Al Board & subgroups/ Al Office meetings

c. LEWP [Law Enforcement Working Party] / IXIM [Working Party on information exchange for internal security] / COSI meetings

d. DG HOME Expert Group on Al.[38]

Whether the Strategic Group on AI is still active, and in what capacity, remains unknown. What is clear is that the EuCB claims to be responsible – at least in part – for weakening multiple different safeguards in the AI Act, and ensuring exemptions for law enforcement agencies. The documents obtained do not make it possible to verify that claim, but it is clear that significant work was put into trying to influence negotiations.

Needless to say, the EU treaties do not foresee a formal role for police agencies in negotiating new legislation, which is supposed to be a prerogative of the European Parliament and the Council of the EU. That they do so is unsurprising – but it is certainly unfortunate that the EU’s secretive and opaque law-making system makes it essentially impossible for the public to be aware of them.

It is also noteworthy that one of the Innovation Hub’s longest-standing projects is one that has developed a set of “accountability principles for AI.” Alongside these principles, the agency has developed a “compliance checker” – an online questionnaire – to let police forces examine whether a particular technology or technique is in line with the Act and the principles.

What we have here is a form of self-regulation. The use of security AI is subject to certain forms of external oversight and accountability under the AI Act, mainly by national and EU data protection authorities. That oversight is limited by the exemptions contained in the Act, and those authorities are already severely short on funding, staff and resources. The accountability principles and compliance checker provide a basis for arguments that no further external scrutiny or accountability is needed. It should also be noted that while the principles are public, the compliance checking tool is only available to law enforcement authorities.

(return to top)

4.1.4 Frontex

Frontex, the European Border and Coast Guard Agency, has long been involved in the EU’s research and innovation agenda.[39] In 2019 it received a new role, giving it the power to identify relevant topics for the EU’s security research programme, currently part of the broader Horizon Europe programme.[40]

The intention is for the agency to identify relevant research topics and evaluate proposals for research projects. It then oversees those projects, and aids in the acquisition and use of the resulting technologies and techniques.[41] This specifically includes “the use of advanced surveillance technology.”[42] All its projects revolve around forms of advanced technology, including some classified as AI under the AI Act.

Recent Horizon Europe projects have covered topics such as:

the use of “cosmic rays” for customs detection equipment at ports;
systems for maritime surveillance, including underwater surveillance;
technologies to analyse identity documents and detect fraud;
“pre-frontier” surveillance systems to monitor activities beyond the EU’s borders; and
systems to gather open-source data for interpreting foreign nationals’ perceptions of the EU and migration possibilities.[43]

The agency also has its own budget to fund research and other “ad-hoc” projects. These research projects cover similar topics to those funded through Horizon Europe. Recent projects include:

the development of coastal surveillance systems;
portable devices for inspecting travel documents; and
the use of drones to detect “land border violations.”[44]

Its 2024 budget for providing “ad-hoc grants,” whether for research or other projects, was just over €7.4 million.[45] Recent work by Algorithm Watch has investigated many of these research projects in more detail.[46]

As part of its growing role in the EU’s research and innovation agenda, Frontex has developed infrastructure for showcasing and testing new technologies. A new Border Management Innovation Centre (BoMIC) is described by the agency as “a Frontex lab-space designed to strengthen the European research and innovation capacity in the field of border security.”[47]

BoMIC is Frontex’s “innovation lab” and is the unit that participates in the EU Innovation Hub for Internal Security (section 4.1.2). It is intended to host a physical testing and demonstration space, including a 1300m² “Testing/Tech lab” and a 300m² “technology exhibition.” This is described in a Frontex presentation as a “key component of the new Frontex HQ.”[48]

That presentation was given at one of Frontex’s many “industry days,” and notes that it will “further enhance” cooperation with the EU’s Joint Research Centre. The key areas of interest for BoMIC are listed as:

border checks;
border surveillance;
border and coast guard equipment;
command, control, communications, computers and intelligence (C4I);
training and learning tools; and
“key enabling technologies” such as computer vision, the Internet of Things (IoT), quantum computing and autonomous systems.

The agency’s most recent programming document, covering the 2024-26 period, makes a number of specific references to AI. However, there are no meaningful details included.

The document says that “automated/AI based tools” will be used to “support” coast guard operations, and to increase sharing of data amongst “coast guard functions.” This is to contribute to the broader aim of increasing information sharing amongst different the different national and EU agencies and units that collectively make up the European Border and Coast Guard.[49]

One of the agency’s strategic objectives for the 2024-26 period is: “Reduced vulnerability of the external borders based on comprehensive situational awareness.” One activity contributing to this objective will be “constant situation monitoring and risk analysis.” One expected result is for “artificial intelligence solutions” to be deployed. Indeed, the use of AI systems is one of the indicators used to measure success.[50]

This means the gathering of large amounts of data, through border surveillance systems and other sources, whether powered by AI technologies or not. The resulting information would be used to inform strategic and operational decision-making: for example, on how many border guards to deploy at sites where refugees are seeking to enter EU territory.

(return to top)

4.2 Technical infrastructure

All artificial intelligence technologies need extensive, and expensive, technical infrastructure to operate. The result is lengthy and frequently destructive chains of extraction and dependency. These run from the mining of the materials needed to create the hardware on which AI software runs, to the data needed to feed the AI software itself, to the disposal of waste electronic equipment.

Some of the most extreme infrastructural requirements come in relation to energy: the company OpenAI, responsible for ChatGPT, “pitched the Biden administration on the need for massive data centers [sic] that could each use as much power as entire cities.” The company argued this was needed “to develop more advanced artificial intelligence models and compete with China,”[51] and companies are already acquiring nuclear power to run data centres.[52] An Executive Order signed by president Biden prior to leaving office introduces requirements for the US state to support further efforts.[53] Influential think tanks are making similar proposals for the UK.[54]

Remarkably, the EU’s AI strategy makes little mention of energy. It merely says the Commission will “support more energy-efficient technologies and infrastructure,” as part of its support for “technologies and infrastructure that underpin and enable AI.” This, the Commission says, will make “the AI value chain greener.”[55] Nor does the AI Act say much on the topic.[56] These limited commitments may be explained by politicians’ and officials’ reliance on the “false assumption that the digital and green transitions are ‘twins’ and always mutually reinforcing.”[57]

There is thus a significant material aspect to the EU’s security AI plans, though it goes completely unmentioned in any of the papers and reports related to those plans. Security AI is certainly likely to require less hardware, processing power, and thus energy than other, more large-scale forms of AI, though the issue remains extremely important. Nevertheless, one aspect of the infrastructure required for deployment of security AI has received substantial attention in policy and planning papers: the need for data.

(return to top)

4.2.1 Security Data Space for Innovation

One of the six political priorities during Ursula von der Leyen’s first term as Commission President was to create a “Europe fit for the digital age.”[58] Under this banner, a host of initiatives were launched, including the Digital Markets Act, the Digital Safety Act, and the AI Act, amongst others.

Underpinning these efforts was the European Data Strategy, announced in February 2020:

The aim is to create a single European data space – a genuine single market for data, open to data from across the world – where personal as well as non-personal data, including sensitive business data, are secure and businesses also have easy access to an almost infinite amount of high-quality industrial data, boosting growth and creating value, while minimising the human carbon [sic] and environmental footprint.[59]

Within this single European data space, there will be a set of underlying “common European data spaces.” These are expected to allow the pooling and sharing of data in different sectors, with a clear emphasis on using that data to develop AI technologies. As the Data Strategy notes:

The availability of data is essential for training artificial intelligence systems, with products and services rapidly moving from pattern recognition and insight generation to more sophisticated forecasting techniques and, thus, better decisions.

According to the European Commission, a data space has five key characteristics:

An IT infrastructure that is used to “pool, access, process, use and share data”;
A set of rules that determine the rights and obligations of those that can access the space;
A requirement for data owners to remain in control of the data they share with the space, and to set the purpose and conditions for re-use;
The presence of vast quantities of data that can be re-used either for free or for a price, depending on the data owner’s preference;
Participation by an “open number” of individuals and organisations.

Data lakes are described as repositories of structured and unstructured data at any scale, to which users have unrestricted access. The Commission describes data spaces, on the other hand, as “more like fish markets.” Unfortunately, this analogy is not explained in any more detail.

However, it is evident that the ultimate goal is for data spaces to be made up of separate but interconnected datasets held by different organisations and institutions. Access to that data would be granted to users of the data space on terms set by the owners of the data. Users would be provided tools “to enable discovery, access and analysis across industries, companies, and entities.” This is then supposed to facilitate the training and testing of new AI tools and algorithms, for which centralised infrastructure – for example, computing power or storage – may be required.

Some 20 data spaces have so far been announced.[60] These include health, agriculture, finance, mobility, energy, public administration, and security.

Making space for security

In the data strategy, there was reference to a data space “to address law enforcement needs.” A year later, in its organised crime strategy, the Commission announced the creation of a specific “European security data space that will be key to develop, train and evaluate tools for law enforcement.”[61] It should be “tailored to the needs of security and immigration stakeholders, including national authorities, EU agencies in charge of European security and justice representatives,”[62] although it appears that initially it will only target police forces.

Later rebadged the Security Data Space for Innovation (SDSI), the Commission has outlined a number of supposed benefits. As well as enabling the development of AI technologies, it would increase the EU’s “technological sovereignty,” through the creation of new national and European datasets. This, in turn, would “eliminate the threat of malicious interference of third countries/parties,” reduce the EU’s dependence on foreign companies, allow the EU to set its own quality standards, and “increase the technological capabilities of the national authorities.”

Aims of the Security Data Space for Innovation

The points below outline the “potential added value of the EU SDSI across dimensions,” according to a study carried out by the consultancy firm EY for the European Commission.[63] The text is the same as that in the study, but for ease of reading is not placed in quotation marks.

Trust and competences

EU SDSI as “one-stop-shop” for innovation in the area of law enforcement. Its aim is to contribute to build trust between LEAs and competences for research and innovation within LEAs [law enforcement agencies].
LEAs would be invited to use the EU SDSI “at their own pace”, e.g. to obtain information about innovation in LEAs, to get to know and interact with counterparts in other LEAs, to obtain support, to access non-sensitive data that could be used for innovation efforts etc.
EU SDSI would provide various services to LEAs, depending on need

Access to data

Access to higher quantity and quality of data for Member States’ LEAs:
- Quantity: Different types of non-sensitive data that LEAs may otherwise not have access to
- Quality: Compliance with agreed standards and principles ensures that data is useful for Member States’ innovation efforts
Data access in full compliance with EU and Member States’ applicable legislation. Member States should remain owner of their data and decide about its use.

Innovation readiness

Improved access to data is expected to facilitate Member States’ innovation efforts in the area of law enforcement, in particular with regard to the testing, training, validation, and approval of AI tools
EU SDSI to focus on specific use cases that are driven by the Member States, e.g. in the area of image recognition, video analysis, etc.
Member States will remain responsible for their own innovation efforts: EU SDSI is seen as an offer of support to Member States

Improved security

Improved ability of Member States is expected to facilitate maintaining the high level of security within the EU
EU SDSI is in principle relevant for all types of LEAs
Could be developed from covering police to e.g. border guards and/or customs in the future
Interaction of EU SDSI with other EU-level data spaces and Member State environments as relevant enabler for improved security
Link of EU SDSI to Europol Sandbox Environment potentially possible: Not only access to non-sensitive data but also improved algorithms based on sensitive data in enclosed facility.

Funding the security data space

In February 2022 a call for proposals under the EU’s Digital Europe Programme[64] was published. The call sought a project to develop the two key requirements for the SDSI. First, a governance model that would define data standards, infrastructure and telecommunication requirements, and so on. Second, the project was “to generate, collect, annotate and make interoperable data suitable to test, train and validate algorithms.”[65]

However, despite the Commission offering up to 50% of an expected €8 million total cost, the plan failed – no proposals were received in response to the call. According to the European Commission, despite strong interest in the idea from member states, they were not in a position “to guarantee sufficient engagement from the relevant national authorities and bring the necessary complementary funding.”[66]

A study by two international consultancy firms, EY and RAND Europe, was used to support the call for proposals.[67] This found that “law enforcement is not a high priority in national AI strategies” and limited uptake or use of “AI-based solutions and data spaces for law enforcement purposes.” It also found that law enforcement agencies make little use of data for innovation.[68] Nevertheless, at a March 2023 workshop hosted by the Commission, “stakeholders agreed that a security data space for innovation is a condition sine qua non for seizing the opportunities of AI and improving access to relevant data.”[69]

At the same time, a call for proposals was published under the Internal Security Fund, which finances policing and security projects. It noted that the SDSI required “incremental development in the coming years.” The call was smaller (with a €1 million budget) and less ambitious than the one published under the Digital Europe Programme. It sought “preparatory work needed for the creation of high-quality large-scale shareable data sets for innovation,” which would include work on data standardisation and anonymisation.[70]

The result was the TESSERA project, which started in March 2024 and consists of seven organisations, including the Spanish interior ministry and the Greek police.[71] It is expected to run for between two and three years. Its specific aims include mapping the types of datasets that could be shared through the SDSI. These include “photos, videos, voices samples, unstructured text (e.g. forums), unstructured hybrid data (e.g. web scraping or emails), structured data (e.g. telecommunication signalisation data).”

At the same time, the project will analyse “technologies that would allow the sharing of operational data, including but not limited to data anonymisation and generation of synthetic data sets, as a possible solution in cases of legal restrictions on collection and sharing of law enforcement datasets.” Other requirements include tools for classifying and annotating datasets, and tools for assessing data quality.[72] It sits alongside similar efforts, such as the Horizon Europe-funded LAGO project.[73]

Whether TESSERA will achieve its goals remains to be seen. It is, nevertheless, evident that developing a European infrastructure for creating new security AI tools and techniques is of significant interest to EU and certain national officials. Whilst the TESSERA project is taking some of the first steps towards developing the SDSI, a related initiative that is likely to contribute to the overall goal is ongoing at Europol.

(return to top)

4.2.2 Europol: sandboxes and pipelines

Europol has been involved in the development of new technologies for a number of years, and the reform of the agency’s mandate in 2022 gave it a stronger role than previously. The best example of this shift can be seen in its relation to the EU’s security research programme, which funds the development of new security technologies. It is currently known as Civil Security for Society and is part of the larger Horizon Europe programme. The agency has been a participant in a number of security research projects over the years. These have primarily sought to develop new big data and machine learning tools for law enforcement agencies.[74]

The new mandate that came into force in 2022 prevented it from participating in those projects. Instead, Europol was given a new remit: to help define priorities for the policing component of the security research programme.[75] Thus, it now has a strategic, agenda-setting influence over the types of investments made by the EU in new police technologies.

Alongside this new role, Europol has also started hosting annual “industry and research days,” much like other EU agencies. These invite private companies, academics and other researchers to present technical “solutions” to law enforcement problems. The first event was held in 2024, focusing on open source intelligence, “emerging platforms,” robotics and AI.[76] The 2025 event has a far broader scope, covering “robots or drones,” quantum technologies, “advanced biometrics for forensic analysis,” open source intelligence for analysing “private channels,” and the use of AI for “target detection, identification and tracking,” amongst other things.[77]

The agency has also launched a series of in-house technical initiatives to develop and make available new tools and techniques, many powered by AI. The agency ultimately aims to create what it calls a “Research and Innovation Pipeline” through which new technologies are taken from the research to the deployment phase. The intention is for many of these technologies to be “data-driven,” and thus there is significant work going into the development of the technical infrastructure needed to develop and test new algorithms. The evidence available suggests that Europol’s efforts in this area may be built upon to create the broader Security Data Space for Innovation.

Strategic research goals

According to an internal document from February 2023, Europol has seven strategic goals with regard to research and innovation:

strengthen member states’ technological capabilities that allow them “to identify, secure and analyse the data needed to investigate criminal offences”;
provide software that reduces manual work for police analysts;
prepare law enforcement authorities for dealing with increasing amounts of data;
support member states in using emerging technologies;
assist the European Commission in drawing up the security research programme;
“drive innovation and reinforce synergies in research and innovation projects”; and
“play a key role in promoting the development and deployment of ethical, trustworthy and human-centric artificial intelligence.”[78]

The agency is obliged to publish its “planned research and innovation activities” in its multiannual work plan. The document is not particularly revealing. It merely states that “Europol Research and Innovation projects will develop AI tools, trained with data provided by MS [member states] for that purpose, to facilitate investigations.”[79] The document also refers to the development of machine learning tools.[80]

The February 2023 document cited above is more specific about these aims, although it was only published in response to an access to documents request from Statewatch. It lists almost three dozen specific technologies for the automated analysis of images, video, text and audio, as well as tools for data presentation and processing (see Table 1: Europol’s priority research topics).

These include topics such as voice print analysis, age and gender detection from audio recordings of voices, and the use of augmented and virtual reality for data analytics. The same document states that “the development, use and deployment of new technologies are guided by the principles of transparency, explainability, fairness and accountability, do not undermine fundamental rights and freedoms and are in full compliance with Union law.”[81]

While this is a worthy aspiration, there is no guarantee that this will be the case. Furthermore, being compliant with the law may not provide as much protection for individuals as might be hoped. As explained in section 4.1.3, interior ministries and police officials helped to water down or remove many of the protections and safeguard that should have been included in the AI Act. The legal standards that must be met by new technologies are thus lower than may otherwise have been the case.

Table 1: Europol’s priority research topics

Image and video analytics

Pre-processing and preparation, Object detection, Image similarity search, Image classification, Face recognition, Personal features extraction, Fingerprint detection from fingerprints in images, Scene recognition, Scene classification, Indoor and outdoor image and geo-location, Deep fake and image/video tampering detection

Text analytics

Named entity extraction, Link extraction (“automatic extraction of links between extracted entities and presented for corroboration”), Robust translation of informal text, Text classification, Stylometric analysis

Audio analytics

Speech to text in low quality audio, Language detection, Age and gender detection from voice, Voiceprint analysis, Accent recognition in voice

Data representation and processing[82]

Graph Analytics, Clustering in network of nodes, AR/VR (augmented reality/virtual reality) data analytics

The European Data Protection Supervisor, the body responsible for oversight of Europol’s processing of personal data, has understandably taken an interest in the agency’s new research mandate. Towards the end of January 2024, the EDPS met with Europol officials (the Fundamental Rights Officer, data protection officials, and representatives of the Innovation Lab[83]) to get an understanding of how it was implementing the new provisions, and its research plans.

The EDPS was told at that meeting that the agency was drafting policies and processes to ensure it complied with the safeguards in its Regulation, and that it would inform the EDPS of these in due course. Information on planned projects would also be provided.[84] However, none of this was forthcoming – hence the EDPS letter, which was sent on 1 October, a little more than eight months after the meeting with Europol officials.[85] It requested that Europol provide to the EDPS:

documents that would allow an assessment of how the agency deals with data protection in research projects;
whether Europol “has started its research and innovation activities already,” or, if not, when it expected to do so; and
how the agency was complying with the rules governing the use of personal data in research projects.

A further request in the letter was censored.[86] Europol’s response, which was expected by 23 October 2024, remains unknown.

A digital playground for the police

Officials at the agency were evidently working hard to implement the new research and innovation mandate, even if they were not keeping the EDPS updated. One key priority is the development of a “research and innovation sandbox,” with the name ODIN: “Operational Data for Innovation.”

The term “sandbox” comes from the world of software development. There, it refers to an isolated technical environment in which software can be developed and tested without having any external effects. As one IT company puts it: “Think of a sandbox as a controlled playground where applications, code, and files can be tested or executed to see how they behave… a digital sandbox allows experimentation and testing without repercussions outside its confined space.”[87]

The AI Act introduces an obligation for national governments to set up “at least one AI regulatory sandbox.” The aim is “to facilitate the development and testing of innovative AI systems under strict regulatory oversight before these systems are placed on the market or otherwise put into service.” Despite its merits, this obligation also externalises the costs of “innovation” by private business onto the public. Access to these sandboxes is to be free, at least for small and medium sizes businesses and start-ups.[88]

Europol’s sandbox is aimed exclusively at the development of AI for law enforcement purposes, whether managed by Europol or by one or more EU member states. To use the wording quoted above, this sandbox would be a digital playground for the police. According to a document from April 2023, the overarching aim is for “value creation at speed”[89] – a term which recalls, at least in part, the Silicon Valley motto of “move fast and break things.”[90]

This apparent need for speed has led Europol to develop a “minimum viable product” version of the sandbox, “so that value creation may begin sooner.”[91] This wording echoes that of the EY and RAND Europe study on the SDSI. This noted that “the EU SDSI could develop from a basic version (Minimum Viable Product) as a short-term solution to an advanced version in the long-term, based on the needs of the Member States.”[92]

The “minimum viable product” approach means developing an environment that can only be accessed by Europol staff, as allowing external access would require stronger security measures and thus imply longer development times and higher costs. Equally, it would not be possible to use personal data in this environment for developing or testing AI tools.[93] This is described as resolving “the need for speed with the need for thoroughness.”

At the same time, the agency plans to develop a separate sandbox that will allow the use of personal data. However, access would still be limited to Europol staff. According to the April 2023 document, in this environment “tool selection and prototyping may already have happened. Now is the time to validate the new capability against a live, operational dataset containing personal data.”[94] If this proves successful, the “capability” may be put into use. The same document states that this sandbox could “become the core of the future EU Security Data Space for Innovation.”

From sandbox to pipeline

The sandbox is designed as one part of a larger whole, which Europol refers to as a “Research & Innovation Pipeline.” Through this, the agency would play a role in finding, testing, developing and deploying AI, machine learning, and other policing technologies. If a particular technology passes through all the stages, they “may progress to full operationalisation.”

Europol’s planned “Research and Innovation Pipeline”[95]

At the first stage, officials working for Europol, the EU Innovation Hub or the European Clearing Board would seek out promising technologies for further development (“opportunity scanning”). These may be “outcomes of the H2020 [security research] projects, or the results of the work of Core Groups or ad hoc collaboration with academic and industry,” a Europol document notes.[96]

The next phase would see “pre-selection and validation” of selected tools or technologies. Those chosen to progress further would be developed and tested within Europol’s “sandbox” using operational data, including personal data. “Technologies that pass through all relevant compliance and security checking of each stage may progress to full operationalisation,” according to the agency.[97]

One way in which technologies that pass through the pipeline may be put to use is through the Europol Tool Repository. This allows Europol and member states to share different tools and technologies via a platform managed by the agency. Europol describes it as “technology sharing as a new form of police collaboration.”[98] As of September 2023, it held 24 tools, had more than 1,500 users, and had seen more than 1,000 downloads of different tools, some of which “have already supported several operations across Europe.”[99]

Europol plans to “further develop the Innovation Pipeline concept” as one of its strategic priorities for the 2024-25 period. This indicates that the pipeline itself is not yet fully-functioning, or even fully planned. The sandbox, meanwhile, is describing as having “paramount strategic significance as it will enable Europol to fulfil its role in leading Law Enforcement Innovation.” It is described as “a precondition and enabler” and an “infrastructural foundation” for “numerous depending initiatives.”[100] This may well include the Security Data Space for Innovation.

(return to top)

< Previous section
3. Security in AI agencies

Next section >
Annex I: High-risk systems under the AI Act

Notes

[1] We have borrowed this term from the Mizue Aizeki, Laura Bingham, and Santiago Narváez: “Infrastructure – digital or material – has real sticking power; that’s the point. Once a highway splits a community in half, a new permanence stifles the din of protest, and people move on. We use the term digital infrastructure to describe the establishment of a foundation that will be fundamental to how world powers will practise migration control; and, as it embeds itself, increasingly beyond challenge – a unified strategic intervention by powerful countries, with the US coveting the vanguard. While it may look like technological experimentation (like AI-powered robot dogs on the border) or one-off opportunistic data-grabs (like networks of international data-sharing agreements), the growth of digital border infrastructure is by design. This is enabled through joined-up digital technologies that settle into the kind of rigid, ‘motiveless’ permanence granted to other infrastructures, like submarine communications cables, protocols and servers that run the internet, an electrical grid or a superhighway.” See: ‘The Everywhere Border: Digital Migration Control Infrastructure in the Americas’, Transnational Institute, 14 February 2023, https://www.tni.org/en/article/the-everywhere-border

[2] Email, 8 February 2025.

[3] A document archive will be published at: https://statewatch.org/securityai

[4] ‘Draft minutes of the 2^nd meeting of the Working Group on Artificial Intelligence’, 14 October 2021, https://statewatch.org/wp-content/uploads/2026/05/annex-11-minutes_2nd-wgai-meeting-document-8-_redacted.pdf

[5] Deloitte study for European Commission, ‘Opportunities and challenges for the use of artificial intelligence in border control, migration and security . Volume 1, Main report’, 28 May 2020, https://op.europa.eu/en/publication-detail/-/publication/c8823cd1-a152-11ea-9d2d-01aa75ed71a1/language-en

[6] eu-LISA, ‘Roadmap for AI initiatives at eu-LISA’, October 2021, https://statewatch.org/wp-content/uploads/2026/05/roadmap-for-ai-initiatives-at-eu-lisa.pdf

[7] Deloitte study for European Commission, ‘Deliverable 3.01: AI Centre of Excellence definition’, HOME/2020/ISFB/FW/VISA/0021, undated, p.3, https://statewatch.org/wp-content/uploads/2026/05/deliverable-d3-01_-ai-centre-of-excellence-definition.pdf

[8] eu-LISA, ‘Roadmap for AI initiatives at eu-LISA’, October 2021, https://statewatch.org/wp-content/uploads/2026/05/roadmap-for-ai-initiatives-at-eu-lisa.pdf

[9] ‘Deliverable 3.01: AI Centre of Excellence definition’, p.7

[10] ‘Deliverable 3.01: AI Centre of Excellence definition’, p.4

[11] ‘Deliverable 3.01: AI Centre of Excellence definition’, p.4

[12] ‘Deliverable 3.01: AI Centre of Excellence definition’, p.6

[13] Email, 8 February 2025.

[14] Justice and Home Affairs Council, ‘Outcome of the Council meeting’, 14755/19, 2 and 3 December 2019, https://www.consilium.europa.eu/media/42594/st14755-en19_final.pdf

[15] ‘EU Innovation Hub on Internal Security’, 5757/20, 18 February 2020, https://data.consilium.europa.eu/doc/document/ST-5757-2020-INIT/en/pdf

[16] Innovation Hub Team, ‘EU Innovation Hub for Internal Security – multi-annual planning of activities 2023-26’, Council doc. 5603/23, LIMITE, 16 February 2023, p.29, https://statewatch.org/wp-content/uploads/2026/05/1335957-v1-eu_innovation_hub_for_internal_security_multi-annual_planning_of_activities_2023-2026_st05603_en-public.pdf

[17] Frontex, ‘Frontex publishes technology foresight on biometrics for the future of travel’, 21 October 2022, https://www.frontex.europa.eu/innovation/eu-research/news-and-events/frontex-publishes-technology-foresight-on-biometrics-for-the-future-of-travel-us6C6v

[18] ‘Europe’s techno-borders’, Statewatch/EuroMed Rights, July 2023, pp.36-39, https://statewatch.org/wp-content/uploads/2026/04/europe-techno-borders-sw-emr-7-23.pdf

[19] ‘EU Innovation Hub for Internal Security – Work plan 2024’, Council document 7745/24 ADD 1, 27 March 2024, https://statewatch.org/wp-content/uploads/2026/05/europol-innovation-hub-report-7745_24_add_1.pdf

[20] EU Innovation Hub for Internal Security, ‘AI cluster workshop – kick-off meeting’, 1 March 2024, https://statewatch.org/wp-content/uploads/2026/05/doc-5_edoc-1369040-v2-draft_-_agenda_ai_cluster_-_kick_off_meeting_-_eu_innovation_hub_-_1_march_2024.pdf

[21] EU Innovation Hub for Internal Security, ‘AI cluster workshop – 8-9 April 2024, Europol HQ – Agenda’, https://statewatch.org/wp-content/uploads/2026/05/doc-6_edoc-1365887-v5-draft_-_agenda_-_eu_innovation_hub_-_ai_cluster_workshop.pdf

[22] The document archive will be published at: https://statewatch.org/securityai

[23] ‘Beyond debiasing: Regulating AI and its inequalities’, European Digital Rights, https://edri.org/wp-content/uploads/2021/09/EDRi_Beyond-Debiasing-Report_Online.pdf

[24] Ibid.

[25] Ibid.

[26] Article 7, Europol Regulation, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:02016R0794-20220628#art_7

[27] Europol, ‘Consolidated Annual Activity Report 2020’, p.13, https://www.europol.europa.eu/cms/sites/default/files/documents/consolidated_annual_activity_report_2020_caar.pdf

[28] Innovation Hub Team, ‘EU Innovation Hub for Internal Security – multi-annual planning of activities 2023-26’, Council doc. 5603/23, LIMITE, 16 February 2023, p.21, https://statewatch.org/wp-content/uploads/2026/05/1335957-v1-eu_innovation_hub_for_internal_security_multi-annual_planning_of_activities_2023-2026_st05603_en-public.pdf

[29] European Clearing Board – Tools, Methods and Innovations in the field of technical support of operations and investigations, ‘Terms of Reference’, 5 March 2021, https://statewatch.org/wp-content/uploads/2026/05/2021-03-05-edoc-1153324-v8-terms-of-reference-eucb.pdf

[30] Europol, ‘Operational Agreements’, undated, https://www.europol.europa.eu/partners-collaboration/agreements/operational-agreements

[31] European Clearing Board, ‘Annex to EuCB Terms of Reference – Management of Core Groups/Strategic Groups’, undated, https://statewatch.org/wp-content/uploads/2026/05/2021-03-05-edoc-1153324-v8-terms-of-reference-eucb.pdf

[32] ‘Section 4: Details of the security budgets’ in ‘At what cost? Funding the EU’s security, defence and border policies, 2021-27’, Statewatch/Transnational Institute, April 2022, https://eubudgets.tni.org/section4/#1

[33] ‘Annex to EuCB Terms of Reference’, p.3

[34] ‘Annex to EuCB Terms of Reference’, p.4

[35] Ibid.

[36] Innovation Hub Team, ‘EU Innovation Hub for Internal Security – multi-annual planning of activities 2023-26’, Council doc. 5603/23, LIMITE, 16 February 2023, p.21, https://statewatch.org/wp-content/uploads/2026/05/1335957-v1-eu_innovation_hub_for_internal_security_multi-annual_planning_of_activities_2023-2026_st05603_en-public.pdf

[37] European Clearing Board Strategic Group on AI, agenda for meeting on 21 March 2023, https://statewatch.org/wp-content/uploads/2026/05/2023-03-21-edoc-1422470-v1-sg-ai-agenda-21-march-2023_redacted.pdf

[38] European Clearing Board Strategic Group on AI, agenda for meeting on 18 September 2024, https://statewatch.org/wp-content/uploads/2026/05/2024-09-18-edoc-1422479-v1-sg-ai-agenda-18-september-2024_redacted.pdf

[39] ‘NeoConOpticon: The EU Security-Industrial Complex’, 17 February 2009, https://statewatch.org/wp-content/uploads/2026/04/neoconopticon-report-1.pdf; ‘Market Forces: the development of the EU security-industrial complex’, August 2017, https://statewatch.org/wp-content/uploads/2026/05/marketforces.pdf

[40] Article 66, Regulation 2019/1896, https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32019R1896

[41] Frontex, ‘Frontex to provide border security expertise to European Commission’s research projects’, 6 February 2020, https://www.frontex.europa.eu/media-centre/news/news-release/frontex-to-provide-border-security-expertise-to-european-commission-s-research-projects-ZrCBoM

[42] Article 10(1)(x), Regulation 2019/1896, https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32019R1896

[43] Frontex, ‘Horizon projects’, undated, https://www.frontex.europa.eu/innovation/eu-research/horizon-projects/

[44] Frontex, ‘Research grants’, undated, https://www.frontex.europa.eu/innovation/eu-research/research-grants/

[45] Frontex, ‘Budget 2024’, 4 March 2024, https://prd.frontex.europa.eu/document/budget-2024/

[46] ‘Automated Fortress Europe’, AlgorithmWatch, 2024, https://algorithmwatch.org/en/automated-fortress-europe/

[47] Frontex, ‘Border Management and Innovation Centre (BoMIC)’, undated, https://www.frontex.europa.eu/assets/EUresearchprojects/News/Day2/4_BoMIC.pdf

[48] Ibid.

[49] Frontex, ‘Single Programming Document 2024-2026’, 23 January 2024, p.168, https://prd.frontex.europa.eu/document/management-board-decision-8-2024-adopting-the-single-programming-document-2024-2026-including-the-multiannual-programming-2024-2026-the-work-programme-2024-and-the-budget-2024-the-establishment-plan/

[50] Frontex, ‘Single Programming Document 2024-2026’, p.38

[51] Shirin Ghaffary, ‘OpenAI Pitched White House on Massive Data Center Buildout’, Government Technology, 25 September 2024, https://www.govtech.com/artificial-intelligence/openai-pitched-white-house-on-massive-data-center-buildout

[52] Tobias Mann, ‘As AI booms, land near nuclear power plants becomes hot real estate’, The Register, 25 March 2024, https://www.theregister.com/2024/03/25/ai_boom_nuclear/

[53] ‘Advancing United States Leadership in Artificial Intelligence Infrastructure’, Executive Order 14141, Federal Register, 14 January 2025, https://www.federalregister.gov/documents/2025/01/17/2025-01395/advancing-united-states-leadership-in-artificial-intelligence-infrastructure

[54] ‘Revitalising Nuclear: The UK Can Power AI and Lead the Clean-Energy Transition’, Tony Blair Institute for Global Change, 2 December 2024, https://institute.global/insights/climate-and-energy/revitalising-nuclear-the-uk-can-power-ai-and-lead-the-clean-energy

[55] European Commission, ‘Artificial Intelligence for Europe’, COM(2018) 237 final, 25 April 2018, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52018DC0237

[56] However, it does include a requirement for standards that should aid in reducing “consumption of energy and of other resources” by AI systems (Article 40(2)). The Commission is obliged to report on the implementation of these standards (Article 112(6)).

[57] Claire Fernandez and Katharina Wiese, ‘The mirage of EU techno-solutionism to the climate crisis’, EUobserver, 7 January 2025, https://euobserver.com/Digital/ar125f5e3f

^[58] European Commission, ‘The European Commission’s priorities’, 16 July 2019, https://commission.europa.eu/strategy-and-policy/priorities-2019-2024_en

^[59] European Commission, ‘A European strategy for data’, undated, https://digital-strategy.ec.europa.eu/en/policies/strategy-data

^[60] European Commission, ‘Staff Working Document on Common European Data Spaces’, SWD(2022) 45 final, 23 February 2022, p.3, https://ec.europa.eu/newsroom/just/redirection/document/83562; European Commission, ‘Data space for security and law enforcement’, https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/digital-2022-cloud-ai-02-sec-law

[61] European Commission, ‘EU Strategy to tackle Organised Crime 2021-2025’, COM(2021) 170 final, 14 April 2021, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021DC0170

[62] European Commission, ‘Staff Working Document on Common European Data Spaces’, SWD(2022) 45 final, 23 February 2022, https://ec.europa.eu/newsroom/just/redirection/document/83562

^[63] Study to support the technical, legal and financial conceptualisation of a European Security Data Space for Innovation, 22 February 2023, p. 2, https://home-affairs.ec.europa.eu/document/download/4ad85efa-cccf-41ac-a3c7-84d33b5102d7_en

[64] This funds projects on supercomputing, AI, cybersecurity and digital skills.

[65] European Commission, ‘Data space for security and law enforcement’, https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/digital-2022-cloud-ai-02-sec-law

[66] European Commission, ‘Staff Working Document on Common European Data Spaces’, SWD(2024) 21 final, 24 January 2024, p.43, https://data.consilium.europa.eu/doc/document/ST-5855-2024-INIT/en/pdf

[67] EY and RAND Europe study for European Commission, ‘Study to support the technical, legal and financial conceptualisation of a European Security Data Space for Innovation’, 22 February 2023, https://home-affairs.ec.europa.eu/system/files/2023-02/Data%20spaces%20study_0.pdf

[68] European Commission, ‘Staff Working Document on Common European Data Spaces’, SWD(2024) 21 final, 24 January 2024, p.42, https://data.consilium.europa.eu/doc/document/ST-5855-2024-INIT/en/pdf

[69] ‘Staff Working Document on Common European Data Spaces’, p.43

[70] European Commission, ‘Call for proposals on data sets for the European Data Space for Innovation’, 21 March 2023, https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/isf/wp-call/2021-2022/call-fiche_isf-2022-tf1-ag-data_en.pdf

[71] TESSERA, ‘Consortium’, undated, https://tessera-project.eu/consortium-page/

[72] European Commission, ‘Call for proposals on data sets for the European Data Space for Innovation’, 21 March 2023, https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/isf/wp-call/2021-2022/call-fiche_isf-2022-tf1-ag-data_en.pdf

[73] ‘LAGO’, CORDIS, updated 15 September 2022, https://cordis.europa.eu/project/id/101073951

[74] GRACE (https://www.grace-fct.eu), STARLIGHT (https://cordis.europa.eu/project/id/101021797), AIDA (https://cordis.europa.eu/project/id/883596)

[75] Article 4(4), Europol Regulation, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:02016R0794-20220628

[76] Europol, ‘Europol Industry and Research Days: Invitation to present your product’, 10 November 2023, https://www.europol.europa.eu/publications-events/events/europol-industry-and-research-days-invitation-to-present-your-product

[77] Europol, ‘Europol Industry and Research Days 2025’, 3 February 2025, https://www.europol.europa.eu/publications-events/events/europol-industry-and-research-days-2025

[78] Europol, ‘Binding document defining the general scope for the research and innovation projects (application of Article 8 of the Management Board Decision further specifying procedures for the processing of information for the purposes listed in Article 18(2)(e) of the Europol Regulation)’, 23 February 2023, EDOC #1268633v8a

[79] Europol, ‘Programming Document 2024-26’, p.39, https://www.europol.europa.eu/cms/sites/default/files/documents/Europol_Programming_Document_2024-2026.pdf

[80] Europol, ‘Programming Document 2024-26’, p.46

[81] Europol, ‘Binding document defining the general scope for the research and innovation projects (application of Article 8 of the Management Board Decision further specifying procedures for the processing of information for the purposes listed in Article 18(2)(e) of the Europol Regulation)’, 23 February 2023, EDOC #1268633v8a, https://statewatch.org/wp-content/uploads/2026/05/europol-binding-document-on-research-and-innovation-projects.pdf

[82] “This work is also linked to the Data Refinery Area concept including, ways to process large volumes of data both structured and unstructured, in a forensically sound environment, data enrichment with OSINT, commercial databases and internal resources, creation and enrichment of ETL pipelines [Extract Transform and Load] including graphical, SQL (Structured Query Language) and relational databases, and data visualization tools including Virtual Reality/Augmented Reality (VR/AR).”

[83] EDPS, ‘Mission report: [censored] exchanges on the implementation of Article 33a Europol Regulation’, undated, https://statewatch.org/wp-content/uploads/2026/05/2024-0737_001_redacted.pdf

[84] Letter from Thomas Zerdick, Head of Unit, EDPS Supervision and Enforcement Unit, to Jürgen Ebner, Deputy Executive Director, Governance Directorate, Europol, ‘Request for information regarding the implementation of Article 33a of the Europol Regulation (Research and Innovation)’, 1 October 2024, https://statewatch.org/wp-content/uploads/2026/05/2024-0737_002_redacted.pdf

[85] Ibid.

[86] Ibid.

[87] Proofpoint, ‘What Is a Sandbox?’, undated, https://www.proofpoint.com/uk/threat-reference/sandbox

[88] Article 57, AI Act, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[89] Europol, ‘Building the Research and Innovation Pipeline: Update on the implementation of article 33a and the R&I Sandbox environment’, 17 April 2023, EDOC #1301551v2, document for meeting of the Information Management Working Group meeting on 16-17 May 2023, https://statewatch.org/wp-content/uploads/2026/05/europol-building-the-research-and-innovation-pipeline.pdf

[90] ‘Did Mark Zuckerberg Say, ‘Move Fast And Break Things’?’, Snopes, 29 July 2022, https://www.snopes.com/fact-check/move-fast-break-things-facebook-motto/

[91] ‘Building the Research and Innovation Pipeline: Update on the implementation of article 33a and the R&I Sandbox environment’, p.3, https://statewatch.org/wp-content/uploads/2026/05/europol-building-the-research-and-innovation-pipeline.pdf

[92] EY and RAND Europe, ‘Summary of the study in support of the Call for Proposals under the Internal Security Fund on Data Sets for the European Data Space for Innovation’, 20 February 2023, p.3, https://home-affairs.ec.europa.eu/system/files/2023-03/EU%20SDSI%20summary%20document_en.pdf

[93] Europol, ‘Building the Research and Innovation Pipeline: Update on the implementation of article 33a and the R&I Sandbox environment’, https://statewatch.org/wp-content/uploads/2026/05/europol-building-the-research-and-innovation-pipeline.pdf

[94] Europol, ‘Building the Research and Innovation Pipeline: Update on the implementation of article 33a and the R&I Sandbox environment’, https://statewatch.org/wp-content/uploads/2026/05/europol-building-the-research-and-innovation-pipeline.pdf

[95] Europol, ‘Building the Research and Innovation Pipeline: Update on the implementation of article 33a and the R&I Sandbox environment’, https://statewatch.org/wp-content/uploads/2026/05/europol-building-the-research-and-innovation-pipeline.pdf

[96] Europol Innovation Lab, ‘Progress Report and Strategic Priorities 2024-2026’, 22 September 2023, EDOC #1321956v13, p.4, https://statewatch.org/wp-content/uploads/2026/04/europol-innovation-lab-progress-report-and-plan-2023-25.pdf

[97] Ibid.

[98] Ibid.

[99] Ibid.

[100] Europol Innovation Lab, ‘Progress Report and Strategic Priorities 2024-2026’, p.5, https://statewatch.org/wp-content/uploads/2026/04/europol-innovation-lab-progress-report-and-plan-2023-25.pdf

Annex I: High-risk systems under the AI Act

The text below is extracted from Annex III of the AI Act.

1. Biometrics, in so far as their use is permitted under relevant Union or national law:

(a) remote biometric identification systems. This shall not include AI systems intended to be used for biometric verification the sole purpose of which is to confirm that a specific natural person is the person he or she claims to be;

(b) AI systems intended to be used for biometric categorisation, according to sensitive or protected attributes or characteristics based on the inference of those attributes or characteristics;

2. Critical infrastructure: AI systems intended to be used as safety components in the management and operation of critical digital infrastructure, road traffic, or in the supply of water, gas, heating or electricity.

3. Education and vocational training:

(a) AI systems intended to be used to determine access or admission or to assign natural persons to educational and vocational training institutions at all levels;

(b) AI systems intended to be used to evaluate learning outcomes, including when those outcomes are used to steer the learning process of natural persons in educational and vocational training institutions at all levels;

(c) AI systems intended to be used for the purpose of assessing the appropriate level of education that an individual will receive or will be able to access, in the context of or within educational and vocational training institutions at all levels;

(d) AI systems intended to be used for monitoring and detecting prohibited behaviour of students during tests in the context of or within educational and vocational training institutions at all levels.

4. Employment, workers’ management and access to self-employment:

(a) AI systems intended to be used for the recruitment or selection of natural persons, in particular to place targeted job advertisements, to analyse and filter job applications, and to evaluate candidates;

(b) AI systems intended to be used to make decisions affecting terms of work-related relationships, the promotion or termination of work-related contractual relationships, to allocate tasks based on individual behaviour or personal traits or characteristics or to monitor and evaluate the performance and behaviour of persons in such relationships.

5. Access to and enjoyment of essential private services and essential public services and benefits:

(a) AI systems intended to be used by public authorities or on behalf of public authorities to evaluate the eligibility of natural persons for essential public assistance benefits and services, including healthcare services, as well as to grant, reduce, revoke, or reclaim such benefits and services;

(b) AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score, with the exception of AI systems used for the purpose of detecting financial fraud;

(c) AI systems intended to be used for risk assessment and pricing in relation to natural persons in the case of life and health insurance;

(d) AI systems intended to evaluate and classify emergency calls by natural persons or to be used to dispatch, or to establish priority in the dispatching of, emergency first response services, including by police, firefighters and medical aid, as well as of emergency healthcare patient triage systems.

6. Law enforcement, in so far as their use is permitted under relevant Union or national law:

(a) AI systems to assess the risk of a natural person becoming the victim of criminal offences;

(b) AI systems in support of law enforcement authorities as polygraphs or similar tools;

(c) AI systems to evaluate the reliability of evidence in the course of the investigation or prosecution of criminal offences;

(d) AI systems for assessing the risk of a natural person offending or re-offending not solely on the basis of the profiling, or to assess personality traits and characteristics or past criminal behaviour of natural persons or groups;

(e) AI systems profiling of natural persons in the course of the detection, investigation or prosecution of criminal offences.

7. Migration, asylum and border control management, in so far as their use is permitted under relevant Union or national law:

(a) AI systems intended to be used as polygraphs or similar tools;

(b) AI systems to assess a risk, including a security risk, a risk of irregular migration, or a health risk, posed by a natural person who intends to enter or who has entered into the territory of a Member State;

(c) AI systems intended to be used for the examination of applications for asylum, visa or residence permits and for associated complaints with regard to the eligibility of the natural persons applying for a status, including related assessments of the reliability of evidence;

(d) AI systems intended to be used for the purpose of detecting, recognising or identifying natural persons, with the exception of the verification of travel documents.

8. Administration of justice and democratic processes:

(a) AI systems intended to be used to assist a judicial authority in researching and interpreting facts and the law and in applying the law to a concrete set of facts, or to be used in a similar way in alternative dispute resolution;

(b) AI systems intended to be used for influencing the outcome of an election or referendum or the voting behaviour of natural persons in the exercise of their vote in elections or referenda.

< Previous section: 4. Building the infrastructure

Next section: Annex II: Information to be registered in the EU database of high-risk AI systems >

Annex II: Information to be registered in the EU database of high-risk AI systems

The following text is taken from annexes to the AI Act. Text in bold italic does not apply to systems for law enforcement, migration, asylum and border control purposes.

ANNEX VIII: Information to be submitted upon the registration of high-risk AI systems in accordance with Article 49

Section A — Information to be submitted by providers of high-risk AI systems in accordance with Article 49(1)

The following information shall be provided and thereafter kept up to date with regard to high-risk AI systems to be registered in accordance with Article 49(1):

The name, address and contact details of the provider;
Where submission of information is carried out by another person on behalf of the provider, the name, address and contact details of that person;
The name, address and contact details of the authorised representative, where applicable;
The AI system trade name and any additional unambiguous reference allowing the identification and traceability of the AI system;
A description of the intended purpose of the AI system and of the components and functions supported through this AI system;
A basic and concise description of the information used by the system (data, inputs) and its operating logic;
The status of the AI system (on the market, or in service; no longer placed on the market/in service, recalled);
The type, number and expiry date of the certificate issued by the notified body and the name or identification number of that notified body, where applicable;
A scanned copy of the certificate referred to in point 8, where applicable;
Any Member States in which the AI system has been placed on the market, put into service or made available in the Union;
A copy of the EU declaration of conformity referred to in Article 47;
Electronic instructions for use; this information shall not be provided for high-risk AI systems in the areas of law enforcement or migration, asylum and border control management referred to in Annex III, points 1, 6 and 7;
A URL for additional information (optional).

Section B — Information to be submitted and kept-up-date by providers of AI systems that the provider has determined are not high-risk, in accordance with Article 6(3) of the Act

The name, address and contact details of the provider;
Where submission of information is carried out by another person on behalf of the provider, the name, address and contact details of that person;
The name, address and contact details of the authorised representative, where applicable;
The AI system trade name and any additional unambiguous reference allowing the identification and traceability of the AI system;
A description of the intended purpose of the AI system;
The condition or conditions under Article 6(3) based on which the AI system is considered to be not-high-risk;
A short summary of the grounds on which the AI system is considered to be not-high-risk in application of the procedure under Article 6(3);
The status of the AI system (on the market, or in service; no longer placed on the market/in service, recalled);
Any Member States in which the AI system has been placed on the market, put into service or made available in the Union.

Section C — Information to be submitted and kept up-to-date by deployers of high-risk systems

The name, address and contact details of the deployer;
The name, address and contact details of the person submitting information on behalf of the deployer;
The URL of the entry of the AI system in the EU database by its provider;
A summary of the findings of the fundamental rights impact assessment conducted in accordance with Article 27;
A summary of the data protection impact assessment carried out in accordance with Article 35 of Regulation (EU) 2016/679 or Article 27 of Directive (EU) 2016/680 as specified in Article 26(8) of this Regulation, where applicable.

ANNEX IX: Information to be submitted upon the registration of high-risk AI systems listed in Annex III in relation to testing in real world conditions in accordance with Article 60

The following information shall be provided and thereafter kept up to date with regard to testing in real world conditions to be registered in accordance with Article 60:

A Union-wide unique single identification number of the testing in real world conditions;
The name and contact details of the provider or prospective provider and of the deployers involved in the testing in real world conditions;
A brief description of the AI system, its intended purpose, and other information necessary for the identification of the system;
A summary of the main characteristics of the plan for testing in real world conditions;
Information on the suspension or termination of the testing in real world conditions.

< Previous section
Annex I: High-risk systems under the AI Act

Next section >
Annex III: AI technologies and techniques of interest to EU policing, migration and criminal justice institutions and agencies

Annex III: AI technologies and techniques of interest to EU policing, migration and criminal justice institutions and agencies

In this section

Criminal justice

Immigration and asylum

Border controls and policing

Administrative projects

Policy projects

Multiple purpose

This annex compiles information from official studies and report on potential uses of AI technology for criminal justice, policing and migration purposes. There is no indication that all of these uses of AI technology may be employed. However, some of those included in the reports are incredible invasive and raise significant legal and ethical questions. They are listed here to give an indication of the potential scope for applying AI technologies in these policy areas.

Criminal justice

Studies by Deloitte[1] and eu-LISA and Eurojust[2] have proposed a host of AI techniques that could be deployed for criminal justice purposes.

Natural language processing technologies

Unstructured data

Legal/criminal procedures and investigations use a lot of what is known as ‘unstructured data’. Unstructured data is normally stored in its original form in ‘data lakes’ – a type of massive data storage. It is not organised or standardised. Types of unstructured data include audiovisual, geospatial and text data. Unstructured data can be collected quickly and stored in a variety of ways. However, processing unstructured data requires specific tools and expert data science knowledge.

Automated document processing

Automated document processing comes in two main forms. The first is a technology known as “computer vision.” This allows a computer to ‘see’ and interpret images or visual information. When processing documents, the computer can ‘read’ text from images or scanned documents. This can then be used to convert read-only documents such as PDFs or scanned papers into text that can be edited and searched on a computer.

The second form of automated document processing is natural language processing (NLP). This technology lets computers ‘understand’ language, and produce it in a form that appears human. NLP allows computers to analyse the contents of a document and sort it accordingly into a user-friendly archive. It is also useful for extracting key information from (e.g. names, dates, addresses etc.) or providing summaries of documents.

These technologies allow to swiftly process large amounts of standardised documents.

Automated document processing can then allow more AI tools to be used, such as:

language translation;
e-Discovery (finding and analysing specific information, also known as ‘named entity recognition and classification’ (NERC)).

However, these tools must be used with extreme caution, especially in legal investigations, as the information extracted may be used as evidence. There is always a risk of error. Meaningful human oversight and decision-making is crucial.

Machine translation

This technology can translate material from one language to another, or even to/from multiple different languages. It can also assist in communication between parties that may speak different languages. This is particularly useful when analysing evidence that contains specialist terms, or is in a less common language.

To ensure that the translation system works well, it needs to be developed as a specialised system. This requires major resources and may be very expensive. Machine translated documents cannot be used as evidence.

However, automated translation is useful at the investigation stage in making evidence more accessible to the whole team. Machine translation provides valuable insight into which parts of the document are most important to have professionally translated (by a human) and sworn in as evidence. It also saves time and costs on sworn translations.

Automation summarisation systems

Summarisation technology can help condense large volumes of textual information, but cannot match human ability to interpret text. It can make information more accessible for further in-depth analysis by a human. Automated text summarisation has effectively been used in academic environments. Within the field of justice specifically, automated summaries produced by these systems may be incorrect.

In any case, any material made by automated systems will need to be approved by humans before being used as evidence. Users of these technologies cannot rely completely on the content produced by automated systems.

Evidence analysis and anonymisation

Natural language processing is currently mainly used in investigating ‘white collar crime’. These are types of crime that are essentially financially motivated and “non-violent” – for example, excise duty fraud or insider trading. Natural language processing techniques can help process digitised text that is often used as evidence in white collar crime cases: invoices, emails, contracts or shipping documentation, for example.

Natural language processing can also further develop communication tools between EU agencies to find more accurate links between cases. For example, the legislation regarding cooperation between Eurojust, Europol and the European Public Prosecutor’s Office (EPPO) introduces ‘hit/no hit’ search systems between the agencies. Natural language processing can strengthen this hit/no-hit function.

As well as being used for investigation and evidence analysis, named entity recognition and classification (NERC) can be used to protect documents/identities by anonymising or pseudonymising data (removing names or creating fake names).

Legal research and analysis

Legal research provides information relevant to a case. It normally involves reviewing statutes or case law. Natural language processing allows legal professionals to search for relevant information, such as relevant statutes, related legislation, case law, or doctrinal opinion. For cases that take place across different nation-states and use multiple languages, tailored solutions may be required. There are some initiatives seeking to install AI tools within legal analysis in Europe.

Biometric recognition and forensic analysis

Images recorded by CCTV cameras are often low quality, and therefore are of limited use by computer vision systems in criminal investigations. However, when the image is higher quality, computer vision systems are a valuable tool for identifying people of interest in recorded or live-streamed media. Specialised video/image enhancing algorithms have been developed to tackle the issue of image quality.

Machine learning algorithms are generally only considered robust when used in narrowly defined tasks. They are also only as good as the data they are trained on. The basic technology used in the analysis of images/video is exactly the same as the technology used for facial recognition by border control authorities or law enforcement. Therefore, biometric identification algorithms can be relatively easily used in forensic image/video analysis systems.

Anonymisation of visual data

Biometric anonymisation techniques can be used to anonymise images or video containing biometric identifiers (i.e. human faces). This can be used to protect the identities of victims, witnesses etc. This technology can also be used to remove identifiers such as car license plates. Anonymisation techniques can also be used on audio evidence (such as telephone conversations/recordings) to conceal/distort voices. This can again preserve the privacy and protect the personal data of victims/witnesses.

(return to top)

Immigration and asylum

The following potential uses of AI were included in a Deloitte report for the European Commission, based on surveys of and interviews with EU and member state officials.[3]

VISA-1 (application chatbot)

An AI chatbot (or virtual assistant) could be used to support individuals requesting a visa during the online application process. Potential uses:

Take in information and automatically fill in forms
Answer questions
Ensure the quality of information provided by the user by validating data

The chatbot could be enhanced with multiple languages/translation ability to answer questions in the user’s native languages.

VISA-3 (application triaging)

AI can be used to classify and sort visa applicants based on an initial AI assessment. This approach is similar to the process of triaging in healthcare where an AI can process scan images and doctors’ medical notes to allocate the appropriate next steps for a patient.

Classification categories could be defined based on a risk level or specific indicators. A more flexible/dynamic system could also group applications based on similarities it observes in the data, such as similar occupation, rather than rigidly defined categories.

As part of the risk assessment, the system could perform automatic searches within national and central databases (Schengen Information System, Visa Information System etc.) to check if there are any results that contribute to a certain risk level. The relevant response could be found in different ways depending on the system and the type of AI risk assessment model used. Automated searches of the EU’s databases with visa and travel authorisation applicants’ data are will be introduced under current EU legislation.

VISA-5 (consolidation of data for consular officials)

Consolidation of data from national and EU databases and other internal and external systems, to provide an overview and highlight key pieces of information which need to be checked manually.

VISA-6 (post-application chatbot)

An AI chatbot/virtual assistant can answer any questions the applicant has after they submit their documents. The chatbot can inform the applicant when action is needed to proceed to the next step in the application process.

VISA-7 (post-decision chatbot)

An AI chatbot/virtual assistant can answer questions from applicants after a visa decision has been made. The chatbot could support applicants in finding the right answer or direct them to the appropriate person/body to speak to. It could also automate scheduling follow-up appointments.

VISA-8 (identification of irregular travelling patterns)

AI can monitor, search and combine data from different sources such as the European Travel Information and Authorization System (ETIAS), the European Entry/Exit System (EES), and passenger name record (PNR) data from airlines. Using this, AI systems can detect ‘irregular’ travelling patterns. Irregularity can be detected from a sequence of stops or from the overall pattern of travel. AI could be used to identify patterns that were not previously observed as strange.

This resembles methods used in fraud detection to analyse spending behaviour, or in cybersecurity to analyse network traffic. AI is commonly used in both cases, particularly within the financial services industry.

VISA-9 (tailored application form)

AI could be used to create a personalised application form by tailoring questions to the applicant. Natural language processing could also be used for real-time personalisation by using information provided by the applicant as input data. Based on this, models could suggest questions – for instance, if a follow-up is likely based on historic factors.

A concrete example of this would be if the system logged a high chance of a traveller from a specific region overstaying. In this case the applicant could be asked more questions on why they are travelling to the Schengen area and asked to provide documents to support this. A human case worker would then be able to verify this.

It should be noted that this use of AI would raise questions of procedural fairness.

ETIAS-1 (risk assessment)

An AI model could be used to predict the risk level of an individual, even if there is no direct hit found in the first automated assessment. The model would perform a risk assessment of ETIAS applications by creating and flagging criteria which could appear ‘risky’. This could be based on the individual’s data, or based on criteria relevant to an ongoing scenario, or developed from reviewing profiles which would return a hit in ETIAS. It is this type of analysis that will be introduced in both the ETIAS and VIS under current legislation.

ETIAS-2 (classifying complex applications)

An AI triaging system could rapidly separate standard applications from more complex ones in need of human review by an appropriate case worker.

ETIAS-4 (application chatbot)

An AI chatbot can support individuals requesting ETIAS permits during the process of completing the online form.

ETIAS-5 (AI chatbot)

An AI chatbot/virtual assistant could answer any questions the applicant has after submitting their documents. It can inform the applicant what action is required to continue to the next stage of the application process.

ETIAS-6 (post-decision chatbot)

An AI chatbot/virtual assistant could answer frequently asked questions from an applicant after an ETIAS decision has been made. The chatbot could support the applicant in finding the correct answer or direct them to the appropriate person/entity to speak to. It could also automatically schedule follow-up appointments.

ETIAS-8 (visa or travel authorisation determination)

An AI model could determine whether an individual should undergo a travel authorisation or full visa procedure, independently of their nationality. This would shift current procedures from being based on nationality, to individual factors. This would involve the analysis of different risk factors, previous travel history, and so on. It could present recommendations for human review, flagging any significant information.

LTSTAY-1 (application chatbot)

An AI chatbot/virtual assistant can support individuals requesting a long-term residence permit.

This approach would be similar to VISA-1 and ETIAS-4.

Initially this case was intended to cover only the pre-application and pre-submission phase of an application for long term stay or residence. This might be expanded with support during and after the application (e.g. for renewal).

LTSTAY-2 (post-submission chatbot)

An AI chatbot/virtual assistant could answer any questions the applicant has after submission of their documents. It informs the applicant when they have taken action to continue to the next stage of the application process.

LTSTAY-3 (application triaging)

An AI triaging system could be used to automatically and quickly classify standard applications from more complex ones in need of human review. Classifying means grouping similar applications (e.g. from a certain country or reason for travel) for review by an appropriate expert. By grouping the applicants, the responsible officers would save time sorting the basic applications, allowing more time to be spent on those classified as complex. Technical approaches would be similar to VISA-3.

LTSTAY-4 (residence permit renewal chatbot)

A chatbot could help at the permit renewal stage. It could find previously submitted documents and help with any questions about what form of renewal to apply for, and so on.

LTSTAY-5 (post-decision chatbot)

To answer any questions the applicant has after the decision has been made on an application for a residence permit. It can support the applicant in finding the right answer or direct them to the correct person/entity to speak to.

LTSTAY-6 (facial recognition for family reunification)

This use case would see facial recognition deployed to determine if two people are related “based on facial characteristics.” This would be because “DNA testing is not always a viable option.” There is no evidence that facial recognition can be used to determine a familial connection between individuals.

LTSTAY-8 (AI to monitor “integration”)

AI could be used to assess “success in integration.” It could also analyse the drivers of success in individual cases. Data would be gathered through the use of an AI chatbot in contact with the immigrant, alongside data received from external sources (e.g. tax statements).

LTSTAY-9 (moving within the Schengen area)

A chatbot/virtual assistant could streamline interactions for individuals who have already received permits for long term stay in an EU member state, who wish to move to another member state.

ASYLUM-1 (grouping of candidates/cases)

A clustering algorithm could be used to group asylum candidates based on the similarity of their profiles and expected risk level. Risk level could be generated from data including applications, documents and interviews, in the form of both structured fields and/or less structured text/speech.

The aim of the tool would be to enable faster and more informed human decision-making by presenting similar cases or flagging notable pieces of information (e.g. outliers, similarities with other cases). Alternatively, it could be used in a post-decision context to assess historic consistency and quality of decision making.

ASYLUM-2 (asylum legislation assessment)

Applying AI to scan through national legislation to identify what is needed for a compliant asylum procedure. The results would be suitable for individuals looking to quickly gain an understanding of the procedure. This may be useful for training.

Alternatively, legislation could be cross-referenced to identify any differences in process in different Member States, for instance. This could help monitor the operation of the Common European Asylum System (CEAS).

ASYLUM-3 (vulnerability assessment)

AI could perform real-time analysis of an applicant’s facial movements, spoken language and body language. It could be used to detect “abnormal” patterns (e.g. signs of distress) which can better inform decision-making by a human social worker/expert (e.g. is the applicant should be granted special procedural guarantees).

Techniques would aim to notice and assess the emotional cues displayed by both what the applicant says/does and the way that they do it, either in terms of modelling apparent emotion types or by detecting fluctuating or unusual behaviour. This raises substantial ethical and legal questions. It should be noted that the infamous iBorderCtrl project had similar aims, for the purpose of border checks.

ASYLUM-4 (age assessment)

This would use an AI model to assess whether a person is a minor (either as a binary classification or by attempting to predict age). The AI model would likely require an image of a face, but could also include other physical factors in its assessment. The model could then be used in combination with existing techniques to enhance human judgement.

The AI could provide additional value by using other outputs extracted from machine learning models. This could include confidence scores and insights into which input data influenced the model’s output (e.g. which regions of a facial image). There are many ethical questions surrounding collecting samples of facial images from minors, and also regarding the reliability of the technology.

ASYLUM-5 (registration chatbot)

An AI chatbot for the asylum registration process. This would include:

providing information to an applicant to guide them through the process
data validation to ensure clean/correct inputs from the applicant
triggering automatic internal systems which currently require manual effort (such as booking interview slots, translators)

The chatbot would support the applicant by prompting for inputs (e.g. follow-up document requests), and by presenting a convenient way of dealing with frequently asked questions. The chatbot could also be deployed in a training setting – creating ‘virtual interviews’ to familiarise junior case workers with potential scenarios.

ASYLUM-6 (chatbot to aid refugee “integration”)

A post-entry chatbot/virtual assistant to aid refugee “integration.” It could respond to questions from recognised refugees. It could also provide suggestions to aid integration, such as local language classes and other events. It can also provide a means of monitoring other aspects of integration success.

ASYLUM-7 (abscondment risk assessment)

An AI model could predict the risk of an applicant absconding/leaving during the asylum procedure. It could take into account variables such as country of origin, previous application history, age, travel patterns, and so on.

ASYLUM-11 (refugee allocation)

AI can be used to place individuals in a certain region of a country where they are more likely to find work and integrate smoothly. AI could match an applicant’s skills to the region’s labour market. It could also factor in existing settlement of others from the same country of origin and total flow levels of asylum seekers to predict integration success.

ASYLUM-13 (assignment to detention centre)

AI could assign individuals seeking asylum to detention centres, “optimising the likelihood of positive integration with other individuals.” It could also consider capacity and cost constraints across the network of detention centres.

AI would consider variables including:

age
employment history
education
cultural background

It could suggest assignment to a particular centre based on this data, or streamline the presentation of this data for a human case worker for manual recommendation. It could also contain aspects of ASYLUM-11 (a similar analytics engine performing geographical assignment, e.g. to a particular member state) and ASYLUM-6 (a chatbot).

ASYLUM-14 (intelligent search engine)

AI could provide credibility assessment tests and enhance risk assessment for deportations to origin country. The specific assessment of individual risk must remain a human activity, but AI can enhance human decision-making. It could provide extracts/summaries of relevant information. Such a system could also be used to generate questions for asylum interviews.

ASYLUM-15 (remote surveillance of asylum-seekers)

Using AI in collaboration with Internet of Things (IoT) technology (e.g. sensors, GPS signals, cameras) to monitor asylum seekers. This is presented as a potential alternative to detention. The UK government has used this form of technology. Its own research found it made no difference to the likelihood of people absconding.

(return to top)

Border controls and policing

(return to top)

The following potential uses of AI were included in a Deloitte report for the European Commission, based on surveys of and interviews with EU and member state officials.[4]

SCHENGEN-1 (training chatbot)

Frequently asked questions chatbot for border officials, helping with interaction and data extraction from databases.

SCHENGEN-2 (AI to flag risk indicators)

An AI model to flag risk indicators during border check interview, based on data including the individual’s origin country, demographics, reason for travel, and so on.

SCHENGEN-3 (triaging border crossings)

AI model to triage border crossers into categories for a second line of action (e.g. interview/interrogation), most likely using historic trends based on the individual’s profile. It could also incorporate external factors such as seasonality and macro situation (overall economic conditions within a country/region).

Specifically, the system would analyse the entering travellers and divide them into a group that can proceed without passing a second border check, and a group that should go through the second check.

SCHENGEN-4 (border flow analytics)

This would apply predictive analytics to the total migration flows at both land and air borders in order to improve staffing planning. By collecting enough data on travel patterns and expected migration flows, it would be possible to reassign border guards to ensure that there are enough officers during migration peaks to deal with larger number of travellers or vice versa.

SCHENGEN-5 (analysing border guard decision-making)

AI could run analytics on border guard behaviour/decisions to understand trends, biases and potential inconsistencies. It could cross reference this with rejection/investigation rates to understand trends.

SCHENGEN-6 (facial and fingerprint recognition)

Facial and fingerprint recognition could be used to verify the identity of travellers more seamlessly, enhancing the current passport check. It should be noted that this is foreseen with the introduction of the Entry/Exit System.

SCHENGEN-9 (fingerprint image rotation)

Use of machine learning to rotate fingerprint images into the correct orientation for further use.

SISSIRENE-1 (alert detection)

Through the use of cameras at border crossing points an AI system could apply computer vision to detect Schengen Information System (SIS) alerts, such as identifying a target person or car number plate. The system would capture the image of the border crossers and send a notification to a border guard if there was a match with an alert stored in the SIS. The border guard could then validate/verify the obtained match and perform the activity requested in the alert. In late 2019 the transport agency of the Australian state of New South Wales announced it had installed computer vision technology into roadside cameras to spot offenders, similar to what is proposed here.

SISSIRENE-2 (automatic form-filling)

An AI chatbot could support SIRENE officers to fill in forms correctly and accurately. For instance, it could help officers to choose the correct form and suggest/provide inputs for the fields in the form. SIRENE stands for supplementary information request at the national entries. SIRENE offices and officers are responsible for exchanging information based on SIS alerts, for example between member states’ border or police agencies.

SISSIRENE-3 (automatic report creation)

AI could automatically create reports (natural language generation). The report would automatically create a summary of key indicators/trends found in the data, personalised to the user (police officer, border guard, etc.). For example, this could be used to create reports on alerted individuals, to whom officers should pay special attention. This can be tailored to the officers working in a certain geographical region or job type. Another example would be reports sent to government bodies to make them aware, e.g. increased numbers of Syrian refugees at the Italian border.

SISSIRENE-4 (knowledge search/management tools)

AI could be used to enable search and exploration of the SIS database. The AI could be tailored to the user and their ways of searching – through a chatbot interface for instance. It could work with a semantic layer to facilitate searching via natural language queries.

SISSIRENE-6 (automatic form completion)

A SIRENE form can be completed using the information from the original alert alongside a report from an officer highlighting the action taken. An AI system could be used to automatically recognise key information from the alert, and to collect and structure information from an officer’s report.

Specifically, using natural language processing, the system would work as a virtual assistant to gather the necessary information from the officer and automatically complete the SIRENE form. This information could then be sent to the other member state.

(return to top)

Administrative projects

The following potential uses of AI were included in a Deloitte report for the European Commission, based on surveys of and interviews with EU and member state officials.[5]

OPS-1 (energy usage analysis)

AI could collect and analyse sensor data to make energy usage more efficient within infrastructure operations or the data centre. For instance, it could optimise cooling mechanisms based on expected load of systems.

OPS-2 (load balancing)

AI can make eu-LISA systems operations and throughput more efficient by applying intelligent routing for load balancing.

OPS-3 (incident prediction)

AI could apply big data analytics to infrastructure performance metrics. This could automate the process of fault identification and recovery. In particular, the AI would focus on prioritising identified faults, based on expected complexity and impact.

OPS-4 (IT resource prediction)

An AI model could predict trends for the efficient provision of IT resources, such as network and storage.

OPS-5 (triaging chatbot for L1/L2)

The objective of this case is to reduce the burden on service desks within eu-LISA. Specifically, the goal is to improve efficiency for both internal staff (e.g. eu-LISA help desk) and external stakeholders (e.g. Member States). By improving waiting times and resolution time, ultimately ‘customer experience’ will also improve.

OPS-6 (improved biometric matching)

This case aims to improve the accuracy of biometric matching, specifically for facial images. This would ensure that officers do not lose time investigating false positive matches and that citizens are not unnecessarily stopped for an invalid reason.

State of the art AI methods would be used, refined specifically for eu-LISA. It could possibly include dataset generation to improve models in the absence of real training data.

OPS-7 (learning chatbot)

This case intends to speed up the learning process of various stakeholders by answering general questions related to newly-developed Core Business Systems at eu-LISA. For example, carriers will be required to use ETIAS to check if a traveller has a valid travel document. Similarly, border guards must be familiar with the ETIAS and EES when performing checks on TCNs.

As ETIAS will be a new system for these users, it will require learning from their side. Thus, this case intends to target such situations to improve their familiarity with the system, while providing any clarification needed. This case would require a chatbot created specifically for each of the systems and tailored to the user.

(return to top)

Policy projects

The following potential uses of AI were included in a Deloitte report for the European Commission, based on surveys of and interviews with EU and member state officials.[6]

POLICY-2 (Linking regulations)

AI can observe the links between regulations, both direct (referred) or indirect (similar regulations).

POLICY-3 (monitoring implementation of EU law)

An AI model can evaluate the extent to which member state legislation is compliant with EU legislation. The model could also potentially provide a downstream recommendation on what infringement procedures could be initiated against non-compliant member states.

POLICY-5 (clustering regulations)

AI can search for similar regulations based on substance and format. This would speed up development in terms of both knowledge and process (filling in templates). This would ensure consistency and appropriate dependencies between regulations.

POLICY-6 (gaps in regulation)

Identify gaps in regulations to focus on areas where regulation can help to better protect citizens and travellers.

POLICY-7 (terminology assessment)

AI can assess if terminology is consistent and highlight cases where wording could be improved. Also, a check could be performed to see if a regulation covers all the possible cases.

POLICY-8 (automated newsgathering)

Analysis of social media, newsfeeds, publications, legislative text by natural language processing and other big data analytics to identify trends.

POLICY-9 (stakeholder communication)

AI could assist with presenting and communicating new policies to civilians and business through visualisation techniques that use dimensionality reduction (and clustering). This can help communicate complex data/insights.

POLICY-11 (policy proposals)

During discussions between stakeholders (agencies, Member States, citizens, business etc.) a common denominator can be sought to select the policy option that addresses the majority of the demands. AI could extract all requirements from different stakeholders and look for an option which fulfils as many of them as possible.

POLICY-12 (predicting policy acceptance)

An AI model could use demographic and political data to assess if citizens or politicians will be in favour of or oppose a new policy regulation.

(return to top)

Multiple purpose

The following potential uses of AI were included in a Deloitte report for the European Commission, based on surveys of and interviews with EU and member state officials.[7]

CROSS-1 (document translation)

AI could provide translation to/from multiple languages of documents provided by the applicant and which need to be understood by an applicant/case worker/other concerned party during assessment of the case.

CROSS-2 (conversion from written to typed text)

AI can use optical character recognition (OCR) for converting handwritten forms into digital data that can be used with other computer systems. OCR would convert handwritten text into machine readable text data.

CROSS-3 (automated structuring of data)

AI can use optical character recognition (OCR) for converting physical forms into digital data that can be used with other computer systems. OCR would analyse the physical forms to create a digital structure containing the interpreted data (text, numbers etc.). The AI could also link into the appropriate downstream systems to further streamline operations.

For instance, instead of a case worker typing in travel details from a physical passport, the OCR system would automatically detect, extract and structure the data, and then store it in an appropriate location.

CROSS-4 (form completion checking)

An AI model could verify if an application is completed correctly before it is submitted. It could engage with the applicant to make them aware of any possible missing/inaccurate data.

CROSS-6 (forged supporting document detection)

AI could detect forged supporting documents such as birth certificates or bank statements. The model would identify signs of fraud:

unfamiliar/incorrect layout, such as misplaced logos and sections
inconsistencies within the content, such as misspelt names
inconsistencies between the content and other information provided

The model would flag potential cases for human review by highlighting areas of the document requiring further investigation.

CROSS-7 (historical case reasoning)

An AI case-based reasoning engine could analyse active cases by retrieving similar historic examples. These can then be presented to a caseworker for review when making decisions about a pending application. Similarity could be based on the applicant profile, “macro context,” or more custom indicators which might be developed internally. The tool could include a dynamic self-learning module. It could adapt the model using new knowledge received in the feedback cycle, and take it into account when processing new applications.

By flagging inconsistencies, caseworkers can be made aware of personal biases or identify general biases and receive appropriate feedback.

CROSS-8 (AI for monitoring AI systems)

AI can monitor other analytics and AI systems to uncover undesirable trends, such as a biased decision making. This could either analyse post-hoc results to check for imbalances between various groups of data or individuals. It could also analyse systems when applying explainability/interpretability techniques to machine learning models, to understand what variables are considered significant in decision making.

CROSS-9 (chatbot)

A chatbot can inform applicants on their rights and the possibilities of appealing a denied application. This is with the aim of ensuring a fair and effective process.

CROSS-10 (assessing human bias)

An AI model could monitor human bias when assessing an application by performing a post-hoc quality/fairness assessment. It would search for correlations between applications and outcomes.

CROSS-12 (forged travel document detection)

By using computer vision, an AI system could detect the use of forged travel documents. It would analyse an image of the provided documents and assess if the physical characteristics of the document match an original one. It assesses if the information provided in the documents is accurate, and the person providing the documents corresponds to the person in the document, rather than a lookalike. The technical approach would be similar to CROSS-6.

CROSS-13 (real-time translation)

AI can provide real-time translations of a discussion between an officer and an applicant during an interview.

CROSS-20 (post-application monitoring)

AI can scan and monitor different systems to assess if the conditions in which a permit was granted to a third-country national (TCN) still apply. The system would use the data from those systems to assess the likelihood of an applicant not complying with the terms set when issuing the permit.

For example, a TCN might receive a residence permit because of marriage to an EU resident. However, the couple could separate soon after the permit is issued. In this case, the conditions for initially providing the permit no longer apply. The system would try to assess whether conditions for the permit are still valid by analysing various sources of data (e.g. address or tax information) and provide insight on the possibility of fraud.

Another example is to monitor if a TCN is complying with the restrictions of the issued work permit, such as number of days worked. This could be checked by analysing tax statements.

CROSS-21 (AI to assist with optimising detention centre allocations)

AI could make detention centre allocation more efficient. It would predict when the individual might be deported, which would come from a model fed with the person’s information and risk assessment data. More data can be processed by prioritising individuals who are likely to be deported quickly. In particular, this case is highlighted as being appropriate for irregular migrants/Dublin cases/deportation.

CROSS-23 (general EU chatbot)

Use of a chatbot could support travellers/citizens with generic questions regarding the rights and obligations within the Schengen area. For instance, information about the right to work for a limited time period and obligation to leave the country after 90 days within the Schengen area.

(return to top)

< Previous section
Annex II: Information to be registered in the EU database of high-risk AI systems

Notes

[1] ‘Cross-border digital criminal justice’, 2020, https://op.europa.eu/en/publication-detail/-/publication/e38795b5-f633-11ea-991b-01aa75ed71a1/language-en

[2] eu-LISA and Eurojust, ‘Artificial intelligence supporting cross-border cooperation in criminal justice’, June 2022, https://www.eurojust.europa.eu/sites/default/files/assets/artificial-intelligence-cross-border-cooperation-criminal-justice-report.pdf

[3] ‘Opportunities and challenges for the use of artificial intelligence in border control, migration and security’, 2020, https://op.europa.eu/en/publication-detail/-/publication/c8823cd1-a152-11ea-9d2d-01aa75ed71a1/language-en

[4] ‘Opportunities and challenges for the use of artificial intelligence in border control, migration and security’, 2020, https://op.europa.eu/en/publication-detail/-/publication/c8823cd1-a152-11ea-9d2d-01aa75ed71a1/language-en

[5] ‘Opportunities and challenges for the use of artificial intelligence in border control, migration and security’, 2020, https://op.europa.eu/en/publication-detail/-/publication/c8823cd1-a152-11ea-9d2d-01aa75ed71a1/language-en

[6] ‘Opportunities and challenges for the use of artificial intelligence in border control, migration and security’, 2020, https://op.europa.eu/en/publication-detail/-/publication/c8823cd1-a152-11ea-9d2d-01aa75ed71a1/language-en

[7] ‘Opportunities and challenges for the use of artificial intelligence in border control, migration and security’, 2020, https://op.europa.eu/en/publication-detail/-/publication/c8823cd1-a152-11ea-9d2d-01aa75ed71a1/language-en

Automating Authority: Artificial intelligence in European police and border regimes

Summary analysis

Full report

Publication information

Next section >Contents

Contents

< Previous sectionPublication information

Next section >Summary analysis

Summary analysis

Mainlining AI

Suprnational AI

Security AI: exclusion and discrimination

Security AI complex

The AI Act: exceptions, exemptions and loopholes

The exception is the rule: the security AI imaginary

The police lobby: watering down safeguards

“Cutting-edge products for the security of citizens”

A “centre of excellence” for security AI?

Public-private partnership

From institutional to technical infrastructure

Europol: AI sandbox

Questioning: the security AI complex

< Previous sectionContents

Next section >Acronyms and abbreviations

Notes

Acronyms and abbreviations

< Previous sectionSummary analysis

Next section >1. Introduction

1. Introduction

A note on terminology and definitions

< Previous sectionAcronyms and abbreviations

Next section >2. Cop out: security exemptions in the Artificial Intelligence Act

Notes

2. Cop out: security exemptions in the Artificial Intelligence Act

2.1 “A historic achievement”

2.2 Summary: exceptions and loopholes

2.3 In detail: the AI Act’s security exemptions

2.3.1 Scope and application of the law

Military and national security

Data-laundering overseas

Geographical discrimination

Research and testing

Temporary general exemptions from the Act

“Sensitive operational data”

2.3.2 (Un)prohibited practices

Profiling

Biometric categorization

Mass biometric surveillance

2.3.3 Risk and impact assessments

Self-assessment

Impact assessments

2.3.4 A “silicon curtain” of secrecy

EU database of high-risk AI systems

The right to information

Watermarking of AI-generated media

Notification that text has been generated by AI

Right to explanations

Information on and consent to being subjected to “real-world testing”

2.3.5 Conformity assessment

External oversight

Ongoing supervision

Urgent authorisation and deployment without authorisation

2.3.6 Data protection

Purpose limitation

2.3.7 Oversight

Reporting of serious incidents

Specific supervisory authorities

Ability to prevent exchanges of information between supervisory authorities

Control over access to technical documentation

2.4 Implementing the Act

< Previous section1. Introduction

Next section >3. Security AI in EU agencies

Notes

3. Security AI in EU agencies

3.1 eu-LISA

3.1.1 Algorithmic profiling of travellers

3.1.2 AI in the shared Biometric Matching System

3.1.3 Digitalising the visa application process: visa chatbot

3.2 Europol

3.2.1 From challenge to opportunity

Next section >
Contents

< Previous section
Publication information

Next section >
Summary analysis

< Previous section
Contents

Next section >
Acronyms and abbreviations

< Previous section
Summary analysis

Next section >
1. Introduction

< Previous section
Acronyms and abbreviations

Next section >
2. Cop out: security exemptions in the Artificial Intelligence Act

< Previous section
1. Introduction

Next section >
3. Security AI in EU agencies

< Previous section
2. Cop out: security exemptions in the Artificial Intelligence Act

Next section >
4. Building the infrastructure

< Previous section
3. Security in AI agencies

Next section >
Annex I: High-risk systems under the AI Act

< Previous section
Annex I: High-risk systems under the AI Act

Next section >
Annex III: AI technologies and techniques of interest to EU policing, migration and criminal justice institutions and agencies

< Previous section
Annex II: Information to be registered in the EU database of high-risk AI systems