## 1. Implement a Formal Policy on AI

## Source details

**Canonical URL:** [1. Implement a Formal Policy on AI](https://www.imf.org/-/media/files/publications/tnm/2024/english/tnmea2024006.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/tnm/2024/english/tnmea2024006.pdf.md)
- [Structured JSON version](/-/media/files/publications/tnm/2024/english/tnmea2024006.pdf.json)

---

### Overview and purpose
- Provides senior officials with an overview of current thinking on artificial intelligence (AI) in tax and customs administration to inform decision making.
- Addresses historic and ongoing AI developments, legal and ethical aspects, principled concerns, and recent regulatory initiatives.
- Presents the concept of an AI “use case,” guidance on promoting AI’s responsible use, and a process for introducing AI into an operational setting.
- Annexes 1−3 contain tools for AI policy, strategy, and risk assessment.

### Rationale: why a formal AI policy is needed
- Generative AI (GenAI) is likely to increase AI use substantially in everyday tasks, supporting core operations or enhancing desktop tools.
- AI raises issues of certainty in two principal ways:
  - Economic impact on tax bases and trade flows, and follow-on effects to policy.
  - Practical implications, utility, and management from an administrative perspective.
- “Certainty of action”—strengthened by clear, predictable, fair, and consistent application of laws and regulations—is vital for voluntary compliance, sound financial planning, and economic stability; deliberate management of AI is necessary to improve certainty.

### Historical and technical context (timeline highlights)
- 1970s—90s: Core process automation; Expert systems; Machine visioning.
- 2000s: Electronic services; Advanced analytics.
- 2010s: Big data, data warehouses; Machine learning.
- 2020—24: Generative AI; Digital transformation; Natural systems.
- 2024+: Generative AI; Digital transformation; Natural systems.
- Figure labeled "Digital Operations and AI: 50 Years of Investment."

### Key findings and implications for administrations
- AI is interdisciplinary, drawing from data science, statistics, mathematics, neuroscience, psychology, linguistics, and economics.
- AI can be applied to any sufficiently understood business problem supported by adequate data, drawing from vast repositories to facilitate analysis and mimic human-like interactions.
- GenAI can produce content (text, data, images, videos) comparable with human outputs; accessibility through desktop and mobile apps accelerates integration into daily operations.
- Administrations must understand AI’s capabilities and limits because AI will continue to evolve rapidly.

### Box 1 — AI myths versus reality (selected points and recent developments)
- Myth vs Reality highlights:
  - AI is already powerful and rapidly improving; GenAI adoption creates an “AI everywhere / AI always on” scenario.
  - AI offers significant public-sector benefits: enhanced efficiency, accuracy, fraud detection, improved compliance rates, and more personalized services.
  - AI will not fully replicate human intelligence; lacks self-consciousness and broad contextual understanding.
  - AI is not inherently unbiased; algorithms can perpetuate and amplify biases present in training data.
- Key development: On November 30, 2022, ChatGPT was released; two months after launch, ChatGPT reached an estimated 100 million active users.
- OpenAI performance benchmarks for ChatGPT 4.0:
  - 90th percentile on the Uniform Bar Exam
  - 88th percentile on the Law School Admission Test
  - 80th, 99th, and 54th percentiles on the Graduate Record Exam Quantitative, Verbal, and Writing tests
- Rapid improvement example: performance on the Uniform Bar Exam increased from the 10th percentile (ChatGPT 3.5) to the 90th percentile (ChatGPT 4.0).

### Why this AI technology is emerging now
- Increased Data Availability: digitalization has resulted in exponential increases in data for ML.
- Advancements in ML Algorithms: Transformer architectures led to Large Language Models (LLMs); ChatGPT relies on GPT, a “Generative Pre-trained Transformer.”
- Computational Power: growth in computational power at lower costs; building and training LLMs can cost tens to hundreds of millions of dollars, though fine-tuning can be relatively low cost.

### Recognizing AI’s systemic adoption: illustrative scenarios
- Policymakers: MP uses a GenAI chatbot to review legislation; later doubts emerge about AI “hallucinations.”
- Taxpayers: taxpayer uses GenAI mobile app for tax advice and declines paid professional advice.
- Importers and Exporters: customs virtual assistant trained on classification codes provides unreliable information at times.
- Auditors: audit team relies on system-generated risk profiles and proposes a large adjustment based on system output.
- Fraudsters: criminal network uses GenAI to plan value-added tax fraud and craft fictitious registrations and invoices.

### Legal and ethical challenges — Five principled concerns
- Concern #1: Biases and Unjustified Use of Data — bias sources: Training Data, Algorithmic Design, Outcomes, Feedback.
- Concern #2: Simplexity — trade-off between simplicity and complexity; avoid “black box” systems while preventing bias from oversimplification.
- Concern #3: Transparency — openness, intelligibility, and explainability; promote publicizing AI deployment, user notifications, and redress.
- Concern #4: Privacy and Data Governance — taxpayers’ right to control personal information; actions include scoping data collection, securing data, notifying breaches without undue delay, and limiting storage duration.
- Concern #5: Accountability — governance, responsibility, traceability, liability, institutional safeguards, redress mechanisms, and documentation.

### Recent regulatory initiatives (selected entries)
- European Union — EU AI Act draft proposal (January 21, 2024) — Hard law regulation; Ex ante risk assessment with specific rules for selected use cases deemed “high risk”
- Council of Europe — Draft Framework Convention on AI (December 18, 2023) — International Treaty; Common ethical principles and guidelines on enforcement mechanisms
- OECD — AI Principles (November 8, 2023) — Intergovernmental guidelines; Common ethical principles on the use of AI systems
- United States — Executive Order on the Safe, Secure and Trustworthy Artificial Intelligence (October 30, 2023) — National guidelines; Decentralized guidelines based on common ethical principles
- China — Interim measures on GenAI (August 15, 2023) — Hard law regulation; Ex post liability based on ethical principles and rules
- Additional national and regional initiatives listed for Colombia, South Korea, Kyrgyzstan, Panama, Argentina, Mexico, Chile, The Philippines, Israel, and others.

### High-level policy recommendations
- Implement a formal AI policy to guide deliberate management of AI use across the administration.
- Sensitize staff to AI and individual duty of care.
- Build from strong fundamentals with an AI strategy.
- Establish a formal inventory of AI use cases.
- Subject use cases to legal and ethical review.
- Where appropriate, keep a “Human-in-the-Loop” (HITL) or “Human-on-the-Loop” (HOTL).
- Risk-assess new use cases prior to introduction.
- Regularly publish the use case inventory (filtered as appropriate).
- Prominently disclose the use of AI in regular operations.
- Evaluate use cases for performance and intent.

### Operational guidance and tools (life-cycle and annexes)
- Life-cycle approach with “control gates” integrating project management, SDLC, and IT Ops:
  - 1. Present the AI use case to a CAB for initial review and endorsement.
  - 2. After CAB endorsement, formulate a project with change controls appropriate to complexity.
  - 3. Assess legal and ethical risks prior to PIR approval.
  - 4. As part of a URR, confirm users are ready for new AI capabilities.
  - 5. After implementation, evaluate the AI use case for intent as part of PIRs and a PRB upon deactivation.
- Annex 1: Example AI policy text (pages referenced: example on page 21).
- Annex 2: References and diagrams for developing an AI strategy (page 25).
- Annex 3: AI Risk Assessment Methodology (page 27) — methodology evaluates an AI use case against 40 specific questions organized into AI input, throughput, and output; includes binary and open-ended questions and does not output an automatic overall risk label.

### Practical policy actions and specific requirements (from Annex 1 example policy)
- Policy should open by explaining what AI is and why regulation is needed; prioritize statements on leadership intent, ethical principles, compliance, transparency, external services, governance, risk management, training, monitoring, and evaluation.
- Establish an Artificial Intelligence Working Group (AIWG) chaired by the Director of Modernization; AIWG to convene quarterly.
- All uses of AI (except defined Embedded AI Tools) must be approved by AIWG and registered in the use case inventory with a formal risk rating.
- Mandatory controls include:
  - Preapproval before introduction.
  - Adherence to approved use case controls and business intent.
  - Adherence to external requirements and information security policy.
  - Requirement for domain expertise and staff sensitization and training.
  - Disclosure of AI-generated content: label content, for example with “elements of this content were generated or influenced by the use of AI systems or tools.”
  - Notification of AI interactions to users unless waived by AIWG.
- Prohibited uses without explicit AIWG authorization:
  - Supplying internal data to external AI services operated outside the administration.
  - Using embedded AI tools to circumvent formal controls.
- Special requirements for high-risk AI:
  - Requirement for high-quality data (standard defined by AIWG).
  - Requirement for consistent performance and thorough testing within well-defined parameters.
  - Requirement for regular evaluation at least on a < time period > basis for performance and alignment with approved business objectives.
- Reporting noncompliance: all staff, contractors, service providers, suppliers, and other covered third parties must report suspected violations through management channels or directly to AIWG.

### Questions worth asking (for senior officials)
- Who should have primary accountability for AI at an institutional level?
  - Possible roles: director for modernization or innovation; Chief Information Officer; Chief Data Officer; cross-functional working group.
- Where do risks and liability exist in AI service supply chains?
  - Consider layered external providers and upstream risks (for example, biased data supplied upstream).
- How much or what level of AI explainability should be targeted?
  - Amount of explainability should be proportionate to tolerance for risk and resources for mitigation.
- To what extent should an AI strategy account for external AI adoption and use?
  - Plan for AI use by external stakeholders and promote disclosure in official interactions.
- How advanced is AI likely to become in the next 5, 10, or 15 years?
  - Conclusion: “In short, very advanced—and to the extent that individuals and agencies not using AI as part of regular work will be at a significant disadvantage.”
- How should an approach to AI vary depending on institutional maturity?
  - GenAI warrants special attention; a policy for AI use is important regardless of overall digital maturity.

*Source: tnmea2024006 - 1. Implement a Formal Policy on AI (Technical Note).*

### 1. Implement a Formal Policy on AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

### 1. Implement a Formal Policy on AI

### Overview and purpose
- The technical note provides an overview of current thinking on artificial intelligence (AI) in tax and customs administration, written primarily for senior officials to inform decision making.
- The note addresses historic and ongoing AI developments, legal and ethical aspects, principled concerns, and recent regulatory initiatives.
- The note presents the concept of an AI “use case,” guidance on promoting AI’s responsible use, and a process for introducing AI into an operational setting.
- Annexes 1−3 provide tools for AI policy, strategy, and risk assessment.

### Rationale: why a formal AI policy is needed
- Generative AI (GenAI) has significantly raised AI’s profile and is likely to increase AI use substantially in everyday tasks, either supporting core operations or enhancing common desktop tools used daily by staff.
- AI’s use raises issues of certainty in two principal ways:
  - Economic impact on tax bases and trade flows, and the follow-on effects to policy.
  - Practical implications, utility, and management from an administrative perspective.
- “Certainty of action”—strengthened by clear, predictable, fair, and consistent application of laws and regulations—is vital for voluntary compliance, sound financial planning, and economic stability; deliberate management of AI is necessary to improve certainty in administrations’ use of AI.

### Historical and technical context
- AI in tax and customs administration spans decades of digital investments, progressing from core process automation in the 1970s and 1980s to:
  - Expert systems and machine visioning in earlier periods.
  - Electronic services, advanced analytics, data warehouses, big data platforms, and machine learning (ML) techniques in subsequent decades.
- Timeline highlights as presented in Figure 1:
  - 1970s—90s: Core process automation; Expert systems; Machine visioning.
  - 2000s: Electronic services; Advanced analytics.
  - 2010s: Big data, data warehouses; Machine learning.
  - 2020—24: Generative AI; Digital transformation; Natural systems.
  - 2024+: Generative AI; Digital transformation; Natural systems.
- Note: Figure 1 is labeled "Digital Operations and AI: 50 Years of Investment."

### Key findings and implications for administrations
- AI is interdisciplinary, drawing from data science, statistics, mathematics, neuroscience, psychology, linguistics, and economics.
- AI can be applied to any sufficiently understood business problem supported by adequate data, drawing from vast repositories to facilitate analysis and mimic human-like interactions.
- GenAI can produce content (text, data, images, videos) comparable with human outputs, and its accessibility through desktop and mobile apps accelerates integration into daily operations.
- Because AI will continue to evolve rapidly, administrations must understand its capabilities and limits to make informed decisions.

### Policy recommendations (high level)
- Implement a formal AI policy to guide deliberate management of AI use across the administration.
- Sensitize staff to AI and individual duty of care.
- Build from strong fundamentals with an AI strategy.
- Establish a formal inventory of AI use cases.
- Subject use cases to legal and ethical review.
- Where appropriate, keep a “Human-in-the-Loop” (HITL).
- Risk-assess new use cases prior to introduction.
- Regularly publish the use case inventory.
- Prominently disclose the use of AI in regular operations.
- Evaluate use cases for performance and intent.

### Operational guidance and tools
- The note emphasizes managing change across the life cycle of an AI use case and provides a section "VI. How to Introduce an AI Use Case" (page 18) addressing that process.
- Annex 1 contains an example of an AI policy (page 21).
- Annex 2 provides references for developing AI strategy (page 25).
- Annex 3 presents an AI Risk Assessment Methodology (page 27).

### Framing questions and ongoing work
- The note concludes by highlighting a selection of open questions around AI being actively explored by experts (see "VII. Questions Worth Asking", page 19).
- Understanding what AI can and cannot do is framed as vital; Box 1 in the note explores current limits and common myths.

*Source: tnmea2024006 - 1. Implement a Formal Policy on AI (Technical Note).*

### BOX 1. AI Myths Versus Reality

### BOX 1. AI Myths Versus Reality

### Myths and Realities
- Myth: AI is mostly hype that can be ignored.  
  Reality: AI is already powerful and rapidly improving. For example, when questions from professional and academic examinations are supplied to current GenAI tools, the answers that the tools provide score very highly, in many subjects outperforming human test takers. These and similar capabilities are being introduced into commercial desktop applications and mobile apps which, in tandem with other channels for accessing AI, are creating an “AI everywhere / AI always on” scenario likely to heavily influence the work of public servants, including tax and customs officers.
- Myth: AI’s benefits are overstated and not applicable to the public sector.  
  Reality: AI offers significant benefits for the public sector, including enhanced efficiency, accuracy, and the ability to uncover insights from large data sets that would be impractical for humans to analyze manually. For tax and customs functions, this could mean more effective fraud detection, improved compliance rates, and more personalized services for both taxpayers and foreign trade operators. As with the adoption of smartphones, GenAI is likely to become ubiquitous in office systems as a tool to enhance productivity.
- Myth: AI will lead to large job losses and very soon.  
  Reality: In tax and customs administrations, AI can streamline operations, reduce errors, and allow staff to focus on higher-value work, such as investigative and strategic decision making. The increase in productivity from the use of generative AI may also lead to an ability to redeploy staff from administrative, back-office tasks to more human contact roles. The scale of this change may be like that experienced by tax and customs administrations in the past when moving from administrative assessment to self-assessment paradigms, and may require a similar level of planning to navigate the choices available, account for costs, and realize potential benefits.
- Myth: AI can fully replicate human intelligence and decision-making processes.  
  Reality: AI mimics certain aspects of human intelligence, such as pattern recognition and problem-solving, but currently does not possess self-consciousness or emotional intelligence. AI systems operate within a set of parameters defined by their programming and the amount and quality of the data used for their training, which is often a limitation. This means that AI currently lacks the human capacity for understanding context in a broad, intuitive way, a distinction that is crucial in scenarios where nuanced decision making or ethical considerations are involved.
- Myth: AI can surpass human intelligence in all aspects.  
  Reality: AI excels in processing large data sets and performing some specific tasks faster and more accurately than humans. However, it lacks the moral reasoning and innate ability for creative thinking that humans have. This distinction is crucial as it highlights AI’s role as a tool ideal for augmenting human capabilities. For example, AI can already diagnose diseases from medical imaging with high accuracy but relies on human medical professionals for the final judgment and patient care.
- Myth: AI tools and technology are self-aware.  
  Reality: AI systems function within the scope of their programming and the data they have been trained on. They do not possess consciousness or self-awareness, nor do they have an independent goal orientation. AI’s decisions are based on algorithms and data patterns, lacking genuine understanding or intent. This is evident in natural language processing applications, where AI can generate human-like text but cannot currently comprehend the meaning of text in the same way that humans do.
- Myth: AI is inherently unbiased and objective.  
  Reality: AI algorithms can perpetuate and even amplify biases present in their training data. This is particularly relevant for tax and customs administrations, where biased AI and machine learning tools could lead to unfair targeting or overlooking certain behaviors (including new or previously unknown forms of fraud not reflected in historic data sets). It underscores the importance of carefully curating training data and continuously monitoring AI systems for biased outcomes, ensuring that automated processes support formal procedures that are fair and equitable.

### Recent AI Developments and Importance
- Key development: On November 30, 2022, ChatGPT was released, rapidly becoming the technology with the fastest uptake and use in history. Two months after launch, ChatGPT reached an estimated 100 million active users.
- Nature of ChatGPT: a chatbot capable of interacting in natural language text, analyzing and answering complex questions, and generating tailored content (for example, email messages, summaries of documents, or jokes and poetry).
- Performance benchmarks reported by OpenAI for ChatGPT 4.0:
  - 90th percentile on the Uniform Bar Exam
  - 88th percentile on the Law School Admission Test
  - 80th, 99th, and 54th percentiles on the Graduate Record Exam Quantitative, Verbal, and Writing tests
- Rapid improvement example: performance on the Uniform Bar Exam increased from the 10th percentile (ChatGPT 3.5) to the 90th percentile (ChatGPT 4.0).
- Implication: AI tools will increasingly draw from economics, finance, accounting, taxation, trade, and tax and customs administration knowledge, placing AI at the center of daily office work for tasks such as responses to questions, advice, guidance, and drafts of emails and memoranda.

### Why This AI Technology Is Emerging Now
- Increased Data Availability:
  - Ongoing digitalization has resulted in an exponential increase in data available for ML purposes.
  - AI can process and analyze “big data” at scale, identifying trends and patterns not visible to humans.
- Advancements in ML Algorithms:
  - Recent improvements, particularly with “Transformer” architectures, led to Large Language Models (LLMs).
  - ChatGPT relies on GPT, a “Generative Pre-trained Transformer” architecture.
  - These technologies improved language modeling, data analysis, pattern recognition, and predictive modeling, enhancing capabilities for identifying tax evasion, smuggling, and fraud more efficiently and effectively than legacy approaches.
- Computational Power:
  - Growth in computational power at lower costs has made it feasible to run complex AI models on enormous data sets.
  - Although expensive to initially build and train (costing tens to hundreds of millions of dollars), LLMs and other “foundation models” can sometimes be fine-tuned at a relatively low cost to improve abilities in particular knowledge domains.
  - Potential examples include chatbots for taxpayer services that can respond in multiple languages.

### Recognizing AI’s Systemic Adoption: Illustrative Scenarios
- Policymakers:
  - A Member of Parliament uses a GenAI chatbot to review a legislative proposal. The chatbot’s detailed response is accepted at face value; later, concerns about AI “hallucinations” raise doubts about reliability.
- Taxpayers:
  - At a trade association meeting, a taxpayer dissatisfied with search engine results uses a GenAI mobile app for tax advice. The app provides specific reasons why techniques presented may not be valid; the taxpayer declines paid professional advice and avoids the presentation advice.
- Importers and Exporters:
  - A customs administration introduces a virtual assistant (chatbot) trained on classification codes and prior rulings. Despite extensive testing, the assistant’s reliability is known to be less than 100 percent and sometimes provides poor information.
- Auditors:
  - Audit team relies on system-generated risk profiles and suggested adjustment values (from expert rules and ML). The team proposes a large adjustment based on the system and the taxpayer agrees quickly, leaving auditors questioning whether deeper review was needed.
- Fraudsters:
  - Criminal network uses GenAI to obtain detailed advice on organizing a value-added tax fraud program, generating fictitious registrations and invoices crafted to avoid detection by known flags.

### Legal and Ethical Challenges — Five Principled Concerns
- Concern #1: Biases and Unjustified Use of Data
  - Bias sources: Training Data, Algorithmic Design, Outcomes, Feedback.
  - Mitigation requires proving impartiality through objective examination of evidence and scrutiny of every element influencing decisions.
- Concern #2: Simplexity
  - Trade-off between simplicity and complexity: overly complex AI can become a “black box”; oversimplification can introduce bias via inadequate data.
  - Adoption requires balancing simple, accessible AI and tools that add value while limiting underlying complexities, considering user sophistication, organizational capacity, and data.
- Concern #3: Transparency
  - Transparency components: Openness (What information is shared?), Intelligibility (How is information received?), Explainability (Why was an AI-influenced decision taken?).
  - Administrative actions to promote transparency include publicizing AI deployment, mandating notifications when interacting with AI, and ensuring explainability and redress.
- Concern #4: Privacy and Data Governance
  - Privacy: taxpayers’ right to control what, how, and when personal information is collected and processed.
  - Actions that promote data privacy: assess needs; scope limitations; gather data tailored to objectives; control quality; secure data; inform subjects; notify breaches without undue delay; limit access and storage duration; periodically review and destroy unnecessary data.
- Concern #5: Accountability
  - Requires enforcement of responsibility to mitigate unintended outcomes and guarantee objectivity.
  - An accountability framework should include governance, responsibility, traceability, liability, institutional safeguards, redress mechanisms, and documentation of processes.

### Recent Regulatory Initiatives (selection presented in Table 1)
- Numerous regulatory initiatives have emerged internationally and nationally to address AI concerns; initiatives involve national legislative bodies and intergovernmental organizations and aim to introduce more certainty in AI use.
- Selected entries (jurisdiction / regulation / date / approach):
  - European Union — EU AI Act draft proposal (January 21, 2024) — Hard law regulation; Ex ante risk assessment with specific rules for selected use cases deemed “high risk”
  - Council of Europe — Draft Framework Convention on AI (December 18, 2023) — International Treaty; Common ethical principles and guidelines on enforcement mechanisms
  - Colombia — Proyecto de Ley Estatutaria 059 (November 14, 2023) — Hard law regulation; Ex ante risk assessment with specific rules for selected use cases deemed “high risk”
  - OECD — AI Principles (November 8, 2023) — Intergovernmental guidelines; Common ethical principles on the use of AI systems
  - United States — Executive Order on the Safe, Secure and Trustworthy Artificial Intelligence (October 30, 2023) — National guidelines; Decentralized guidelines based on common ethical principles
  - China — Interim measures on GenAI (August 15, 2023) — Hard law regulation; Ex post liability based on ethical principles and rules
  - South Korea — Artificial Intelligence Responsibility and Regulation, Bill 2123709 (August 8, 2023) — Hard law regulation; Ex ante risk assessment with specific rules for selected use cases deemed “high risk”
  - Kyrgyzstan — Draft Digital Code of the Kyrgyz Republic (August 5, 2023) — Hard law regulation; Ex ante risk assessment with specific rules for selected use cases deemed “high risk”
  - Panama — Anteproyecto de Ley n°014 que regula la Inteligencia Artificial en la Republica (July 6, 2023) — Hard law regulation; Common ethical principles on the use of AI systems
  - Argentina — Marco legal para la regulación del desarrollo y uso de la Inteligencia Artificial (June 1, 2023) — Hard law regulation; Ex ante risk assessment with common ethical principles and guidelines
  - Mexico — Ley para la obótican ética de la inteligencia artificial y la obótica (May 23, 2023) — Hard law regulation; Common ethical principles on the use of AI systems
  - Chile — Boletin 15869-19 (April 24, 2023) — Hard law regulation; Ex ante risk assessment with specific rules for selected use cases deemed “high risk”
  - The Philippines — House Bill No. 7396 (March 1, 2023) — Hard law regulation; Common ethical principles on the use of AI systems
  - Israel — Policy on Artificial Intelligence, Ministry of Innovation, Science and Technology (October 30, 2022) — National guidelines; Decentralized “soft law” guidelines based on common ethical principles
- Note: Legal norms generally conform to a hierarchy where international treaties and hard law regulations take precedence over executive orders and guidelines. AI = artificial intelligence; GenAI = generative AI; OECD = Organisation for Economic Co-operation and Development.

*Source: Authors.*

### 1. Implement a Formal Policy on AI

### 1. Implement a Formal Policy on AI

### Develop and publish a formal AI policy
- Open the policy by explaining what AI is and why regulation is needed.
- Prioritize statements regarding:
  - (1) senior leadership’s expectations and intent
  - (2) ethical principles and values governing the use of AI
  - (3) compliance with external regulations and standards
  - (4) requirements for transparency and explainability
  - (5) use of external AI services
  - (6) governance and oversight mechanisms
  - (7) procedures for managing AI risks
  - (8) training and awareness
  - (9) monitoring and evaluation
- Annex 1 provides an example of a formal AI policy for tailoring and adoption.

### Sensitize staff and emphasize individual duty of care
- Organize a deliberate project to sensitize staff to AI and their responsibilities as public servants, regardless of whether a formal policy is in place.
- Encourage staff, particularly senior leadership, to proactively understand:
  - the benefits, limitations, and potential consequences of AI systems and tools that support regular work
  - the requirement to explain how relevant AI makes decisions
  - potential biases that could exist in the AI
  - implications of relying on AI outputs
  - where human involvement is essential
- Staff should perform duties emphasizing the rights of taxpayers and ethical use of AI in compliance with laws and regulations.

### Build an AI strategy grounded in core fundamentals
- Prepare an AI strategy consistent with broader reform and modernization objectives.
- Acknowledge that aspects of AI are likely transformational, presenting opportunities and risks, while core fundamentals of tax and customs administration will persist.
- Create a dedicated AI strategy in many administrations, but do not let AI work substitute for strengthening weak core operations.
- Annex 2 provides a concept to navigate these issues.

### Establish and maintain a formal inventory of AI use cases
- Compile a comprehensive inventory of AI currently used or planned, referencing the concept in Tables 2 and 3.
- Document all deliberate and other nontrivial uses of AI as use cases and catalog them with attributes.
- Account for future plans and record categories of AI technology relied on (for example, expert rules, ML, natural language processing, and GenAI).

### Subject use cases to legal and ethical review
- Scrutinize each use case to ensure compliance with legal requirements and sound ethical practices.
- Annex 3 provides a methodology to support a risk assessment of each use case, accounting for principled concerns in Section III (biases and unjustified use of data, simplexity, transparency, privacy and data governance, and accountability).
- Specify an overall, relative risk classification for each use case (high, medium, low) using the concept provided in Tables 2 and 3.
- Ensure high-risk use cases are regularly elevated for review and consideration of mitigation strategies as part of enterprise risk-management processes.

### Mandate human involvement where appropriate (HITL and HOTL)
- Require human involvement in scenarios where AI-influenced decisions could cause significant harm.
- Use the concepts of “Human-in-the-Loop” (HITL) and “Human-on-the-Loop” (HOTL):
  - HITL: humans may be directly involved in all final decisions.
  - HOTL: humans play an oversight role, fine-tuning AI and stepping in on higher-level, critical decisions.
- Apply HITL and HOTL in areas where there is a temptation to fully “automate and forget” processes (for example, selectivity in customs).

### Risk-assess new use cases prior to introduction
- Apply the same risk review methodology to all new AI use cases explored for introduction.
- Evaluate full consequences before implementation, including acquisition costs, ongoing operations, and mitigation costs if AI leads to unintended consequences (for example, audit results overturned on appeal).

### Regularly publish a filtered use case inventory
- Publish a filtered copy of the AI use case inventory to advance transparency and promote certainty.
- Exclude from publication:
  - AI use cases that reasonably relate to the protection of national security interests (particularly border security in customs administration)
  - AI use cases with little potential impact on core business operations or the affairs of taxpayers and traders (for example, trivial embedded AI tools such as “spellcheck”)
- Provide a channel for soliciting feedback on the published inventory.

### Prominently disclose AI use in regular operations
- Make clear and obvious to all audiences when AI is being used.
- Notify users that AI is being used (for example, virtual assistants should inform users they are bots).
- Label all outputs that include AI-generated content—emails, drafts or summaries of documents, data, or images—(for example, with a statement such as “elements of this content were generated by the use of AI systems or tools”).

### Evaluate operational performance and intent of use cases
- Regularly reevaluate the actual impact of each operational AI use case.
- Recognize that AI may produce unanticipated and undesirable outcomes and may not deliver the originally intended basic functionality.
- Regular evaluation should focus on the actual decisions made or influenced by AI.

### How to introduce an AI use case (life-cycle approach)
- Use a structured approach based on an IT service’s life cycle with “control gates” to evaluate and approve significant changes through governance structures (committees).
- Integrate change control across:
  - Project management
  - Technical development (Systems Development Lifecycle, SDLC)
  - IT operations (IT Ops)
- Tailor industry-standard mechanisms for AI, suitable primarily for mid- to large-sized administrations.
- Key steps and control gates illustrated in the life-cycle concept:
  - 1. Present the AI use case to a CAB for initial review and endorsement.
  - 2. After CAB endorsement, formulate a project that includes a set of change controls appropriate to the complexity of the AI use case.
  - 3. Assess legal and ethical risks prior to PIR approval.
  - 4. As part of a URR, confirm that users are ready to use the new AI capabilities.
  - 5. After implementation, evaluate the AI use case for intent as part of PIRs and a PRB, upon deactivation.
- Abbreviations used: SDLC; IT Ops; CAB; PIR; SCR; SRR; CDR; TRR; SAR; DRR; URR; PRB.

### Questions worth asking (for senior officials)
- Who should have primary accountability for AI at an institutional level?
  - Possible roles: director for modernization or innovation; Chief Information Officer; Chief Data Officer; cross-functional working group.
- Where do risks and liability exist in AI service supply chains?
  - Consider layered external providers and upstream risks (for example, biased data supplied upstream that leads to an unjustified enforcement decision).
- How much or what level of AI explainability should be targeted?
  - The ideal amount of explainability is likely proportionate to an administration’s tolerance for risk and resources for mitigation.
- To what extent should an AI strategy account for external AI adoption and use?
  - Plan for AI use by external stakeholders and promote disclosure of AI use in official interactions with tax and customs officers.
- How advanced is AI likely to become in the next 5, 10, or 15 years?
  - The note concludes: “In short, very advanced—and to the extent that individuals and agencies not using AI as part of regular work will be at a significant disadvantage.”
- How should an approach to AI vary depending on institutional maturity?
  - GenAI warrants special attention; a policy for AI use is important regardless of overall digital maturity.

### Overview of annexes and tools
- Annex 1: Example of an AI Policy
  - Provides text of an illustrative AI policy document; AI policy should be endorsed by leadership and disseminated with outreach to sensitize staff.
- Annex 2: References for Developing an AI Strategy
  - Two diagrams to explore AI strategy, dependencies, use cases, and sequencing; includes a structured concept and an example road map.
- Annex 3: AI Risk Assessment Methodology
  - A methodology applied in practice by evaluating an AI use case against 40 specific questions.
  - Questions are organized into three areas: AI (1) input, (2) throughput, and (3) output.
  - Two types of questions: yes/no (binary) and open-ended; the most important questions are highlighted for prioritization.
  - The methodology does not output an automatic overall risk classification label; results are for subjective interpretation case-by-case.
  - Note references: 25 (including Cantekin (2023), and Gesley and others (2023)).

*Source: tnmea2024006 - 1. Implement a Formal Policy on AI (PDF).*

### ANNEX 1. Example of an AI Policy

### ANNEX 1. Example of an AI Policy

### I. Introduction
- The < Tax or Customs Administration > has determined to make strategic use of artificial intelligence (AI) to advance its mandate to improve voluntary compliance < and facilitate trade >.
- The < Tax or Customs Administration > is committed to AI’s responsible use, ensuring at all times full compliance with legal requirements and generally accepted ethical norms.
- The decision to use AI is reflected in the < Tax or Customs Administration > strategic plan and digital transformation strategy.
- This policy outlines expectations to be adhered to by all internal staff, contractors, service providers, suppliers, and other third parties that carry out work on behalf of the < Tax or Customs Administration >.

### II. Definitions
- Artificial Intelligence. The simulation of human intelligence using computer systems to make predictions, inform decisions, and generate content, often relying on machine learning algorithms, expert rules, machine visioning, and other related technologies.
- Artificial Intelligence Working Group. The pinnacle body in the < Tax or Customs Administration > tasked with oversight and compliance with this policy, along with the evaluation of current and future AI use cases. Referred to hereinafter as the “AIWG.”
- Artificial Intelligence System. Any dedicated computer system (or service) that makes use of AI technology to generate outputs, make predictions, recommendations, or decisions that influence staff, taxpayers, service providers, or other third parties that carry out work on behalf of < Tax or Customs Administration >, excluding “Embedded AI Tools,” as discussed in the following section.
- Artificial Intelligence Use Case. A specific business scenario within which the use of AI is employed to carry out defined business objectives.
- Embedded AI Tools. Tools that make use of AI technology that have already been approved for use within defined constraints and do not require additional review by the AIWG. An example of an embedded AI tool would include the “spellcheck” function in popular desktop productivity software.
- Personal Information. Information that may identify, relate to, or is capable of being associated with a physical person, whether directly or indirectly.
- Taxpayer Information. Information that may identify, relate to, or is capable of being associated with a physical person or legal entity, whether directly or indirectly, that may be or become a registered taxpayer with the < Tax or Customs Administration >.

### III. Governance and Oversight
- An AIWG will be established and operate as the pinnacle governing body for AI in the < Tax or Customs Administration >.
- The AIWG will convene quarterly to provide direction and oversight of all AI-related topics, including implementation of AI strategy, and review and approval of current and future AI use cases.
- The AIWG will be chaired by and accountable to the < Tax or Customs Administration >’s Director of Modernization, who will approve the AIWG’s terms of reference.

### IV. General Standards for AI Use
- With the limited exception of Embedded AI Tools available in approved software, and within any additionally defined constraints, all uses of AI must be approved by the AIWG to ensure that it is:
  - yLawful. All uses of AI must comply with applicable laws and regulations, which may be present in contractual obligations in the form of limitations or other restrictions.
  - yEthical. All uses of AI must comply to the extent practical with the ethical guidelines defined subsequently in this policy.
  - yJustifiable. All operational uses of AI must be for a valid and justified business purpose. This includes experimentation in a development context, which, if it occurs, must have clearly defined objectives. Open-ended, uncontrolled experimentation is not permitted.

### V. Specific Requirements for AI Use
- yPreapproval Before Introduction. All uses of AI must be evaluated and approved in advance by the AIWG. Exceptions will be reviewed on a case-by-case basis. Approvals authorizing use are recognized once an AI use case is registered in the < Tax or Customs Administration > use case inventory (in either development or production status) along with a formal risk rating.
- yAdherence to Approved AI Use Case Controls. Restrictions, procedures, and limitations must be documented as part of an approved AI use case and complied with at all times. The primary responsibility for ensuring adherence lies with the ranking official of the office designated as the business owner for the use case (hereinafter referred to as the “Owner”).
- yAdherence to Approved Business Intent. An AI use case, once approved, may not deviate in operation significantly from its originally scoped purpose. Any substantive change must be raised for consideration by the AIWG prior to introduction.
- yAdherence to External Requirements. All use cases must ensure ongoing compliance with external legislation, regulations, and policies. These may include, but are not limited to, data privacy and the protection of personal and taxpayer information. The primary responsibility for ensuring compliance rests with the Owner, to be supported by guidance from the AIWG.
- yAdherence to Domain Expertise Requirements. The approval, implementation, and operation of an AI use case must be accompanied by the requisite domain expertise and human resources (in the form of staff having the subject matter knowledge to administer and use the AI, ensuring appropriate use of Human-in-the-Loop or Human-on-the-Loop concepts).
- yAdhere to Staff Sensitization and Training Requirements. All staff must be sensitized to AI and this policy. The staff that will interact with particular AI systems, services, or tools must receive additional appropriate training on that AI in advance of use (confirmed through a formal user readiness review, where applicable).
- yAdherence to Information Security Policy. Prior to approval, all use cases must be formally evaluated for compliance with the < Tax or Customs Administration >’s Information Security Policy, using a procedure specified by the Office of the Chief Information Security Officer. Once approved, the Owner must ensure continued and ongoing compliance.
- yDisclosure of AI-Generated Content. Any content produced by AI (including generative AI tools) must be labeled accordingly (for example, by affixing the statement “elements of this content were generated or influenced by the use of AI systems or tools”).
- yNotification of AI Interactions. Unless granted a waiver by the AIWG, all uses of AI must be prominently disclosed to users. This disclosure may take different forms but should make clear and obvious the presence and use of AI (for example, notifying a user that they are interacting with a chatbot rather than a human).

### VI. Ethical Guidelines
- The < Tax or Customs Administration > recognizes that in some instances particular uses of AI may be legal but unethical. Accordingly, the < Tax or Customs Administration > has determined that its use of AI will be, at all times, consistent with generally accepted ethical norms by deliberately evaluating and risk assessing AI use cases in the AIWG’s approval process for:
  - yPotential Biases. Use cases must be evaluated to ensure the impartiality of decision making and that AI decisions reflect deliberate intent and objective reality. The evaluation must, to the extent practical, consider (1) training data, (2) algorithmic design, (3) outcomes, and (4) feedback.
  - y“Simplexity.” The complexity of AI adopted or developed for a use case must be deliberately limited to the extent practical (limiting “black boxes”) while ensuring that the AI provides the minimum capabilities required to satisfy the intended business objectives.
  - yTransparency. Use cases must be evaluated for (1) openness, (2) intelligibility, and (3) explainability. In instances where a lack of transparency may exist, mitigating measures must be developed, approved, and implemented prior to the use case entering an operational state.
  - yPrivacy and Data Governance. Use cases must be evaluated to confirm that an appropriate set of controls are (or will be) in place to guarantee the privacy of personal and taxpayer information, in a manner consistent with existing data governance policies.
  - yAccountability. Use cases must be evaluated to determine whether adequate mechanisms are in place to ensure accountability of AI outcomes. The evaluation must consider (1) governance arrangements (safeguards), (2) liability (redress mechanisms), (3) responsibility, and (4) traceability (documented processes and institutional frameworks).

### VII. Prohibited Uses of AI
- Certain uses of AI are prohibited in the absence of explicit authorization from the AIWG, including:
  - ySupplying Internal Data or Information to External AI Services. You may not provide any of the < Tax or Customs Administration >’s data or information to AI systems, services, or tools that are operated outside of the < Tax or Customs Administration >. This prohibition includes “chatbots” accessible over public networks.
  - yUsing Embedded AI Tools to Circumvent Controls. You may not use embedded AI tools in a manner that circumvents controls that should normally apply to an AI use case that has been formally reviewed and approved by the AIWG. When in doubt, it is your responsibility to contact the AIWG for guidance.

### VIII. Special Requirements for High-Risk AI
- Any approved AI use case designated as “high-risk” is subject to the following:
  - yRequirement for High-Quality Data. Using a standard to be defined by the AIWG, the data sets used for training high-risk AI systems, services, or tools must be objectively designated as high quality prior to use in any operational environment.
  - yRequirement for Consistent Performance. Because of the risk of harm, a high-risk AI use case, once developed, must be thoroughly tested and function within well-defined parameters agreed by the AIWG before introduction into an operational setting.
  - yRequirement for Regular Evaluation. With the well-defined parameters describing a “regular” state of operation, the AI use case must be evaluated on at least a < time period > basis for performance and alignment with approved business objectives.

### IX. Reporting Noncompliance
- All internal staff, contractors, service providers, suppliers, and other covered third parties aware of any conduct that may violate this policy must report it.
- Reports may be filed by internal staff through regular management channels or, as appropriate, directly to a representative of the AIWG.
- All reports will be promptly reviewed and, if warranted, investigated.

*Source: ANNEX 1. Example of an AI Policy (tnmea2024006).*

### REFERENCES

### REFERENCES

### References
- Aslett, Joshua. 2024. “IMFx: VITARA—Information Technology and Data Management.” edX. https://www.imf.org/en/Capacity-Development/Training/ICDTC/Courses/VITARA-ITD
- Aslett, Joshua, Gustavo González, Stuart Hamilton, and Miguel Pecho. 2024. “Tax Administration: Essential Analytics for Compliance Risk Management.” IMF Technical Notes and Manuals 2024/01, International Monetary Fund, Washington, DC.
- Cantekin, Kayahan, and U.S. Global Legal Research Directorate Law Library Of Congress. 2023. “Regulation of Artificial Intelligence Around the World.” https://www.loc.gov/item/2023555920/
- Gesley, Jenny, Kelly Buchanan, Michael Chalupovitsch, Tariq Ahmad, Clare Feikert-Ahalt, and U.S. Global Legal Research Directorate Law Library Of Congress. 2023 “Safety and Security of Artificial Intelligence Systems.” https://www.loc.gov/item/2023555908/
- González, Ignacio. 2018. “Analytics and Big Data the New Frontier.” CIAT Review No. 44. https://www.ciat.org/Biblioteca/Revista/Revista_44/ingles/2018_TR44_ebook_ing.pdf
- Hadwick, David. 2024. “AI TAX ADMIN EU—Open-Access Repository of the Use of AI by Tax Administrations in the EU.” University of Antwerpen. https://www.uantwerpen.be/en/projects/aitax/
- Hu, Krystal. 2023. “ChatGPT Sets Record for Fastest-Growing User Base.” Reuters. https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/
- MIT Technology Review Insights (MIT). 2023. “The Great Acceleration: CIO Perspectives on Generative AI.” MIT Technology Review, Cambridge, MA.
- OpenAI. 2023. “GPT-4 Technical Report.” https://cdn.openai.com/papers/gpt-4.pdf
- Organisation for Economic Co-operation and Development (OECD). 2020. Tax Administration 3.0: The Digital Transformation of Tax Administration. Paris: OECD Publishing. https://doi.org/10.1787/ca274cc5-en
- Organisation for Economic Co-operation and Development (OECD). 2023. “Inventory of Tax Technology Initiatives.” OECD Forum on Tax Administration. https://www.oecd.org/tax/forum-on-tax-administration/tax-technology-tools-and-digital-solutions/
- Webson, Albert, and Ellie Pavlick. 2022. “Do Prompt-Based Models Really Understand the Meaning of Their Prompts.” Department of Computer Science, Brown University. https://arxiv.org/abs/2109.01247
- World Customs Organization (WCO). 2023. “Research & Policy Note on Generative Artificial Intelligence for Customs.” https://www.wcoomd.org/-/media/wco/public/global/pdf/topics/research/papers/researchpolicynote_genai_en_06122023.pdf?la=en

### Glossary
- Analytics: The practice of using mathematics and statistics to analyze data, identify patterns, develop insights, and support decision making.
- Advanced Analytics: A subset of analytics that uses more complex techniques to predict future outcomes, including machine learning, predictive modeling, data mining, and statistical analysis.
- Algorithm: Logic, methods, or sets of specific steps for solving a problem that may or may not be codified for use in a computer programming language.
- Artificial Intelligence (AI): The simulation of human intelligence, carried out using computer systems and software, relying heavily on statistics and data science.
- Bias: Subjective prejudices, whether of a human or an AI system or tool, that unjustifiably favor or disfavor individuals and groups.
- Black Box: A system or model whose internal workings are not transparent or easily understood, making it difficult to interpret how it processes inputs to produce outputs.
- Big Data Platform: Specialized software and hardware designed to store, process, and analyze large and complex data sets.
- Bot: Short for “robot,” an automated software program designed to perform tasks or interact with users, often through text or voice interfaces, mimicking human behavior.
- Chatbot: A software program designed to simulate conversation through text or voice interactions.
- Control Gate: A decision point or checkpoint that must be passed before a change can proceed to the next phase of the change management process, often through governance structures in the form of committees.
- Data: Facts, figures, measurements, or observations collected and used as a basis for reasoning, analysis, discussion, or calculation.
- Data Governance: The processes and mechanisms by which data life cycle strategies are administered and enforced, accounting for data quality, security, controls, and compliance.
- Data Mining: The use of statistics and data science to analyze large data sets and identify anomalies, correlations, patterns, and trends.
- Data Science: An interdisciplinary field that combines statistics, computer science, and domain knowledge to analyze data and develop intelligence products that support decision making and solving of complex problems.
- Data Set: A set of logically related data often organized in one or more structured tables and used for analysis, processing, and research.
- Data Warehouse: A central repository of data used for analytics and reporting, which sources its data from other databases and is sometimes designed to serve as a single source of truth for an organization.
- Digital Services: The provision of IT resources to facilitate or perform tasks often, but not always, making use of business systems.
- Digital Transformation: The process of digitizing work, optimizing, and then transforming business processes to improve efficiency, enhance customer experiences, and create new business models.
- Expert Rule (ER): Logic in the form of a rule derived from the specialized knowledge and experience of experts in a particular field, used to inform decision making, problem-solving, and the creation of automated systems.
- Expert System: A computer program that uses artificial intelligence to simulate the decision-making ability of a human expert in a specific field, building from a legacy of using expert rules and knowledge sourced “top-down” from subject matter experts.
- Explainability: The degree to which the logic applied by AI to reach a conclusion can be intelligibly understood.
- Exploratory Analysis: A shorthand description for Exploratory Data Analysis, which is the process of using analytics to identify, understand, and summarize the main characteristics of data sets, often using visual methods to reveal patterns, trends, and anomalies.
- Foundation Model: A general-purpose, often large AI model that serves as a base for fine-tuning or adapting to specific tasks and applications.
- Hallucination: A scenario where AI outputs based on patterns in data do not reflect actual reality.
- Generative AI (GenAI): A category of AI technology that is capable of producing content (text, data, images, videos) comparable to outputs created by humans.
- Individual Duty of Care: The obligation of public servants to act responsibly and ethically in the performance of their duties, ensuring that their actions do not cause harm to individuals or the public.
- Large Language Model (LLM): A category of AI foundation model trained on vast amounts of text data, designed to understand and generate human-like language.
- Machine Learning (ML): A major branch of artificial intelligence, involving the use of data and algorithms to enable computers to learn without explicit programming.
- Machine Visioning (MV): The use of computer systems to capture, interpret, and analyze visual information, enabling them to understand and take decisions based on images and videos.
- Natural Language Processing (NLP): A category of AI focused on enabling computers to understand, interpret, and generate human language.
- Simplexity: The trade-off between simple and complex aspects of systems where, particularly with AI, value and risk need to be carefully considered.
- Social Network Analysis (SNA): The analysis of social relationships and structures through examination of networks formed by individuals or organizations, often relying on a category of specialized technology and tools.
- Statistics: A body of mathematical science that seeks to describe and interpret data using methods that can provide measures of probability, uncertainty, and variability.
- Training Data: Data used to train algorithms by enabling them to identify and learn patterns, which help them make informed predictions or decisions.
- Use Case: A specific business scenario within which an IT system, service, or other tool is employed to carry out defined business objectives.
- Use Case Inventory: A formal catalog of AI use cases, along with both summary and use case attributes, allowing for active and deliberate management.
- Virtual Assistant: More robust than a Chatbot, a digital service, software application, or system designed to perform tasks, provide information, and assist users through text or voice interactions.

### Publication identifiers
- 32 Technical Notes and Manuals
- Understanding Artificial Intelligence in Tax and Customs Administration 33
- 2904357984009
- ISBN 9798400290435
- Understanding Artificial Intelligence in Tax and Customs Administration
- TNM/2024/06

*Source: tnmea2024006 - REFERENCES*

---


_Source: https://www.imf.org/-/media/files/publications/tnm/2024/english/tnmea2024006.pdf_