## pdeoraifea

## Source details

**Canonical URL:** [pdeoraifea](https://www.imf.org/-/media/files/publications/dp/2021/english/pdeoraifea.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/dp/2021/english/pdeoraifea.pdf.md)
- [Structured JSON version](/-/media/files/publications/dp/2021/english/pdeoraifea.pdf.json)

---

### Executive Summary — Overview and purpose
- Discusses the impact of the rapid adoption of artificial intelligence (AI) and machine learning (ML) in the financial sector, emphasizing benefits for financial deepening and efficiency and concerns about widening the digital divide between advanced and developing economies.
- Distills and categorizes unique risks to financial integrity and stability, policy challenges, and potential regulatory approaches.
- Notes the evolving nature of AI/ML and that the full extent of strengths and weaknesses is not yet fully understood, implying a need to strengthen prudential oversight.

### Executive Summary — Key benefits, drivers, and measured adoption signals
- AI/ML bring new opportunities and benefits across sectors, including finance, enabled by recent advances in computing and data storage power, big data, and the digital economy.
- The COVID-19 crisis has accelerated AI/ML adoption due to increased use of digital channels.
- Competitive pressures drive rapid adoption in finance by facilitating gains in efficiency and cost savings; reshaping client interfaces; enhancing forecasting accuracy; and improving risk management and compliance.
- AI/ML systems offer potential to strengthen prudential oversight and equip central banks with new tools for monetary and macroprudential mandates.
- Measured adoption signals:
  - "77 percent" of respondents in a recent survey (WEF 2020) anticipate that AI will be of high or very high overall importance to their businesses within two years.
  - McKinsey (2020a) estimates the potential value of AI in the banking sector to reach "$1 trillion".
  - The aggregate potential cost savings for banks from AI/ML systems is estimated at "$447 billion by 2023" (Digalaki 2021).
- Bridging the digital divide requires a digital-friendly policy framework anchored around four broad policy pillars: investing in infrastructure; investing in policies for a supportive business environment; investing in skills; and investing in risk management frameworks.

### Key use cases and sector applications
- Forecasting: ML algorithms used for forecasting economic, financial, and risk variables; often outperform traditional statistical or econometric models.
- Natural language processing (NLP): Used in chat bots, contract reviewing, and report generation.
- Image recognition: Used for facial and signature recognition to assist AML/CFT customer due diligence and strengthen systems security.
- Anomaly detection: Classification algorithms applied to insider trading, credit card and insurance fraud detection, and AML/CFT.
- Investment management (Box 2): increased market liquidity provision via high-frequency algorithmic trading; expanded, lower-cost wealth advisory services; more customized portfolios; new return profiles.
- Credit underwriting (Box 3):
  - ML reduces banks’ losses on delinquent customers by up to "25 percent" (Khandani, Adlar, and Lo 2010).
  - Automated financial underwriting systems benefit underserved applicants and facilitate low-cost automated evaluation of small borrowers.
  - AI/ML-assisted underwriting harnesses social, business, location, and internet data in addition to traditional data.
  - Caution: robust validation and monitoring processes are necessary when using AI/ML in credit underwriting.
- Risk and compliance management (Regtech):
  - Identity verification, AML/CFT, fraud detection, risk management, stress testing, microprudential and macroprudential reporting, compliance with COVID-19 relief requirements.
  - AI/ML is the top technology under consideration among regtech firms (Schizas and others 2019).
- Suptech (Prudential Supervision):
  - AI/ML used in data analysis, processing, validation, and plausibility; applications concentrated in misconduct analysis, reporting, and data management.
  - Benefits include real-time anomaly flagging and predictive analyses to improve supervision.
  - Challenges: data standardization, quality, completeness; resource and skills gaps; privacy, cybersecurity, explainability, and embedded bias risks.
- Central banking:
  - AI/ML could improve nowcasting, market sentiment assessment, uncertainty monitoring, internal processes, and systemic risk monitoring.
  - Adoption has been slow due to cultural, political, legal factors, and lack of adequate capacity.

### Main risks and channels to financial stability
- Embedded bias:
  - AI/ML systems can systematically and unfairly discriminate due to incomplete/unrepresentative training data, data reflecting prevailing prejudices, and human bias in design and training.
  - Mitigation requires bias detection and mitigation plans, disclosure of data sources, monitoring tools, and governance/ethical frameworks.
- Explainability and complexity ("black box"):
  - ML models often lack interpretability because of model complexity, unknown input signals, and ensembles of models.
  - Trade-off: greater model flexibility/accuracy typically reduces explainability.
  - Remedies include explainability by design, model inspection, individual prediction explanations, global surrogate models, and ML-based explanation techniques (Shapley values; Lime).
  - Regulatory guidance needed to set explainability expectations by model impact.
- Cybersecurity:
  - Novel AI/ML-specific threats include data poisoning attacks, input attacks (adversarial perturbations), and model extraction/model inversion (including membership inference).
  - Consequences: undermined integrity and trust, corrupted risk assessments, and exfiltration of sensitive training data.
  - Mitigants: expand cybersecurity perimeter to cover AI/ML threats; require AI/ML-specific protections, detection and reporting systems, protection of training data feeds, and model/data privacy strategies.
- Data privacy:
  - AI/ML can unmask anonymized data via inference and can remember or leak sensitive information.
  - Need stronger legal and regulatory frameworks and technical tools to maintain anonymity and privacy; update frameworks to require adherence to enhanced privacy standards and AML/CFT requirements.
- Robustness:
  - Performance deterioration during structural shifts (example: ML algorithms trained on pre–COVID-19 data experienced performance deterioration; Harker 2020; BoE 2020).
  - Governance needs analogous to software development best practices: quality control, separation of duties, continuous monitoring across development, testing, and deployment; checks for bias, data poisoning, security risks, and performance.
- Systemic risk amplification channels:
  - AI/ML service provider concentration could create single points of failure and systemic importance.
  - Homogeneity in risk assessments and credit decisions due to concentrated third-party providers and concentrated data could produce herding and out-of-sample risk.
  - Widespread automation can increase procyclicality of financial conditions by accelerating credit underwriting and risk management cycles.
  - In tail-risk events, inaccurate ML assessments could amplify shocks and complicate policy responses.
  - Regulatory gaps may emerge if technological advances outpace existing regulations or if providers fall outside regulatory perimeters.

### Policy recommendations and preparatory actions
- Broadly welcome AI/ML adoption while preparing to capture benefits and mitigate risks to financial system integrity and safety.
- Preparatory actions for authorities and institutions:
  - Strengthen institutional capacity and monitoring frameworks of oversight authorities.
  - Recruit relevant expertise and expand personnel pools to include AI/ML specialists and data scientists.
  - Engage stakeholders to identify risks and remedial regulatory actions; improve external communication.
  - Update relevant legal and regulatory frameworks, including privacy and AML/CFT requirements.
  - Develop and deploy governance and ethical frameworks (examples noted: European Union “Ethics Guidelines for Trustworthy AI” April 2019; OECD principles May 2019; Group of Twenty declaration June 2019).
  - Expand consumer education.
  - Implement AI/ML-specific cybersecurity mitigations within broader frameworks.
- Regulatory design considerations:
  - Consider different levels of explainability depending on model impact.
  - Establish minimum standards and guidelines for AI/ML governance, risk management, internal controls, model and data controls.
  - Coordinate nationally via AI strategies and internationally through cooperation and knowledge sharing to ensure less-developed economies can access benefits.
  - Collaboration among financial institutions, central banks, supervisors, and other stakeholders to avoid duplication and counter potential risks.

### International and development considerations
- International cooperation and knowledge sharing are important to coordinate safe deployment of AI/ML systems and share experiences and knowledge on techniques, use cases, and regulatory approaches.
- Current deployment and benefits concentrated largely in advanced economies and a few emerging markets; developing economies risk falling behind due to lack of investment, access to research, and human capital.
- Multilateral organizations and expanded membership in intergovernmental AI working groups could help transfer knowledge, raise investments, build capacity, and facilitate peer learning.

### Conclusion and monitoring needs
- AI/ML adoption in the financial sector is transformative and offers substantial benefits but introduces unique risks to explainability, bias, privacy, cybersecurity, robustness, and financial stability.
- Because the technology is evolving and its full strengths and weaknesses are not yet understood, countries should strengthen monitoring and prudential oversight and pursue national AI strategies that involve public and private bodies.
- Regional and international cooperation, capacity building, and skills development are essential to spread benefits and contain risks.

*Source: pdeoraifea — Executive Summary; Introduction; Boxes 2–6; Chapter 3: Risks and Policy Considerations; Conclusion.*

### Executive Summary

### Executive Summary

### Overview and purpose
- Discusses the impact of the rapid adoption of artificial intelligence (AI) and machine learning (ML) in the financial sector, emphasizing benefits for financial deepening and efficiency and concerns about widening the digital divide between advanced and developing economies.
- Distills and categorizes unique risks to financial integrity and stability, policy challenges, and potential regulatory approaches.
- Notes the evolving nature of AI/ML and that the full extent of strengths and weaknesses is not yet fully understood, implying a need to strengthen prudential oversight.

### Key benefits and drivers of adoption
- AI/ML bring new opportunities and benefits across sectors, including finance, enabled by recent advances in computing and data storage power, big data, and the digital economy.
- The COVID-19 crisis has accelerated AI/ML adoption due to increased use of digital channels.
- Competitive pressures drive rapid adoption in finance by:
  - Facilitating gains in efficiency and cost savings.
  - Reshaping client interfaces.
  - Enhancing forecasting accuracy.
  - Improving risk management and compliance.
- AI/ML systems offer potential to strengthen prudential oversight and equip central banks with new tools for monetary and macroprudential mandates.

### Measured adoption signals (exact figures from source)
- A recent survey of financial institutions (WEF 2020) shows that 77 percent of all respondents anticipate that AI will be of high or very high overall importance to their businesses within two years.
- McKinsey (2020a) estimates the potential value of AI in the banking sector to reach $1 trillion.
- Bridging the digital divide requires a digital-friendly policy framework anchored around four broad policy pillars: investing in infrastructure; investing in policies for a supportive business environment; investing in skills; and investing in risk management frameworks.

### Key use cases highlighted
- Forecasting: ML algorithms used for forecasting economic, financial, and risk variables; often outperform traditional statistical or econometric models.
- Natural language processing (NLP): Used in chat bots, contract reviewing, and report generation.
- Image recognition: Used for facial and signature recognition to assist AML/CFT customer due diligence and strengthen systems security.
- Anomaly detection: Classification algorithms applied to insider trading, credit card and insurance fraud detection, and AML/CFT.

### Main risks and concerns
- Embedded bias in AI/ML systems.
- Opaqueness of AI/ML outcomes ("black box" explainability challenges).
- Robustness issues, particularly with respect to cyber threats and privacy.
- New sources and transmission channels of systemic risk, including:
  - Greater homogeneity in risk assessments and credit decisions.
  - Rising interconnectedness that could quickly amplify shocks.
- Use of nontraditional data (for example, social media data, browsing history, location data, unstructured email text via NLP) raises legal, regulatory, ethical, privacy, and data-quality concerns (cleanliness, accuracy, relevancy, potential biases).

### Policy recommendations and preparatory actions
- Broadly welcome AI/ML adoption while preparing to capture benefits and mitigate risks to financial system integrity and safety.
- Preparations should include:
  - Strengthening the capacity and monitoring frameworks of oversight authorities.
  - Engaging stakeholders to identify possible risks and remedial regulatory actions.
  - Updating relevant legal and regulatory frameworks.
  - Expanding consumer education.
- Actions should be taken within the context of national AI strategies and involve all relevant public and private bodies.

### International and development considerations
- Cooperation and knowledge sharing at the regional and international level are increasingly important to coordinate safe deployment of AI/ML systems and share experiences and knowledge.
- Cooperation is critical to ensure less-developed economies share the benefits; current deployment and benefits have been concentrated largely in advanced economies and a few emerging markets.
- Developing economies risk falling behind due to lack of necessary investment, access to research, and human capital; multilateral organizations and expanded membership in intergovernmental AI working groups could help transfer knowledge, raise investments, build capacity, and facilitate peer learning.

### Conclusion
- AI/ML adoption in the financial sector is transformative and offers substantial benefits but also introduces unique risks to explainability, bias, privacy, cybersecurity, robustness, and financial stability.
- Addressing these risks requires strengthened prudential oversight, updated legal and regulatory frameworks, stakeholder engagement, consumer education, national AI strategies, and international cooperation.

*Source: Executive Summary, pdeoraifea - Executive Summary*

### introduction of ATMs, electronic card payments, and online banking). However, confidentiality and the proprietary

### pdeoraifea - introduction of ATMs, electronic card payments, and online banking). However, confidentiality and the proprietary

### Adoption and benefits of AI/ML in banking
- AI/ML penetration in the banking industry has accelerated in recent years, driven by rising competition from financial technology (fintech) companies (including fintech lenders) and AI/ML’s capacity to improve:
  - client relations (for example, through chatbots and AI/ML-powered mobile banking),
  - product placement (for example, through behavioral and personalized insights analytics),
  - back-office support,
  - risk management,
  - credit underwriting (Box 3),
  - cost savings.
- The aggregate potential cost savings for banks from AI/ML systems is estimated at $447 billion by 2023 (Digalaki 2021).

### Risks and constraints
- AI/ML adoption is slowed by confidentiality and the proprietary nature of banking data.
- Key risks and challenges:
  - Dependence on availability of large volumes of good-quality, timely data.
  - Data privacy and cybersecurity concerns due to storage and use of large quantities of sensitive data.
  - Difficulties in explaining the rationale of AI/ML-based financial decisions as algorithms may uncover unknown correlations where underlying causality is unknown.
  - Model performance deterioration in the event of major and sudden movements in input data, potentially leading to inaccurate decisions during crises.

### Box 2 — Artificial Intelligence in Investment Management: Sample Use Cases
- Increased market liquidity provision through a wider use of high-frequency algorithmic trading and more efficient market price formation.
- Expanded wealth advisory services by providing personal and targeted investment advice to mass-market customers in a cost-effective manner, including for low-income populations.
- Enhanced efficiency with AI/ML taking on a growing portion of investment management responsibilities.
- More customized investment portfolios based on AI/ML targeted customer experiences.
- Development of new return profiles through the use of AI/ML instead of established strategies.

### Box 3 — Artificial Intelligence in Credit Underwriting
- AI/ML predictive models can help process credit scoring, enhancing lenders’ ability to calculate default and prepayment risks.
  - Research finds that ML reduces banks’ losses on delinquent customers by up to 25 percent (Khandani, Adlar, and Lo 2010).
  - Evidence suggests automated financial underwriting systems benefit underserved applicants, resulting in higher borrower approval rates (Gates, Perry, and Zorn 2002), and facilitate low-cost automated evaluation of small borrowers (Bazarbash 2019).
- AI/ML-assisted underwriting harnesses social, business, location, and internet data in addition to traditional data.
- AI/ML reduces turnaround time and increases the efficiency of lending decisions; can generate credit scores from a client’s digital footprint (social media activity, bills payment history, and search engine activity).
- Potential application in commercial lending for risk quantification of commercial borrowers.
- Caution: financial institutions and supervisors should build robust validation and monitoring processes when using and assessing AI/ML in credit underwriting.

### Risk and Compliance Management (Regtech)
- AI/ML advances are reshaping regulatory compliance by leveraging broad sets of data, often in real time, and automating compliance decisions, improving compliance quality and reducing costs.
- AI/ML is the top technology under consideration among regtech firms (Schizas and others 2019).
- Expanded regtech use cases across banking, securities, insurance, and other financial services include:
  - identity verification,
  - anti-money laundering/combating the financing of terrorism (AML/CFT),
  - fraud detection,
  - risk management,
  - stress testing,
  - microprudential and macroprudential reporting,
  - compliance with COVID-19 relief requirements (Box 4).
- Regulators have generally supported regtech adoption and promoted strategies to boost awareness, innovation, and regulatory engagement.

### Box 4 — Artificial Intelligence in Regulatory Compliance: Sample Use Cases
- AML/CFT compliance: AI/ML-powered technologies, including analysis of unstructured data and consumer behavior, are used to reduce false positives, allowing financial institutions to devote more resources to likely-suspect cases.
- Mapping and updating regulatory obligations: AI/ML-based applications help identify and update relevant regulations, reducing costs and improving compliance.
- Conduct risk management: AI/ML and natural language processing monitor sales calls to ensure accurate disclosure of product features and risks.
- Stress-testing: AI/ML-enabled data analytics improve analysis of complex balance sheets and stress testing models.
- COVID-19 relief: AI/ML and robotic process automation process large volumes of applications and loan documents to determine eligibility for payment modification terms.

### Prudential Supervision (Suptech)
- AI/ML has a role primarily in data collection and data analytics for supervision.
  - Many Financial Stability Board member country authorities use ML and NLP tools in data analysis, processing, validation, and plausibility (FSB 2020).
  - AI can draw deeper insights from any type of data, flag anomalies in real time, and provide predictive analyses to improve supervision quality and agility.
- Current suptech use cases are mainly concentrated in misconduct analysis, reporting, and data management; smaller shares cover virtual assistance, microprudential, macroprudential, and market surveillance (di Castri and others 2019).
- AI in market conduct supervision supports collection and analytics of structured and unstructured data, workflow automation, and conduct risk profiling and early warnings.
- Remote working trends due to COVID-19 are driving authorities to use technology for remote supervision; supervisors may lean more on AI/ML-supported off-site supervision post pandemic.
- Challenges and risks for supervisory authorities:
  - Data standardization, quality, and completeness, especially when leveraging nontraditional sources such as social media (FSB 2020).
  - Resource and skills gaps; need to expand personnel pool to include AI/ML specialists and data scientists.
  - Risks associated with privacy, cybersecurity, outcome explainability, and embedded bias.

### Box 5 — Artificial Intelligence in Supervision: Sample Applications
- Banca d’Italia: Exploring loan default forecasting with ML using data from different sources.
- Banco de España: NLP processes institutions’ environmental, social, and governance disclosures; supervised ML assists with misconduct detection.
- Bank of Russia: System analyzing retail loan portfolios using algorithms for large data arrays.
- Bank of Thailand: AI/ML analysis of board meeting minutes to assess regulator compliance of the board.
- De Nederlandsche Bank: ML uses transactional data to detect networks of related entities to assess exposure to networks of suspicious transactions.
- European Central Bank: Machine reading of “fit and proper” questionnaire; ML assists early identification of distress in “less significant institutions”; NLP and ML search supervisory review decisions to identify emerging trends and clusters of risks.
- Monetary Authority of Singapore: Project where credit risk assessments by supervisors are conducted using algorithms instead of sampling.
- Oesterreichische Nationalbank: ML and deep learning algorithms predict probability that a data set contains errors needing rectification by reporting entity.

### Central Banking
- AI/ML applications could help central banks implement mandates by improving understanding of economic and financial developments, supporting better-tuned monetary and macroprudential policies, improving operations, strengthening systemic risk monitoring, and potentially speeding crisis response.
- Adoption has been slow due to cultural, political, legal factors, and lack of adequate capacity (Danielsson, Macrae, and Uthemann 2020).
- Central bank experiments and research focus on improving near-term forecasting and monitoring market sentiment; applications to internal processes and back-office functions also under development.
- Risks and constraints for central banks:
  - Use of large nontraditional and unstructured data sets could expose central banks to data biases avoided in sampling methods.
  - ML algorithms remain susceptible to sudden structural shifts in data from unforeseen events, potentially undermining crisis monitoring and response.
  - Challenges in acquiring quality and representative data, and managing data privacy and security issues.
  - Concerns amplified if central bank lacks resources and skills to safely operate AI/ML or mitigate associated risks.

### Box 6 — Artificial Intelligence in Central Banking: Sample Applications
- Strengthening nowcasting:
  - Sveriges Riksbank: real-time indicators including ML-processed fruit and vegetable prices scraped daily from the Internet to test improvements in short-term inflation forecasts.
  - Reserve Bank of New Zealand: experimenting with ML on about 550 macroeconomic indicators to improve nowcasts of GDP growth, with results so far outperforming comparable statistical benchmarks.
- Assessing market sentiment:
  - Banca d’Italia: real-time tracking system of consumer inflation expectations using ML and textual analysis of millions of daily Italian Twitter feeds; Twitter-based indicators correlate with standard measures and provide superior forecasting for survey-based monthly inflation expectations.
  - Bank Indonesia: testing ML techniques to identify stakeholder expectations of Bank Indonesia’s policy rate from published news articles for Board of Governor’s meeting support.
- Monitoring uncertainty:
  - Banco de Mexico: sentiment-based risk index using AI/ML to analyze Twitter messages; index shocks correlate positively with increases in financial market risk, stock market volatility, sovereign risk, and foreign exchange rate volatility.
  - Banco Central de Chile: daily-frequency index of economic uncertainty for Chile using AI/ML to analyze Twitter feeds; index shows significant spikes coinciding with episodes of substantial economic uncertainty.
  - De Nederlandsche Bank: exploring AI for detecting liquidity problems at banks in anticipation of potential deposit runs (Triepels, Daniels, and Heijmans 2018).
- Improving internal processes:
  - Banco de España: AI tool for sorting banknotes between fit and unfit for circulation.

*IMF Departmental Papers — Powering the Digital Economy (excerpts).*

### 3. Risks and Policy Considerations

### 3. Risks and Policy Considerations

### A. Embedded Bias
- Concern: AI/ML systems can systematically and unfairly discriminate against certain individuals or groups (embedded bias as defined by Friedman and Nissenbaum (1996)).13
- Primary channels for bias:
  - Incomplete or unrepresentative training data (predictive algorithms favor groups better represented in training data) (Goodman and Flaxman 2016).
  - Data that underpin prevailing prejudices (example: Amazon recruiting tool dismissing female candidates because it was trained on historical hiring decisions favoring men over women).14
  - Human bias introduced during feature selection, model design, and training feedback (psychological, social, emotional, cultural factors can influence researcher choices).15
- Potential benefits: AI/ML can reduce human bias by eliminating irrational subjective interpretations and enabling greater scrutiny of predictive processes (Mayson 2019; Silberg and Manyika 2019; Miller 2018).16
- Regulatory/operational implications:
  - Regulators may view embedded bias as an operational and reputational risk.
  - Financial institutions deploying AI/ML significantly, particularly for credit, financial services, and risk management, should develop and implement bias mitigation and detection plans as part of operational risk management. Plans could include:
    - assurances about algorithm robustness against systemically generating biased decisions,
    - disclosure of data sources,
    - awareness of potential bias-generating factors in the data,
    - monitoring and evaluation tools,
    - approaches to meet standing anti-discrimination rules in AI/ML deployment.
- Policy action: Develop and deploy governance and ethical frameworks for AI/ML (examples cited: European Union “Ethics Guidelines for Trustworthy AI” April 2019; OECD principles May 2019; Group of Twenty declaration June 2019).

### B. Unboxing the “Black Box”: Explainability and Complexity
- Issue: ML models often function as “black boxes,” making detection of appropriateness of decisions difficult and increasing organizational vulnerabilities (Guidotti and others 2019; Silberg and Manyika 2019).
- Reasons for lack of explainability:
  1. Models are complicated and not easily interpreted.
  2. Input signals might not be known.
  3. Ensembles of models rather than a single independent model.
- Trade-offs:
  - Model flexibility (capacity to approximate functions; related to number of parameters) versus explainability: more flexible/accurate models are less explainable than linear models.
  - Stronger explainability might enable outsiders to manipulate algorithms (Molnar 2021).
- Methods to address explainability (see Box 7):
  - Explainability by design: inherently interpretable models (linear models, decision trees) but may constrain accuracy.
  - Model inspection: testing sensitivity to variable changes during development.
  - Individual prediction explanation: local approximations to explain individual predictions (e.g., why credit was or was not granted).
  - Global model explanation: use interpretable surrogate models to explain complex models’ behavior.
  - Models to explain ML models: use ML-based techniques (e.g., Shapley values; Lime) and perturbation comparisons to ascertain decision process.
- Regulatory implications:
  - Consider different levels of explainability depending on model impact or governing regulation.
  - Regulatory guidance needed to create frameworks and strategies for managing explainability risks at different levels.
  - Current inclusion of ML explainability is limited to a few regulatory frameworks (European Union, Hong Kong SAR, Netherlands, Singapore).

### C. Cybersecurity
- AI/ML adoption introduces novel cyber risks beyond traditional threats, focusing on manipulation of data across the AI/ML lifecycle to exploit algorithmic limitations (Comiter 2019).
- Specific AI/ML cyber threats:
  - Data poisoning attacks: influence ML during training by adding special samples, causing incorrect learning or Trojan models; require privileged access but can be undetectable if they avoid regular diagnostics (Liu, Dolan-Gavitt, and Garg 2018).
  - Input attacks19: introduce perturbations to data inputs to mislead AI systems in operation (e.g., imperceptible image alterations provoking misclassification).
  - Model extraction/model inversion attacks: attempt to recover training inputs/data or the model itself; membership inference checks if a specific instance was in the training set; feasible as black-box attacks via read-only access through APIs.
- Consequences:
  - Undermine integrity and trust in financial sector.
  - Corrupted systems could impair capacity to assess, price, and manage risks, possibly leading to unobserved systemic risks.
  - Attackers could acquire sensitive financial and personal training data.
- Recommended mitigants:
  - Expand regulatory cybersecurity perimeter to cover AI/ML-specific threats.
  - Require providers and users to implement AI/ML-specific mitigations within broader cybersecurity frameworks, including detection and reporting systems, protection of training data feeds, and strategies to secure model and data privacy.

### D. Data Privacy
- AI/ML raises unique privacy concerns beyond established big data issues: capacity to unmask anonymized data via inference, AI/ML remembering information about individuals in training data, and outcomes leaking sensitive data directly or by inference.
- Existing developments:
  - Tools to maintain data anonymity and privacy are being developed.
  - Legal data policy frameworks are being implemented globally.
- Gaps and needs:
  - More work required to strengthen AI/ML robustness against data leakage.
  - Legal and regulatory frameworks should be updated to require AI/ML systems and related data sources to adhere to enhanced privacy standards and relevant anti-money laundering/combating the financing of terrorism requirements.

### E. Robustness
- Robust AI/ML algorithms are crucial to build public trust and safeguard financial stability, especially given concentration of AI/ML service providers and limited oversight capacity in many jurisdictions.
- Robustness challenges:
  - Cybersecurity and privacy protections (discussed above).
  - Performance issues: minimizing false signals during structural shifts and governance over development processes.
- Performance sensitivity:
  - AI/ML performs well in relatively stable data environments but can deteriorate during structural shifts (example: misalignment of ML-generated risk assessments during the COVID-19 pandemic; ML algorithms trained on pre–COVID-19 data experienced performance deterioration; Harker 2020; BoE 2020).
  - Example: aggregate credit scores in the United States improved during the acute part of the crisis despite record job loss and rising defaults, reflecting temporary relief measures not captured by algorithms.
- Governance needs:
  - New governance frameworks analogous to software development best practices: quality control, agility, separation of duties, constant monitoring across development, testing, and deployment.
  - Checks for embedded bias, data poisoning, security risks, and performance should comply with these best practices.

### F. Impact on Financial Stability
- Dual potential:
  - Positive: Well-designed and controlled AI/ML can increase efficiencies; improve assessment, management, and pricing of risks; enhance regulatory compliance; and provide new tools for prudential surveillance and enforcement.
  - Negative: New and unique risks from opacity, susceptibility to manipulation, robustness issues, and privacy concerns can undermine trust and create new transmission channels for systemic risk.
- Specific systemic risk channels and concerns:
  - AI/ML service providers could become systemically important participants in financial market infrastructure due to specialization and network effects, increasing single-point-of-failure vulnerability.
  - Concentration of third-party AI/ML algorithm providers could drive homogeneity in risk assessments and credit decisions; coupled with interconnectedness, this could enable systemic risk buildup. Concentration of data and growing use of alternative data could produce uniformity (herding) and out-of-sample risk.
  - Widespread AI/ML use could increase procyclicality of financial conditions by automating and accelerating procyclical credit underwriting and risk management, and potentially obscuring procyclicality due to explainability issues.
  - In tail-risk events, inaccurate ML risk assessments and reactions could quickly amplify and spread shocks and complicate policy responses.
  - Challenges in interpretation, sustainability of analytical power, and prediction raise concerns that economic policies or market strategies based on these models will be difficult to interpret or predict, creating additional asymmetric information with uncertain impact on financial stability.
  - Regulatory gaps if technological advances outpace existing regulations, particularly where providers may fall outside existing regulatory perimeters.21
- Regulatory responses and needs:
  - Jurisdictions vary: some have holistic approaches (examples noted: Monetary Authority of Singapore22; De Nederlandsche Bank23), others rely on existing governance expectations.
  - Regulators generally focus on AI/ML governance frameworks, risk management, internal controls, and stronger model and data controls.
  - Addressing challenges requires broad regulatory and collaborative efforts, clear minimum standards and guidelines, and stronger focus on securing technical skills.
  - Collaboration among financial institutions, central banks, supervisors, and other stakeholders is important to avoid duplication and counter potential risks.
  - Many leading jurisdictions rely on national AI strategies to promote development while preventing regulatory gaps. Annex 3 provides an overview of country approaches to developing AI national strategies.

*IMF Departmental Paper — Chapter 3: Risks and Policy Considerations*

### 4. Conclusion

### 4. Conclusion

### Drivers of AI/ML acceleration in finance
- Rapid increases in computational powers, data storage capacity, and big data.
- Significant progress in modeling and use-case adaptations.
- The COVID-19 pandemic accelerating the shift toward a more contactless environment and increasingly digital financial services, strengthening the appeal of AI/ML systems to providers of financial services.

### Benefits and emerging policy challenges
- Benefits of AI/ML systems:
  - Potential for significant cost savings and efficiency gains for financial institutions.
  - Access to new markets and better risk management.
  - New customer experiences, products, and lower costs.
  - Powerful tools for regulatory compliance and prudential oversight.
- Policy challenges and risks:
  - Ethical questions and new unique risks to the financial system’s integrity and safety, the full extent of which is yet to be assessed.
  - Innovations are evolving and morphing as new technologies come into play, complicating policymaking.

### Preparations and institutional responses recommended for regulators
- Regulators should broadly welcome AI/ML advancements and undertake preparations to capture benefits and mitigate risks, in step with the Bali Fintech Agenda’s call on national authorities to embrace the fintech revolution.
- Specific preparatory actions include:
  - Timely strengthening of institutional capacity.
  - Recruiting relevant expertise.
  - Building up knowledge.
  - Improving external communication with stakeholders.
  - Expanding consumer education.
- Note: Deployment of AI/ML systems in the financial sector has proven most effective when there are national AI strategies in place that involve all relevant public and private bodies.

### Cooperation and knowledge sharing
- Regional and international cooperation and knowledge sharing are increasingly important to:
  - Coordinate actions to support the safe deployment of AI/ML systems.
  - Share experiences and knowledge on techniques, methods, use cases, and regulatory and supervisory approaches.
  - Ensure less-developed economies have access to relevant knowledge.

### Uncertainty, monitoring, and prudential oversight
- The evolving nature of AI/ML technology and applications means users, technology providers and developers, and regulators do not currently understand the full extent of the strengths and weaknesses of the technology.
- There may be many unexpected pitfalls yet to materialize.
- Countries will need to strengthen their monitoring and prudential oversight.

*IMF DEPARTMENTAL PAPERS · Powering the Digital Economy — 4. Conclusion*

### References

### References

### AI, Machine Learning, and Financial Services
- Alonso, C., A. Berg, S. Kothari, C. Papageorgiou, and S. Rehman. 2020. “Will the AI Revolution Cause a Great Divergence?” IMF Working Paper 20/184, International Monetary Fund, Washington, DC.  
- Arner, D., J. Barberis, and R. Buckley. 2017. “FinTech, RegTech, and the Reconceptualization of Financial Regulation.” Northwestern Journal of International Law & Business 37 (3). https://scholarlycommons.law.northwestern.edu/njilb/vol37/iss3/2/.  
- Bank of England (BoE). 2020. “The Impact of COVID on Machine Learning and Data Science in UK Banking.” Quarterly Bulletin 2020 Q4. https://www.bankofengland.co.uk/quarterly-bulletin/2020/2020-q4/the-impact-of-covid-on-machine-learning-and-data-science-in-uk-banking.  
- Bazarbash, M. 2019. “FinTech in Financial Inclusion: Machine Learning Applications in Assessing Credit Risk.” IMF Working Paper 19/109, International Monetary Fund, Washington, DC.  
- Bolhuis, M. A., and B. Rayner. 2020. “Deus ex Machina? A Framework for Macro Forecasting with Machine Learning.” IMF Working Paper 20/45, International Monetary Fund, Washington, DC.  
- Doerr, S., L. Gambacorta, and J. M. Serena. 2021. “Big Data and Machine Learning in Central Banking.” BIS Working Paper 930, Bank for International Settlements, Basel.  
- European Central Bank (ECB). 2019. “Bringing Artificial Intelligence to Banking Supervision.” https://www.bankingsupervision.europa.eu/press/publications/newsletter/2019/html/ssm.nl191113_4.en.html.  
- Gambacorta, L., Y. Huang, H. Qiu, and J. Wang. 2019. “How Do Machine Learning and Nontraditional Data Affect Credit Scoring? New Evidence from a Chinese Fintech Firm.” BIS Working Paper 834, Bank for International Settlements, Basel.  
- Gensler, G., and L. Bailey. 2020. “Deep Learning and Financial Stability.” MIT Working Paper, Massachusetts Institute of Technology, Boston, MA. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3723132.  
- HKMA. 2020. “Reshaping Banking with Artificial Intelligence.” https://www.hkma.gov.hk/media/eng/doc/key-functions/finanical-infrastructure/Whitepaper_on_AI.pdf.  
- McKinsey. 2020a. “AI-Bank of the Future: Can Banks Meet the AI Challenge?” https://www.mckinsey.com/industries/financial-services/our-insights/ai-bank-of-the-future-can-banks-meet-the-ai-challenge#.  
- McKinsey. 2020b. “The State of AI in 2020: Survey.” https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/global-survey-the-state-of-ai-in-2020.  
- World Economic Forum (WEF). 2018. “The New Physics of Financial Services: Understanding How Artificial Intelligence is Transforming the Financial Ecosystem.” http://www3.weforum.org/docs/WEF_New_Physics_of_Financial_Services.pdf.  
- World Economic Forum (WEF). 2020. “Transforming Paradigms: A Global AI in Financial Services Survey.” World Economic Forum and Cambridge Centre for Alternative Finance. http://www3.weforum.org/docs/WEF_AI_in_Financial_Services_Survey.pdf.

### Credit, Credit Scoring, and Consumer Finance Studies
- Fuster, A., P. Goldsmith-Pinkham, T. Ramadorai, and A. Walther. 2020. “Predictably Unequal? The Effects of Machine Learning on Credit Markets.” Mimeo. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3072038.  
- Gates, S. W., V. G. Perry, and P. M. Zorn. 2002. “Automated Underwriting in Mortgage Lending: Good News for the Underserved?”, Housing Policy Debate, 13 (2): 369–91.  
- Khandani, A., K. Adlar, and A. Lo. 2010. “Consumer Credit-Risk Models via Machine Learning Algorithms.” Journal of Banking & Finance 34 (11): 2767–787.  
- Lessmann, S., B. Baesens, H. V. Seow, and L. C. Thomas. 2015. “Benchmarking State-of-the-Art Classification Algorithms for Credit Scoring: An Update of Research.” European Journal of Operational Research 247 (1): 124–36.  
- Petropoulos, A., V. Siakoulis, E. Stavroulakis, and A. Klamargias. 2019. “A Robust Machine Learning Approach for Credit Risk Analysis of Large Loan Level Datasets Using Deep Learning and Extreme Gradient Boosting.” IFC Bulletin 49: 1486–506, Bank for International Settlements, Basel.  
- Wang, J. C., and C. B. Perkins. 2019. “How Magic a Bullet Is Machine Learning for Credit Analysis? An Exploration with FinTech Lending Data.” Federal Reserve of Boston Working Paper 19–16.  
- Gambacorta et al. 2019. (See above in AI, Machine Learning, and Financial Services.)

### Regulation, SupTech/RegTech, Governance, and Ethics
- De Nederlandsche Bank (DNB). 2019. “General Principles for Use of Artificial Intelligence in Finance.” DNB, Amsterdam.  
- di Castri, S., S. Hohl, A. Kulenkampff, and J. Prenio. 2019. “The Suptech Generations.” FSI Insights on Policy Implementation, Bank for International Settlements, Basel. https://www.bis.org/fsi/publ/insights19.pdf.  
- Financial Stability Board (FSB). 2020. “The Use of Supervisory and Regulatory Technology by Authorities and Regulated Institutions: Market Developments and Financial Stability Implications.” Financial Stability Board, Basel, Switzerland. https://www.fsb.org/wp-content/uploads/P091020.pdf.  
- FinCoNet. 2020. “SupTech Tools for Market Conduct Supervisors.” Paris.  
- Institute of International Finance (IIF). 2017. “Deploying Regtech against Financial Crime.” Report of the Regtech Working Group, IIF, Washington, DC. https://www.iif.com/portals/0/Files/private/32370132_aml_final_id.pdf.  
- MAS. 2018. “Principles to Promote Fairness, Ethics, Accountability, and Transparency (FEAT) in the Use of Artificial Intelligence and Data Analytics in Singapore’s Financial Sector.”  
- OECD. 2019. “Recommendation of the Council on Artificial Intelligence.” OECD Legal Instruments, OECD, Paris. https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449.  
- OECD. 2021. “STIP Compass Database.” OECD, Paris. https://stip.oecd.org/stip.htm.  
- Sahay, R., U. E. von Allmen, A. Lahreche, P. Khera, S. Ogawa, M. Bazarbash, and K. Beaton. 2020. “The Promise of Fintech: Financial Inclusion in the Post–COVID-19 Era.” Departmental Paper 20/09, International Monetary Fund, Washington, DC.  
- United Nations (UN). 2019. A United Nations System-Wide Strategic Approach and Road Map for Supporting Capacity Development on Artificial Intelligence, CEB/2019/1/Add.3 (17 June 2019). rom undocs.org/en/CEB/2019/1/Add.3.  
- UNESCO. 2021. Intergovernmental Meeting of Experts (Category II) related to a Draft Recommendation on the Ethics of Artificial Intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000373434.  
- Financial regulation and supervisory AI pieces by ECB, DNB, FSB, di Castri et al., and MAS (see entries above).

### Technical Machine Learning Research, Interpretability, and Security
- Chandola, V., A. Banerjee, and V. Kumar. 2009. “Anomaly Detection: A Survey.” ACM Computing Surveys 41 (3). https://doi.org/10.1145/1541880.1541882.  
- Cloudera Fast Forward. 2020. “Interpretability.” Cloudera Fast Forward Labs Research, Santa Clara, CA. https://ff06-2020.fastforwardlabs.com/ff06-2020-interpretability.pdf.  
- Corbett-Davies, S., E. Pierson, A. Feller, S. Goel, and A. Huq. 2017. “Algorithmic Decision Making and the Cost of Fairness.” In Proceedings of the 23rd International Conference on Knowledge Discovery and Data Mining, 797–806.  
- Comiter, M. 2019. “Attacking Artificial Intelligence: AI’s Security Vulnerability and What Policymakers Can Do About It.” Belfer Center for Science and International Affairs, Harvard Kennedy School. https://www.belfercenter.org/publication/AttackingAI.  
- Goodfellow, I., Y. Bengio, and A. Courville. 2016. Deep Learning. Cambridge, MA: MIT Press.  
- Goodfellow, I. J., J. Shlens, and C. Szegedy. 2014. “Explaining and Harnessing Adversarial Examples.” Cornell University. https://arxiv.org/abs/1412.6572.  
- Guidotti, R., A. Monreale, S. Ruggieri, F. Turini, D. Pedreschi, and F. Giannotti. 2019. “A Survey of Methods for Explaining Black Box Models.” ACM Computing Surveys 51 (5): 1–42.  
- Hinton, G., Y. LeCun, and Y. Bengio. 2015. “Deep Learning.” Nature 521 (7553): 436–44.  
- Hirshberg, J., and C. D. Manning. 2015. “Advances in Natural Language Processing.” Science 349 (6245): 261–66.  
- Liu, K., B. Dolan-Gavitt, and S. Garg. 2018. “Fine-Pruning: Defending against Backdooring Attacks on Deep Neural Networks.” In Research in Attacks, Intrusions, and Defenses, edited by M. Bailey, T. Holz, M. Stamatogiannakis, and S. Ioannidis, 273–94. Cham, Switzerland: Springer.  
- Molnar, C. 2021. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. https://christophm.github.io/interpretable-ml-book/.  
- Percha, B., and R. B. Altman. 2015. “Learning the Structure of Biomedical Relationships from Unstructured Text.” PLoS Computational Biology 11 (7): e1004216.  
- Ribeiro, M. T., S. Singh, and C. Guestrin. 2016. "’Why Should I Trust You?’: Explaining the Predictions of Any Classifier.” Cornell University. https://arxiv.org/abs/1602.04938.  
- Ribeiro et al., Guidotti et al., Cloudera, and Molnar (see entries above) on interpretability and explaining black box models.  
- Shapley, Lloyd S. 1953. “A Value for N-Person Games.” Contributions to the Theory of Games 2 (28): 307–17.  
- Szegedy, C., W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. 2014. “Intriguing Properties of Neural Networks.” Cornell University. https://arxiv.org/abs/1312.6199.

### Fairness, Bias, and Human Factors
- Friedman, B., and H. Nissenbaum. 1996. “Bias in Computer Systems.” ACM Transactions on Information Systems 14 (3): 330–47.  
- Corbett-Davies et al. 2017. “Algorithmic Decision Making and the Cost of Fairness.” (See above in Technical Machine Learning Research.)  
- Kleinberg, J., H. Lakkaraju, J. Leskovec, J., Ludwig, J., and S. Mullainathan. 2018a. “Human Decisions and Machine Predictions.” The Quarterly Journal of Economics 133 (1): 237–93.  
- Kleinberg, J., J. Ludwig, S. Mullainathan, A. Rambachan. 2018b. “Algorithmic Fairness.” AEA Papers and Proceedings 108: 22–27.  
- Kleinberg, J., J. Ludwig, S. Mullainathan, C. Sunstein. 2019. “Discrimination in the Age of Algorithms.” Journal of Legal Analysis 10: 113–74.  
- Mayson, S. G. 2019. “Bias In, Bias Out.” The Yale Law Journal 128 (8): 2218–300.  
- Miller, A. P. 2018. “Want Less-Biased Decisions? Use Algorithms.” Harvard Business Review, July 26. https://hbr.org/2018/07/want-less-biased-decisions-use-algorithms.  
- Silberg, J., and J. Manyika. 2019. “Notes from the AI Frontier: Tackling Bias in AI (and in Humans).” McKinsey Global Institute.  
- Hao, K. 2019. “This is How AI Bias Really Happens—And Why It’s So Hard to Fix.” MIT Technology Review, February 4.  
- Plous, S. 2002. “The Psychology of Prejudice, Stereotyping, and Discrimination: An Overview.” In Understanding Prejudice and Discrimination, edited by S. Plous. New York: McGraw-Hill.  
- Ponti, E.M., H. O’Horan, Y. Berzak, I. Vulić, R. Reichart, T. Poibeau, E. Shutova, and A. Korhonen. 2019. “Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing.” Computational Linguistics 45 (3): 559–601.  
- Triepels, R., H. Daniels, and R. Heijmans. 2018. “Detection and Explanation of Anomalous Payment Behavior in Real Time Gross Settlement Systems.” In Enterprise Information Systems, edited by S. Hammoudi, M. Śmiałek, O. Camp, and J. Filipe, 145–61. Cham: Springer Verlag.

### Macro, Policy, and Broader Digital Economy Context
- Danielsson, J., R. Macrae, and A. Uthemann. 2020. “Artificial Intelligence As a Central Banker.” VOX CEPR Policy Portal, March 6. https://voxeu.org/article/artificial-intelligence-central-banker.  
- Haksar, V., Y. Carrière-Swallow, E. Islam, A. Giddings, K. Kao, E. Kopp, and G. Quiros. 2021. “Towards A Global Approach to Data in the Digital Age.” IMF Staff Discussion Note, International Monetary Fund, Washington, DC.  
- Harker, P. 2020. “The Economics of Artificial Intelligence and Machine Learning.” Speech at the Official Monetary and Financial Institutions Forum, September 29, Philadelphia, PA.  
- Khan, A. 2018. “A Behavioral Approach to Financial Supervision, Regulation, and Central Banking.” IMF Working Paper 18/178, International Monetary Fund, Washington, DC.  
- Khan, A., and M. Malaika. 2021. “Central Bank Risk Management, Fintech, and Cybersecurity.” IMF Working Paper 2021/105, International Monetary Fund, Washington DC.  
- Sy, A., R. Maino, A. Massara, H. Perez-Saiz, and P. Sharma. 2019. “Fintech in Sub-Saharan African Countries: A Game Changer?” Departmental Paper 19/04, International Monetary Fund, Washington, DC.  
- Google and International Finance Corporation. 2020. “e-Conomy Africa 2020.” https://www.ifc.org/wps/wcm/connect/e358c23f-afe3-49c5-a509-034257688580/e-Conomy-Africa-2020.pdf?MOD=AJPERES&CVID=nmuGYF2.  
- Group of Twenty (G20). 2019. “G20 Ministerial Statement on Trade and Digital Economy.” https://www.mofa.go.jp/files/000486596.pdf.  
- Stanford University. 2019. “The AI Index 2019 Annual Report.” AI Index Steering Committee, Human-Centered AI Institute, Stanford University, Stanford, CA. https://hai.stanford.edu/sites/default/files/ai_index_2019_report.pdf.  
- World Economic Forum entries listed above.

*Source: pdeoraifea - References (pdeoraifea - References).*

---


_Source: https://www.imf.org/-/media/files/publications/dp/2021/english/pdeoraifea.pdf_