Interview with Katherine Mayo: An agent-based analysis of real-time payments and fraud risk mitigation

In their paper Fraud Risk Mitigation in Real-Time Payments: A Strategic Agent-Based Analysis, Katherine Mayo, Nicholas Grabill and Michael Wellman consider real-time payments, and employ an agent-based model to investigate potential strategies for banks in the face of fraud. We asked Katherine about this work, why it is an important topic, and how the team went about tackling the problem.

What is the topic of the research in your paper and why is it such an important area for study?

Payments generally adhere to the following sequence of steps: initiation by the sender, processing by the bank, and then final funds are released to the receiver. The standard debit or credit transactions most people are familiar with often suffer delays in their processing of one or more days. However, recent advancements in technology have allowed for the introduction of a new, faster payment type boasting drastic decreases in processing times. So-called real-time payments (RTPs) can be processed in about 10 seconds, allowing for near-immediate receipt of funds. Many RTP systems are in use around the world today including FedNow in the United States and the Faster Payments Service in the United Kingdom.

The immediacy of RTPs can be highly beneficial for customers, especially when a payment is important and/or time sensitive (i.e., bill pay). However, customers aren’t the only ones to benefit. Malicious actors known as fraudsters may exploit the difficulty RTPs pose for fraud detection systems. To comply with the near-immediate processing requirement, banks may favor automated systems relying on information that can be inspected quickly, but may prove to be a less accurate indicator of fraud. Prior studies have highlighted the RTP fraud issue in the United Kingdom. One study found a 132% increase in fraud the year the Faster Payments Service was introduced. A second study identified authorized push payments, common in RTPs, to be the second largest type of payments fraud in the country in 2018.

Given the risk fraud poses to these systems, we seek to study how banks may strategically mitigate their RTP fraud risk by making trade-offs between different mitigation tools available to them. In particular, this paper focuses on investment in an RTP fraud detector—which suffers from subpar detection capabilities compared to detectors for standard payments—and the ability to limit customer use of these riskier payments. We also note the subsequent effects of bank choices on the strategic behavior of a fraudster and the effects on the payment network as a whole.

How did you go about tackling the problem?

Our approach employs an agent-based model of the payments system and analyzes a game played within it using empirical game-theoretic analysis. The model consists of banks, customers, and a fraudster all represented by nodes in a network. Nodes are connected by directed edges which represent financial relationships. Between bank nodes, the edges form the interbank network representing the willingness of banks to interact with one another on behalf of themselves or their customers. Banks hold deposits in bank accounts for customers, which customers may draw on to make payments within the network. These payments are modeled as a series of updates to the values on the edges between the payment’s sender and its receiver, capturing the exchanging of funds between all parties.

An RTP fraud game played by bank nodes and a fraudster node within the model is defined to capture the salient details of the fraud mitigation question. The bank nodes in the game attempt to balance mitigating their fraud risk, while attracting customers through their RTP offering. To do so, they select from a set of strategies determining an investment level for RTP fraud detection and setting a maximum payment value above which they will not allow their customers to send RTPs. A larger investment in fraud detection yields a more accurate fraud detector but at a higher one-time cost to the bank. As the goal of this work is to reason about the use of fraud detection systems, rather than build one, we abstract away implementation details surrounding fraud detection by representing all detectors solely as black-boxes characterized only by their accuracy detecting where payments are fraudulent or not. In selecting their strategy, banks seek to maximize their payoff at the end of the game which incorporates benefits customers provide the bank and the costs incurred by the bank for fraud and detector use. The fraudster’s strategies dictate its behavior, in particular how it selects which banks to target and which payment types to attempt, either standard or real time. The fraudster’s goal is simply to maximize the amount of fraud it commits over the course of the game.

After the banks and fraudster select their strategies, customer and fraudster payments are randomly generated over many time steps. Afterwards, the banks and fraudster receive their payoffs for the performance of their strategies. The game is analyzed using a method known as empirical game-theoretic analysis (EGTA), which employs extensive simulation of strategy profiles (assignments of strategies to agents) in the game with the goal of identifying Nash equilibrium behavior of the agents.

Could you talk about the main contributions of your work?

Our main contributions are two-fold. First, we offer insight into the strategic behavior of banks subject to RTP fraud risk. We find banks liable for RTP fraud will invoke both mitigation measures: investing highly in RTP fraud detection and restricting customer access. However, the ability to invoke two mitigation methods allows banks to strike a balance that does not require overly stringent limitations on customers. Banks also appear to be willing to accept partial liability for RTP fraud without implementing restrictions on customer use. Importantly, we discover the mitigation techniques strategically adopted by banks drastically impact fraudsters with minimal effects on customer experience.

Our second contribution is the introduction of a new method for assessing the strategy space known as the Strategic Feature Gains Assessment. This method employs empirical game-theoretic analysis in order to understand the benefit different subsets of the strategy space provide to agents. In this work, we apply the assessment to gain a broader understanding of the importance of the two possible mitigation techniques on their own and relative to one another. Through the application of this assessment, we uncover the importance of customer limitations for mitigating fraud risk. In particular, if a bank is only able to start with one technique, it should consider limits to customer use ahead of implementing fraud detection. This is important confirmation that RTP systems invoking restricted access measures are on the right track and those that do not may wish to consider them.

Could you tell us a bit about the experiments you carried out and what they revealed?

First, we apply EGTA to identify the Nash equilibria for several configurations of the RTP fraud game defined by customer demand for RTPs and the liability banks incur for fraudulent RTPs. Our results indicate that the potential to be liable for fraud has the greater effect on banks’ strategic choices. We find without liability for fraud, banks implement no mitigation measures and with partial liability they start to implement at least one measure. As banks become fully liable for the fraud that occurs, they are more likely to invest highly in RTP fraud detection and set customer limits, though never becoming overly restrictive. The fraudster, in response to banks, will target only RTPs while it is lucrative with little risk. However, as banks begin to implement both mitigation measures to combat fraud, the fraudster is pushed to attempt fraud with both payment types to maximize success. In all configurations it tends to be best for the fraudster to use historical information on success rate to select their bank target.

Not only are we interested in bank and fraudster behavior but we also seek to understand the effect their strategic choices have on the network as a whole. To study this, we measure various outcomes for network participants under the Nash equilibrium for each configuration. One particularly interesting measure is the rate of success for fraudsters and customers attempting payments, which we compare to the case where banks allow RTPs without implementing any mitigation techniques. The results show the Nash equilibria yield much lower success rates for fraudsters, while showing minimal impact on customers.

Lastly, we introduce the Strategic Feature Gains Assessment to better understand the individual and relative importance of the two mitigation techniques. This assessment measures the benefit a group of strategies (the deviation set) offers an agent for deviating from a base set of strategies, quantified as the maximum payoff gain an agent would receive for deviating from a strategy in the base set to one in the deviation set. For the purposes of this analysis, we conduct several assessments. The first studies the relative benefit of each technique given prior access to the other technique (i.e. the benefit of setting investment in fraud detection given the base set of only limiting customer access and vice versa). From the results, we are able to see restricting customers first offers the larger payoff gains. The second set of assessments determine the technique that will be implemented first, in the case where only one may be selected. We examined this under two scenarios: banks initially do not allow RTPs and RTPs are allowed with no mitigation measures. In all cases, a bank would choose to implement restrictions to customers first.

About Katherine

Katherine Mayo recently received her PhD from the University of Michigan where she was a member of the Strategic Reasoning Group. She is passionate about leveraging artificial intelligence to understand complex phenomena in the economics and finance domains. Her recent work, which has appeared at venues such as ICAIF and IJCAI, applies methods from agent-based modeling and empirical game theory to study strategic decision-making in financial payment networks.

Read the work in full

Fraud Risk Mitigation in Real-Time Payments: A Strategic Agent-Based Analysis, Katherine Mayo, Nicholas Grabill and Michael P. Wellman, IJCAI 2024.

Lucy Smith , Managing Editor for AIhub.

Lucy Smith , Managing Editor for AIhub.