Algorithms for Life: Game Theory

Welcome to UDAM Research, Yo-Odame Research, from our Algorithms for Life series by Valor Engels. Glad to be here for this one. Imagine you're stuck in traffic. Your phone shows a faster route, so you take it. So does everyone else. Now both roads are worse. Yeah, that is, uh, it's just a universally frustrating experience, right? It's truly maddening. Right. And it perfectly illustrates this mathematical concept called the price of anarchy. The price of anarchy. I love that term. It's a great term. Basically, the price of anarchy is the, um, the efficiency tax you pay for living in a world where nobody coordinates. Like when everyone's just looking out for themselves. Exactly. And in traffic networks specifically, studies show this lack of coordination costs us roughly 15 to 35 percent in efficiency. Wait, 15 to 35 percent just because we aren't talking to each other? Yeah. Everyone looks at their map app, acts in their own immediate rational self-interest to save, you know, three minutes. And as a result, everyone collectively loses 20 minutes. That is the perfect entry point into our topic today. Because that traffic jam, that feeling of making the right choice only to have it completely backfire because of everyone else, that's the absolute essence of game theory. It really is. To put it simply, game theory is the mathematical study of what happens when your best move depends entirely on what everyone else does. And, you know, most people hear game theory and think it's just about like chess or poker. Cold war nuclear strategy. Exactly. But it's not. It's about literally everything. It's about how you negotiate your salary, how you deal with your coworkers. Even how you use dating apps. Oh, heavily dating apps. Right. And for this deep dive, we are embarking on this massive exploration of human behavior. We've got a three-part mission today. A very ambitious one. Yeah. First, we're going to explore why deeply smart people routinely make collectively stupid decisions. Which happens a lot. Constantly. Second, we'll look at how the invisible design of rules saves and ruins lives, from how we allocate kidneys to how we place kids in classrooms. Mechanism design. Yeah. And finally, we'll look at what happens when the players sitting across the table aren't humans at all, but rapidly learning algorithms. It's a real journey from the quirks of human psychology to structural engineering. And then right out to the bleeding edge of artificial intelligence. Because the rules of the game are changing faster than we realize. By the end, you'll be asking, are you playing the game or is the game playing you? To even begin to answer that, though, we have to establish a baseline, right? Like, what do we actually mean when we use the word rational? Right. The rational actor. Yeah. In game theory, the absolute baseline of pure rationality is anchored by a concept called Nash equilibrium pronounced Nash equilibrium. Named after John Nash, the mathematician. Right. The guy from the movie A Beautiful Mind. Yeah. Russell Crowe. So to understand a Nash equilibrium, you have to imagine a completely stable state in an interaction. It's a scenario where no player can improve their outcome by changing their strategy alone. Okay. So I can't do any better unless someone else moves first. Exactly. If everyone else keeps their strategy the same, you have absolutely zero incentive to change yours. You are already doing the mathematical best you can. You're locked in. Makes sense. But, and this is a crucial distinction we have to make right away, Nash equilibrium describes where systems ultimately settle. It doesn't necessarily describe how individuals actually think in the moment. Okay. Let's unpack this. So if Nash equilibrium is like water finding its level, why do humans seem to splash around so much before settling? Splashing around. I like that. I mean, if I pour water into a glass, it doesn't just instantly teleport to a perfectly flat surface, right? It sloshes against the sides, it swirls, and eventually gravity pulls it flat. Humans seem to do a ton of sloshing. We don't just instantly flow to the logical point. That is a phenomenal metaphor. We splash around because we aren't perfectly rational supercomputers. We're biological creatures. We have emotions, limited attention spans, biases. We get tired. Exactly. There's this very significant gap between theory on a chalkboard and real human reality in a boardroom, and we can actually measure the splashing. How do we measure it? Well, there was a landmark meta-analysis published by Kammerer and Ho in 1999. They looked at 122 different experimental studies of how humans actually play games in labs. That's a huge data set. Massive. And what they found was deeply fascinating. Human subjects do actually converge toward that flat surface, toward the Nash equilibrium, but it takes time. So they don't just see the matrix immediately. Not at all. In relatively simple games, it takes about 10 full rounds of play for the splashing to stop. 10 rounds, just to figure out the mathematically obvious move. Yep, they have to poke and prod at the system, make mistakes and adjust. That feels so inherently human. We have to like touch the hot stove a few times before we trust the physics of the heat. That's exactly it. And even then, after those 10 rounds of learning, there is still a 15 to 25% deviation from the predicted perfect equilibrium. So even when we figure it out, we're still 20% off. Right. People get close, but they almost never play perfectly. And the problem gets significantly worse when people only play a game once. Like a one-shot game. Yeah. Think about how much of your life is a one-shot game. You only buy that specific house once. You only negotiate that specific job offer once. Right. There's no time for trial and error. Exactly. When there's no time to learn, human deviation from rationality is massive. And the best way to see this is by looking at the ultimatum game. I love the ultimatum game. It makes economists so mad. It really does. It was first conducted by Guth, Schmittberger and Schwarze in 1982. The setup is elegantly simple. Okay, walk us through it. You have two people in a room, a proposer and a responder. The proposer is given a pot of free money, say $10, and they get to decide how to split it. Between the two of them. Right. And the responder only has two choices. They can accept the split and they both keep the money, or they can reject the split, in which case the money is burned and nobody gets a cent. Okay, let me put myself in the shoes of a purely rational actor here. If I am a perfectly self-interested economic robot and you offer me one single penny out of that $10, I should accept it. Mathematically, yes. Because one penny is mathematically better than zero pennies. Exactly. And if you are also a perfectly rational robot, you know that I will accept the penny. So as the proposer, your optimal move, the absolute best thing you can do, is to offer me the minimum possible keep $9.99 and walk away. That's the Nash equilibrium, right? That is exactly what pure theory dictates. Proposers offer near zero. Responders accept literally anything. So the water should settle immediately at the 9.99 to 1 penny mark. It should. But what happens when you bring real human beings into the lab? Proposers generally offer roughly 40% of the pot. So they offer a pretty fair split right off the bat. They do. And even more interestingly, when proposers do try to act like rational robots and offer an unfair split like keeping $9 and offering one responders reject those unfair offers 40 to 50% of the time. Wait, 40 to 50%. I want the listener to really internalize what that means. People will literally choose to set their own free money on fire rather than accept an insult? Yes. Out of sheer spite, they will take zero just to ensure you also get zero. That is wild. We're talking about a massive 35 to 45 percentage point deviation from pure textbook rationality. Which forces game theorists to stop and recalibrate. I mean, if we aren't calculating Nash equilibria, if we're willingly burning cash to punish a stranger, what exactly is the architecture of our decision making? If I had to guess, it's that we don't just care about the money. We care about the social reality of the money. Yes, exactly. Human thinking involves systematic probabilistic errors, and we have deep evolutionary wiring for fairness. We don't exist in vacuums. So how do economists model that? To explain this statistically, researchers developed a much better model called Quantal Response Equilibrium, or Q-R-E pronounced K-Wan-Tol Response Equilibrium. Q-R-E. Got it. Instead of assuming players make perfect choices, Q-R-E assumes players play what are called noisy best responses. Noisy best responses. I like that phrase. It sounds like my entire career. It describes all of us. Under Q-R-E, individuals generally try to do what's best for themselves, but their execution is flawed. They make calculation mistakes. They let their ego into the room. They get tired. And their responses end up smeared out like a probability distribution, right? Yeah. Rather than a single sharp point on a graph. Yes. It's like throwing darts. Oh, that makes sense. The Nash equilibrium is the absolute dead center of the bullseye. The perfect rational choice. Exactly. But human beings have shaky hands. We're aiming for the center, but the dart hits an inch to the left or an inch to the right. We hit the board, but we're scattered around the center in a cloud of probability. That is a brilliant way to visualize it. And beyond that physical or emotional noise, there is a hard limit on our cognitive capacity. Our brain power. Right. Researchers use cognitive hierarchy models to map this out. Think of a game where you actively have to outsmart someone else. Most normal human beings do not sit down and think 10 steps ahead like a chess grandmaster. No, absolutely not. My brain hurts just trying to think three steps ahead. And studies consistently show that average people typically reason only one or two levels deep. There's this incredibly revealing experiment called the beauty contest game. Okay. I have to jump in here because the beauty contest game is my absolute favorite thought experiment. If you're listening to this, try to play along in your head. It's a fun one. The rules are simple. Imagine a massive room full of people. Everyone has to write down a number between zero and 100. Okay. The winner is the person whose guess is exactly two-thirds of the average of everyone else's guesses. So think about what you would guess. It is a brilliant trap because it forces you to think about how other people think. Exactly. Yeah. So let's walk through the levels. If you assume everyone else in the room is just completely random and clueless, the average of all their random guesses from zero to 100 will be 50. Right. So your winning move is two-thirds of 50, which is roughly 33. That is level one thinking. You are one step ahead of the crowd. But what if the crowd is smart? Right. What if everyone else in the room also did that exact same math? If everyone realizes the average should be 50, then everyone guesses 33. And if everyone guesses 33, then the new average is 33. So you need to be one step ahead of them. Two-thirds of 33 is 22. That is level two thinking. And if you keep following that logic down the rabbit hole, level three, level four, level five, the numbers get smaller and smaller. Until they hit zero. Exactly. The only mathematical place where the sequence finally stops and stabilizes, the Nash equilibrium, is zero. Because two-thirds of zero is zero. If you are a perfectly rational game theorist, the only answer you can possibly write down is zero. But here's the magnificent irony of the human condition. If you confidently walk into that room and guess zero, you will lose the game. You will lose spectacularly. Why? Because you assumed a level of rationality in the rest of the room that simply does not exist. Real world experiments show that most people stop at level one or level two. They get tired or they assume everyone else is dumb. Yeah. The actual winning guess in most of these studies is usually somewhere around 22 or 33. If you play the pure, perfect Nash equilibrium, you fail. Because being perfectly rational in an irrational world is a fundamentally irrational strategy. That is a great way to phrase it. Being mathematically perfect when your opponents are boundedly rational will cost you the game. You have to play the players, not just the board. But wait, I want to push back on this a little bit. If human beings deviate by 15 to 45 percentage points, if we burn our own money out of spite, and if we only think one or two levels deep before getting a headache, isn't Nash equilibrium basically useless for predicting real life? It seems that way, doesn't it? Especially in those one-shot scenarios I mentioned earlier, like buying a house or negotiating a salary. You only get one shot, so the water never has time to settle. Why do economists even care about Nash equilibrium if humans are this messy? Well, if we connect this to the bigger picture, you have to understand the critical difference between prediction and stress testing. Okay, what's the difference? Is Nash equilibrium a perfect crystal ball for predicting exactly what an irrational human being will do on a Tuesday morning? No, absolutely not. But major Fortune 500 companies use it heavily, not for calculating human behavior, but for something called structured anticipation. Structured anticipation. What does that look like in a boardroom? It looks like, looking at the worst case scenario, they use Nash equilibrium to stress test their corporate strategies against the absolute bedrock of a market. It tells you where the structural gravity of a situation is pulling everyone. So it's about a long game. Yes. Even if people deviate in the short term, even if there is splashing and noise and emotion, the Nash equilibrium tells you where the unyielding market forces are eventually going to drag the system over a long enough timeline. So the takeaway for the listener is this. If you are in a repeated game, a long-term relationship, a daily interaction with a vendor, expect things to eventually settle near that rational equilibrium. The gravity will win. Right. But in a one-shot game, expect massive, messy, emotional human deviation. Never rely on the person sitting across from you being perfectly rational. So we have these boundedly rational, emotional humans who care about fairness but get confused by complex math. This creates a perfect bridge to the next massive question. What happens when they interact? Exactly. What happens when you put multiple of these messy humans into a room together and force them to interact? We've talked about how people splash around the Nash equilibrium, but how do the rules of the room change the splashing? To explore this, we have to talk about the absolute crown jewel of game theory, the Prisoner's Dilemma. The famous Prisoner's Dilemma. Let's get into it. It's the canonical demonstration of how smart, self-interested individuals can make collectively disastrous decisions. So smart individuals making collectively stupid decisions. Exactly. It's a two-player game where both players would be significantly better off if they just cooperated, but each individual has an undeniable mathematical incentive to defect and betray the other person. Walk us through the classic setup. Paint the picture of the interrogation room for the listener. Sure. Imagine you and an accomplice have been arrested for a crime. The police put you in two completely separate, soundproof interrogation rooms. You have absolutely no way to communicate with your partner. Okay. Totally isolated. Right. The detective comes in and gives you a choice. If you both stay silent, if you cooperate with each other, the police only have enough evidence to give you both one year in jail. That is a great outcome for both of you. One year is entirely manageable. I just have to trust that my partner keeps their mouth shut. Right. But here's the trap. The detective says if you betray your partner and testify against them, but your partner stays silent, you will walk out of here completely free today. Oh, wow. And your partner will get 10 years in prison. And the reverse is also true. If you stay silent and your partner betrays you, you get the 10 years and they go free. And what if we both talk? If you both panic and betray each other, you both get five years. It's agonizing because no matter what my partner does, my individual selfish best move is always to betray them. Let's look at the math. Right. If my partner stays silent, betraying them gets me zero years instead of one year. If my partner betrays me, betraying them back gets me five years instead of 10 years. Mathematically, I must betray them. Exactly. The Nash equilibrium of the prisoner's dilemma is mutual defection. Two perfectly rational people will both betray each other, both get five years in prison, and completely miss out on the one year sentence they could have enjoyed if they had just trusted each other. It is the tragedy of rationality. It really is. But wait, earlier with the ultimatum game, we established that humans are not perfectly rational. We established that we care deeply about fairness and spite. So when researchers actually run the prisoner's dilemma with real people, do we actually betray each other as much as the math says we should? This is where we arrive at what might be the signature finding of this entire deep dive. It comes from a monumental meta-analysis published by David Sallie in 1995. OK, what did he find? Sallie looked at decades of prisoner's dilemma experiments, thousands and thousands of human subjects to see what actually gets human beings to cooperate instead of defecting. And the numbers he found are absolutely staggering. Give me the numbers. He discovered that if you simply allow the participants to have face-to-face communication before they are separated into the interrogation rooms, cooperation boosts by 40 to 50 percentage points. Let me stop you there. 40 to 50 percentage points. Yeah. Just from talking. Just from talking, looking someone in the eye and saying, hey, we are going to get through this together. That is incredible. Furthermore, if you change the rules so that their decisions are visible to each other rather than anonymous, visibility adds another 25 to 40 percentage points. Because they know they'll be seen. Right. And if you introduce an external rule, the threat of punishment for betraying cooperation jumps by another 30 to 50 percentage points. Here is where it gets really, really interesting. And I want the listener to grasp this. We are talking about the exact same human beings across these different setups. The exact same people. It is not that some people are inherently born as glowing angelic cooperators and other people are born as evil sociopathic defectors. The exact same person who will ruthlessly betray a stranger in an anonymous one-shot computer game will happily and loyally cooperate if they can just look the person in the eye first. Yes. The biology didn't change. The rules changed. Different rules yield entirely opposite behavior. Structure determines character. Structure determines character. That is perhaps the most profound practical lesson a game theory has to offer humanity. If you put good people in a badly structured game, they will behave badly. If you put selfish people in a well-structured game, they will cooperate. And this concept was proven beautifully, not just with humans, but with code. Right. In the 1980s by Robert Axelrod. Ah, yes. Axelrod's computer tournament. This is an amazing piece of history. Let's set the scene for Axelrod's tournament because it sounds like something out of a sci-fi novel. It was the dawn of complex computing. Axelrod wanted to know how cooperation could ever evolve in a world of purely selfish individuals. I mean, if biology is just selfish genes fighting for survival, how did altruism ever arise? Good question. So he hosted a computer tournament. He sent letters to experts all over the world, top economists, evolutionary biologists, computer scientists, sociologists, and invited them to submit computer programs, to play a repeated prisoner's dilemma against each other. It was literally a digital gladiator arena. These academics submitted incredibly complex, highly sophisticated blocks of code. They wrote programs designed to trick opponents, feign weakness, exploit patterns, and brutally outmaneuver the competition. Some of these programs were hundreds of lines of dense code. And yet, when Axelrod ran the simulation, having these programs play hundreds of rounds against each other, the ultimate winner of the massive tournament was a program that was only four lines of code. Submitted by a psychologist named Anatole Rapoport, it was called Tit for Tat. Tit for Tat. Four lines of code beat the smartest algorithms in the world. It did. Tit for Tat is beautifully, almost aggressively simple. On the very first move of the game, it defaults to trust. It cooperates. It starts nice. Yes. And after that first move, it simply looks at whatever its opponent did on the previous round, and it mirrors it. If you cooperate, it cooperates. If you betray it, it betrays you right back on the very next turn. So why did it win? How did a mirror defeat the most complex strategic algorithms on Earth? Axelrod analyzed the results deeply, and he found that Tit for Tat succeeded because it possessed four key characteristics that the other, more complex programs lacked. OK, what were they? First, it is nice. It never, ever defects first. It never initiates violence. OK. Second, it is retaliatory. It is not a pacifist. If you betray it, it instantly punches back. It doesn't let itself be a sucker. Retaliatory. Makes sense. Third, it is forgiving. And this is crucial. If the opponent realizes their mistake and goes back to cooperating, Tit for Tat immediately drops the grudge and goes back to cooperating too. So nice. Retaliatory, forgiving. And the fourth. And fourth, it is clear. Its logic is so simple that its opponents instantly understand how to play with it. There is no confusing its intentions. Be nice, set firm boundaries, forgive easily, and communicate clearly. It sounds less like game theory and more like a blueprint for a healthy marriage. It sounds like the perfect algorithm for human relationships. It does. But nothing is perfect. There is a twist in the Tit for Tat story. It has a fatal flaw. The fatal flaw of noise. Yes. In Axelrod's original, pristine computer tournament, every single move was executed exactly as intended. There were no typos. But in the real world, the world you and I live in, there is noise. What do you mean by noise in this context, like literal sound? No, think of noise as friction or simple human error. You might deeply intend to cooperate with your coworker, but you're exhausted. And your email sounds incredibly harsh. You might mean to be helpful to your partner, but you forget a key detail. The signal drops. Ah, I see. So what happens to a strict Tit for Tat strategy if there is just a 5% error rate in communication? If two Tit for Tat players are happily cooperating and one accidentally defects due to a misunderstanding, what does the other one do? Well, Tit for Tat is perfectly retaliatory. So if I accidentally send a harsh email, you, playing Kit for Tat, must retaliate with a harsh email of your own. Right. But then my algorithm looks at your harsh email and I retaliate against your retaliation. We lock into this endless death spiral of mutual destruction, punishing each other forever over a dropped cell phone signal. Precisely. In a noisy environment, strict Tit for Tat destroys itself. It lacks the capacity to absorb an accident. So researchers had to develop upgrades to make the strategy robust against the friction of real life. How do you upgrade it? One of the best upgrades is called generous Tit for Tat. It does the exact same thing as the original, but it includes a small built-in probability of forgiveness, usually around a 5% to 10% forgiveness rate. So even if you defect, there's a 5% to 10% chance I'll just look the other way and cooperate anyway just to try and hit the reset button on the relationship? Yes. It acts as a shock absorber for reality. Another highly effective upgrade is a strategy called win-stay-lose shift. Win-stay-lose shift. How does that work? It is incredibly simple cognitively, which is how humans actually operate. It just says, if my last move resulted in a good outcome for me, I'll do it again. If my last move resulted in a bad outcome, I'll change my behavior. Both generous Tit for Tat and win-stay-lose shift survive real-world noise vastly better than strict, unyielding retaliation. It's comforting to think that forgiveness is mathematically optimal, but the drama in the academic community didn't stop there. Because a few decades later, the idea that niceness always wins was seriously threatened. Oh, yes. The 2012 extortion drama. Right. In 2012, two researchers, Press and Dyson, published a discovery of zero-determinant strategies, and it sent absolute shockwaves through the field. It was a very dark day for game theorists who believed in the inherent power of cooperation. Press and Dyson mathematically proved that a player could use a specific, incredibly complex probabilistic strategy to unilaterally control the ratio of payoffs in a repeated game. In English, what does that mean? It means you could force your opponent into an extortionate relationship. The zero-determinant strategy was designed so that no matter what your opponent did, you would always ensure you got a larger share of the pie. Wow. And here is the truly insidious part. Because of the way the math worked, your opponent's only rational choice to maximize their own meager points was to accept being exploited. They mathematically had to bow to the extortion. So they proved that nice strategies could actually be systematically dominated by extortion. The bad guys could mathematically win. Yes. I can imagine evolutionary biologists sweating through their tweed jackets reading that paper. If extortion is the mathematical victor, how did human society ever build roads and hospitals? Exactly. But the despair was thankfully short-lived. Because just one year later in 2013, a researcher named Hilby and his colleagues published a brilliant reversal. The Hilby reversal. What did they find? They pointed out a crucial flaw in the zero-determinant model. They showed that while zero-determinant extortion strategies work flawlessly in a vacuum, a one-on-one interaction against a trapped opponent, they are completely evolutionarily unstable in a broad population. Why? What happens when an extortionist meets a crowd? When an extortion strategy is introduced into a large population of interacting players, the population eventually recognizes the extortion. The players realize they are being scammed. And what do humans do when they realize they're being scammed? They retaliate. Or they simply refuse to play with the extortionist entirely. The extortionist might dominate a few one-on-one encounters, but eventually they are ostracized and their strategy collapses under its own weight. So niceness wins again. It does. Hilby proved that niceness does win again, but it's an important caveat. It is a hard-fought battle where populations have to actively enforce cooperation and punish extortion. Which brings us from the realm of computer simulations back down to the reality of the office floor. We know that not everyone in your department or your neighborhood is going to read Hilby's 2013 paper on evolutionary game theory. Probably not, no. So how do the masses, everyday normal people, actually behave when placed in cooperative scenarios? To understand the masses, we look at the vital work of Urs Fischbacher and his colleagues from 2001. They wanted to categorize how normal people actually approach cooperation, and their findings should fundamentally change how anyone listening to this thinks about leadership. Okay, what do they categorize? They found that a massive majority of humanity, roughly 60% of people, fall into a category they called conditional cooperator. A conditional cooperator? Yeah. Defined as a person who matches the cooperation level of others. What does that look like in practice? It means they are chameleons. They look around the room, they see what everyone else is doing, and they match that baseline. So if the group is working hard? If the group is working hard, arriving on time, and sharing credit, the conditional cooperator says, okay, that's the culture here. I will work hard and share credit. But if the group is slacking off? If the group is slacking off, stealing lunches from the fridge and pointing fingers, the conditional cooperator says, well, I'm not going to be the only sucker working hard, and they slack off too. They just match the room. Exactly. Now, there is another distinct group in Fischbacher's research. About 30% of the population are pure free riders. The 30%? Right. They are the defectors. They will take advantage of the system, dodge work, and optimize for their own selfishness, regardless of what the rest of the room is doing. If you are listening to this right now, and you manage a team or run a community organization, or even just organize a family reunion, that 60-30 statistic should completely rewire how you operate. Think about the rules your HR department or your manager sets up. So often, leadership designs rules explicitly to catch and punish the 30% of free riders. Yeah, draconian trapping software. Strict punch-in times. Hostile oversight. But what does that actually do? It creates an atmosphere of deep distrust. It makes the environment toxic. And the 60% of conditional cooperators look around, sense the toxicity, and adjust their behavior downward. It is a profound leadership failure. Instead of fixating on the 30% of bad actors, you must design your rules to make good behavior highly visible and celebrated for the 60% who just want to match the room. If you manage a team, don't design your rules around punishing the 30% free riders. Design your rules to make good behavior visible for the 60% who just want to match the room. Exactly. If the conditional cooperators can clearly see each other cooperating, they will lock into a high-cooperation state, and the cultural gravity will pull the organization upward. There's also cross-cultural variations to this, as shown by Henrik and colleagues in 2010. But you have to be careful of anti-social punishment, where defectors actually try to punish cooperators. But overall, designing for the conditional cooperator is key. We've seen how rules change behavior organically, how structure determines character. But what if we work backward? What if we know the exact societal behavior we want, and we custom-build the math of the game to get it? This is a specialized, highly impactful field of economics known as mechanism design. Mechanism design. Let's define that. It is the reverse engineering of game theory to get desired behavior. You start with the goal you want to achieve, and you meticulously design the incentives and the rules to ensure that rational players inevitably arrive at that specific goal. And mechanism design isn't just academic whiteboard theory. It literally saves human lives. One of the most beautiful examples of this is the Roth Kidney Exchange. An incredible real-world application. Let's paint the picture of the before state, because the before state was a tragedy of inefficiency. Imagine someone you love deeply needs a kidney transplant to survive. You are perfectly willing to give them one of your kidneys. You are ready for surgery. Right. But the doctor runs a blood test, and there's a biological roadblock. Yeah. Your blood types or tissue types are incompatible. If you give them your kidney, your loved one's immune system will violently reject it. It's devastating. Under the old pre-mechanism system, you were simply out of luck. The swap rate for incompatible willing donors was practically non-existent. Maybe zero to five percent. You had the will to save a life, but no mechanism to execute it. So economists Alvin Roth, Typhon Semmes, and Utku Unver stepped in. They didn't try to change human biology, obviously. And they didn't try to guilt more people into donating. They engineered a new mathematical mechanism to facilitate what are called chain swaps. How does a chain swap work? It relies on a centralized algorithmic intervention. Suppose donor A wants to give to patient A, but they don't match biologically. Across the country, donor B wants to give to patient B, but they also don't match. OK. Two incompatible pairs. Right. The algorithm looks at the entire national pool of these incompatible pairs. It analyzes the complex web of blood types. And it discovers that donor A actually perfectly matches patient B, and donor B perfectly matches patient A. So they cross over. Yes. The algorithm coordinates a simultaneous, mutually beneficial swap. And it doesn't just stop at two pairs, right? These chains can become incredibly complex. You can have donor A giving to patient B, donor B to patient C, all the way down the line. Precisely. Because of this new mechanism, the efficiency of the system skyrocketed. Suddenly, 30 to 40 percent of incompatible pairs were able to successfully swap kidneys. Thousands and thousands of lives were saved. And not from an increase in human generosity. The bravery was already there. It was purely from a structural upgrade. Exactly. The rules of the game were fixed. It's a triumph. But mechanism design has a darker side. When you attempt to engineer complex systems involving millions of human lives, the side effects can be devastating. And this brings us to a gut punch reversal. England's school equity paradox. The school choice problem is notoriously difficult. Let's look at the original flawed system, often called the Boston Mechanism. If you are a parent, you will feel the stress of this immediately. Under the Boston Mechanism, parents had to rank their top school choices. If you ranked a school as number one, you had priority. Right. But if that school was incredibly popular and you didn't get in, you were bumped down to your number two choice. However, by the time they looked at your backup, all those seats were likely already taken by the parents who ranked it as their number one choice. It creates a terrifying strategic gamble. If your true first choice is an elite school, applying to it is a massive risk. If you miss, you lose your safe backup school, too. So the system forced parents to lie. Studies show that only about 60 percent of parents were actually reporting their preferences truthfully. Only 60 percent. The other 40 percent were actively gaming the system, hiding their true preferences, and strategically ranking safe schools first. And worse, this heavily penalized unsophisticated families, usually disadvantaged families, who didn't understand the unwritten strategy of the gamble. They'd rank elite schools honestly, get rejected, and end up in the worst schools. It was a disaster. So Mechanism designers swooped in with the Deferred Acceptance Algorithm, or DA, for short, pronounced Deferred Acceptance. How does the DA algorithm fix the gamble? In the Deferred Acceptance Algorithm, applications are iteratively processed. A school looks at the applicants and provisionally accepts a student based on priority-like test scores or proximity. But crucially, the acceptance is only provisional. It is deferred. OK, so if a different student with higher priority applies later? The school can bump the first student out. So if my kid gets bumped, what happens? Do they fall into the abyss like the old system? No, and this is the genius of it. The bumped student immediately moves down their list to their second choice, where they are evaluated against the current provisional pool. They might bump someone else out, causing a cascade. It's a matching algorithm. Exactly. The mathematical magic is that DA is strategy-proof, meaning truth-telling is always, mathematically, the absolute best approach. There is zero strategic advantage to lying. And on paper, it worked beautifully. When DA was introduced, truthful reporting by parents jumped from 60% to 80 or 85%. Parents could finally just list the schools they actually wanted without gambling. It sounds like a massive win. But here is the twist. A 2021 study by Terrier, Pathak, and Wren looked at what actually happened to the demographics. And they found that the new algorithm actually reduced access to good schools for disadvantaged families. Wait, how does a perfectly fair algorithm hurt the people it was supposed to help? Well, under the old scary Boston mechanism, affluent parents who had a lot to lose often played it safe. They didn't want to risk losing a perfectly good neighborhood school by gambling on an elite school. So they hedged. And that hedging left a vacuum at the elite schools. Exactly. A vacuum that was sometimes filled by less strategic, lower-income parents. But when the DA algorithm removed all the strategic risk, the affluent parents realized they could swing for the fences with zero penalty. So they flooded the elite schools with applications. Right. And because elite schools prioritize acceptances based on metrics that correlate with wealth-like test scores from expensive tutoring or geographic proximity to expensive neighborhoods, the affluent kids won the seats. But wait, I have to argue the other side here. Deferred acceptance is objectively better. It eliminates the guessing game. The equity issue is just a side effect of broader societal inequality. The algorithm itself is perfectly fair. This raises an important question. If the side effect of your perfect mechanism hurts the most vulnerable people in the system, is the system actually fixed? Strategy-proof and equitable are not the same thing. That's a profound point. Mathematical fairness can sometimes amplify pre-existing inequality. Researchers at CPEO actually proposed a structural fix to this, reserving roughly 15 percent of the seats at top schools specifically for disadvantaged students. And did that work? Simulations showed this simple patch could close the equity gap by 16 to 17 percent without destroying the strategy-proof nature of the algorithm. Mechanism design works great when everyone's cards are on the table. But what happens when someone is hiding their hand? That brings us to the infrastructure of hidden information. Economists categorize this into two main concepts. The first is adverse selection. Adverse selection. Let's define that. Adverse selection occurs when there is hidden information before a contract is signed. One party knows something critical that the other party doesn't. Give us an example. The classic example is George Akerlof's 1970 paper on the market for lemons. Imagine a used car market. There are good cars, peaches, and terrible cars with hidden flaws. Lemons. Only the seller knows if it's a peach or a lemon. So the buyer, taking on risk, will only offer an average price. Right. But the seller of a peach refuses to sell their great car for an average price, so they leave the market entirely. Eventually, only lemons are left, and the market collapses. Hidden defects cause market collapse. We see this everywhere. In housing, sellers who have hidden structural defects discount their homes 5 to 15% more aggressively to force a quick sale before an inspector figures it out. It happens at the micro level and the macro level. Like high-frequency trading, HFT firms exploit millisecond information gaps between stock exchanges. They see a price move a fraction of a second before anyone else and exploit that hidden gap. That front-running generates an estimated $20 billion annually. That's adverse selection hidden knowledge before the deal. The second concept is moral hazard. Moral hazard. This occurs after a contract is signed. It's risk-shifting. The classic example is renting a car. Once you buy the expensive daily insurance waiver, the financial risk is completely transferred from you to the insurance company. So you might drive a little faster or park closer to the shopping carts. Exactly. So how do we solve this? How do we build trust when we can't see the other person's hand? What are the solutions? One of the primary mechanisms is called signaling, developed in a 1973 model by Michael Spence. Spence argued that for a signal to be believable, it must be inherently costly to fake. Like a college degree. Labor statistics show a 25 to 40 percent wage premium for college grads. Right. And a large part of that premium is just a costly signal to prove competence to employers. It proves you have the diligence to sit through four years of rigorous grading. It's too costly for a highly unreliable person to fake. Reputation is another infrastructure we use. Look at the famous Resnick eBay study. They found that moving a seller's profile from zero reviews to exactly 100 positive reviews boosts the transaction success rate from 85 percent to 97 percent. The digital stars act as a structural proxy for trust. And we need these proxies because of human psychology. We have a deep phenomenon known as betrayal aversion. People hate being lied to far more than they hate losing money to bad luck. Because betrayal attacks the rules of the game itself. Exactly. Studies show that betrayal reduces your future trust by 30 to 50 percentage points more than an equivalent random loss. And sometimes trying to design a system to fix a problem creates the ultimate betrayal, a perverse incentive. Let's talk about the United Nations Clean Development Mechanism story. This is the ultimate cautionary tale. The intent was to pay developing nations to reduce greenhouse gas emissions. But it created the HFC-23 loophole. Hydrofluorocarbon-23, a deeply toxic greenhouse gas. The UN said to chemical plants, if you capture and destroy this HFC-23 gas instead of venting it, we will reward you with carbon credits you can sell. Let's look at the math here. Destroying the HFC-23 only cost the industry roughly $100 million. But the UN formula awarded them carbon credits worth $4.7 billion. So what did the plants do? They intentionally increased their pollution. They churned out extra toxic gas just so they could turn around, incinerate it, and get paid billions of dollars. They turned polluters into ransom seekers. We see similar backfires when we try to engineer civic duties. In 2020, England aggressively changed its organ donation rules to a soft opt-out policy. The logic was, if you presume everyone implicitly consents unless they opt out, donation rates should skyrocket. Right. The models projected a 78 percent consent rate. But the actual consent rate fell to 61 percent. Why? Because of family uncertainty. Under the old opt-in system, families knew for sure what their loved one wanted. But under opt-out, families faced terrifying uncertainty in a moment of grief. So they defaulted to saying no. The math ignored the emotion. And the ethical complexities go even deeper with Iran's compensated kidney market. Iran legally pays cash to living kidney donors. And from a pure math perspective, it eliminated their waitlist. It did. But the reality is that deeply impoverished people are selling body parts out of sheer desperation. It brings in Michael Sandel's critique. Turning a civic duty into a market transaction can destroy intrinsic motivation. OK, let's pivot. Earlier, we talked about the price of anarchy and traffic. But there's a modern version of this tax, and it's hitting your wallet directly. What happens when the players are algorithms instead of humans? This is the absolute frontier of antitrust law. We have to look at the Department of Justice's case against RealPage. And as a reminder, we are presenting this strictly as allegations and legal theory. Right. RealPage is a software company for property managers. The DOJ alleges that RealPage uses a hub-and-spoke model to artificially inflate rent prices. In the old days, a cartel was landlords meeting in a smoke-filled room to fix prices. But the DOJ alleges RealPage acts as the digital hub. The individual landlords are the spokes. RealPage allegedly collected non-public data from competitors to generate unified rental pricing. This is algorithmic collusion. Algorithms independently converging on anti-competitive prices. No human ever spoke to another landlord. The algorithm was the agreement. And it's not isolated. The Federal Trade Commission brought a case against Amazon regarding Project Nessie. The FTC alleges Amazon used an algorithm to see if competitors would follow price hikes. And if competitors matched the higher price, Nessie locked in the inflation, generating an alleged $1 billion in excess revenue. They also allegedly penalized sellers via the buy box. But here is the antitrust dilemma. The Sherman Antitrust Act requires a meeting of the minds, a conspiracy. The law is designed for smoke-filled rooms, not software code. Is the legal framework just broken? The framework is bending, not breaking. The DOJ's hub-and-spoke theory is testing this. Plus, Senator Klobuchar introduced the Preventing Algorithmic Collusion Act, creating legal presumptions. And the European Union AI Act classifies these algorithms as high risk. But RealPage is easy because there's a company in the middle. What happens when independent pricing algorithms just learn to collude on their own? That brings us to the Calvano et al. study. They looked at Q-Learning agents' advanced AI pricing algorithms. They set two independent algorithms to compete with no explicit programming to collude. And what happened? The algorithms independently discovered super competitive prices. Without any secret meetings, they achieved what human cartels do. They learned to keep prices artificially high through trial and error. And it's not just pricing bots. The newest players sitting across the table from us are large language models, or LLMs. Yes, when AI learns to play. A 2024 study had LLMs play the Ultimatum game and the Prisoner's Dilemma. Oh, did you? GPT-4 mimics human fairness norms in the Ultimatum game, rejecting unfair offers. But in the Prisoner's Dilemma, it is overly cooperative. It cooperates far more frequently than humans do, revealing biases from its training data. But that changes based on the prompt. I was reading a Moonlight review of Willis et al. They had LLMs play 1,000 round iterated tournaments with 10% noise. And the critical takeaway is that the prompt dictates if the agent is cooperative or aggressive. If you prompt it to be ruthless, it's ruthless. Prompt design is no longer just engineering. It is mechanism design. Exactly. And we see this collision in the wild. A 2025 study by Shaw looked at ride-sharing apps like Uber and Lyft. They used game theory to dynamically price rides. And drivers are colluding against the algorithms. Drivers collectively log off at the airport to trigger a massive price surge, then log back in simultaneously to capture higher fares. It's incredible. So what does this all mean for you? Let's build a rapid-fire toolkit based on everything we've covered today. All right. Number one, diagnose the game. Is it one shot or repeated? Match your strategy to the structure. Number two, design for the 60%. Build visible systems for conditional cooperators, not the 30% free riders. Number three, build your reputation infrastructure to solve information asymmetry. And number four, add forgiveness. Generous tit-for-tat beats strict retaliation when real-world noise happens. That's the toolkit. Remember that traffic jam from the opening? The problem was never the drivers. It was the road. Game theory's deepest lesson is that intelligence doesn't guarantee good outcomes. Structure does. The same brilliant people will cooperate or betray, depending entirely on the rules of the game. As we enter a world where you're negotiating not just with boundedly rational humans, but with rapidly learning algorithms, your ability to read the structure of the game is your only true defense. As you wrap up this UDM Research episode, remember, you're already in dozens of games. Your compensation structure, your team dynamics, your market position. Audit whether the rules reward the behavior you actually want. And if they don't, change the rules. For the full briefing and more episodes like this one, visit udm.ai. Yo, yo, Dom A dot AI. Keep analyzing the board. Research dot UDA dot me. That is Y-U-D-A dot M-E.