Episode 2
How the rich escaped taxation
Over the last 40 years, many of the wealthiest Americans have effectively seceded from taxation.
While regular Americans - from surgeons to bus drivers - pay large chunks (sometimes half!) of their income in taxes, lots of those with billions of dollars pay a teeny, tiny sliver of their wealth in taxes each year. And their heirs can inherit billions of dollars tax free.
How have we let this happen? And why don't most people know about it? Boston College scholar Ray Madoff - author of The Second Estate - talks about what the heck is going on.
Did you know you can also watch the show? Subscribe to the show on YouTube!
Transcript
So we're gonna start off with a little bit of trivia today.
Kara Miller:Who do you think was the lowest paid person at Meta in 2024?
Kara Miller:It was Mark Zuckerberg, and he made a dollar.
Kara Miller:And he's not the only very rich person who's collected a dollar for a year of work.
Kara Miller:Larry Ellison at Oracle has done the same.
Kara Miller:Sergey Brin and Larry Page at Google.
Kara Miller:Evan Spiegel at Snapchat.
Kara Miller:And why do these folks do this?
Kara Miller:Because the income tax, which I'm guessing most of us really dislike and have to pay every spring, is one of the highest taxes around.
Kara Miller:And one of the best ways to avoid the income tax is to avoid income.
Kara Miller:Welcome to It Turns Out.
Kara Miller:I'm Kara Miller.
Kara Miller:My guest today is Ray Madoff.
Kara Miller:She's a professor at Boston College.
Kara Miller:Her expertise is on tax law.
Kara Miller:She's the author of this new book, The Second Estate, a fascinating book.
Kara Miller:I loved reading it, but I felt like throwing it across the room a lot.
Kara Miller:Ray, thanks for being here.
Ray Madoff:Thanks so much for having me, Kara.
Ray Madoff:It's wonderful to be here.
Kara Miller:So let's just pick up where I left off with, all these incredibly rich people making a dollar a year.
Kara Miller:Now these are people that in some cases have a $100,000,000,000 or hundreds of billions of dollars.
Kara Miller:They probably do care about money.
Kara Miller:Why would they make a dollar a year if presumably they could ask for 50,000,000 and make it or, like, a 100,000,000 and make it?
Ray Madoff:Yes.
Ray Madoff:It's because, as you discussed in the beginning, the most heavily taxed type of income is salary income.
Ray Madoff:So the very first step of people who want to avoid taxes and can afford to do so is to avoid salaries.
Ray Madoff:And we see this all over, not just Mark Zuckerberg, as you mentioned, Elon Musk.
Ray Madoff:He he took zero so little that California went after Tesla and said, hey.
Ray Madoff:There's a labor problem here.
Ray Madoff:You're not paying him enough.
Ray Madoff:And they forced him to pay him a little bit more to meet the California fair fair labor, prices.
Ray Madoff:I'd say the highest paying one of the group is Warren Buffett, and he only gets a $100,000 a year, which is, you know, you think pretty low for the best stock picker in pretty much the history of the world.
Kara Miller:Modest.
Kara Miller:Although he's got, you know, obviously more than a $100,000, like, in billions.
Ray Madoff:Right?
Ray Madoff:And Jeff Bezos, he gets $80,000 a year, which has been enough to get him the child tax credit, which he has taken.
Ray Madoff:And so all of these people are keeping their taxes down by keeping their salaries down.
Ray Madoff:But as you say, they're not avoiding compensation altogether.
Ray Madoff:What they're doing is instead of taking, instead of relying on these salaries, these highly taxed salaries, And they're taxed so high, by the way, because these are, because wages are truly subject to double taxation.
Ray Madoff:They are subject to tax both for income taxes and for payroll taxes.
Ray Madoff:And for people who make salaries, these taxes are quite significant, even modest salaries.
Ray Madoff:Somebody who makes as little as $60,000 still pays $17,000 in payroll and income taxes.
Ray Madoff:And these taxes could make the difference between their ability to pay their rent or pay buy food or keep their car going.
Ray Madoff:It's really essential money, and yet the richest Americans are sidestepping this.
Ray Madoff:Now, of course, they're not sidestepping competition altogether, and we don't have to Not at all.
Ray Madoff:Right?
Ray Madoff:Yeah.
Ray Madoff:Right.
Ray Madoff:We don't have to feel too sorry for them.
Ray Madoff:If we look at what's happened just since 2023, right, a little over, you know, about two years ago, So we have Warren Buffett's wealth has increased by $29,000,000,000.
Ray Madoff:This was just the increase.
Ray Madoff:Jeff Bezos by 80,000,000,000, Zuckerberg by a 147,000,000,000, and Elon Musk by a 177,000,000,000.
Ray Madoff:This is just
Kara Miller:It's incredible.
Ray Madoff:Two years.
Ray Madoff:It's amazing.
Ray Madoff:But the advantages for them, this type of income is tax free.
Ray Madoff:They get to enjoy this growth of of wealth entirely free of income taxes and free of reporting.
Kara Miller:So you make the case that part of the reason that this wealth has grown so quickly is really because of the tax system.
Kara Miller:And you talk about 1937, I think it was, John D.
Kara Miller:Rockefeller, the first billionaire in The US dies.
Kara Miller:And then there's a few billionaires in the middle of the twentieth century.
Kara Miller:But really, late eighties, Bill Gates emerges with his first billion dollars, the first of many, obviously.
Kara Miller:And and since then, since the late eighties, there's just been a huge growth in the number of billionaires.
Kara Miller:So I wonder how the tax system, you know, not not, you know, people just earning more for their salaries.
Kara Miller:How did the tax system enable this wealth to just keep growing so quickly?
Ray Madoff:Yeah.
Ray Madoff:And so this is a really, important thing that I think it's natural for people to be confused about because the tax system has historically been created to be a bulwark against concentrations of wealth when you think about it.
Ray Madoff:Right?
Ray Madoff:We have progressive income tax rates.
Ray Madoff:We have an additional estate tax that's imposed on top of the income tax.
Ray Madoff:It all has it was all enacted to curtail concentrations of wealth, to show the legitimacy of a capitalist system that we could extract large amounts of money from our wealthiest citizens.
Ray Madoff:Right?
Ray Madoff:And yet what's happened over the past forty years is a number of changes have occurred that have basically allowed the wealthy to move into this tax free version of American life, to essentially secede from, from supporting the cost of government.
Ray Madoff:Right.
Ray Madoff:And what has changed is that prior to 1982, if somebody owned a lot of stock, the only way they could profit from from that stock the only way a company could share the profits from that stock was to issue dividends.
Ray Madoff:And dividends were taxed at the highest tax rate.
Ray Madoff:And so so when people owned stock, they got dividends, and they paid taxes.
Ray Madoff:However, in 1982, a subtle change occurred in the SEC rules.
Ray Madoff:And for the first time, companies were allowed to purchase their own shares of stock on the open market.
Ray Madoff:Now this sounds like some some you know, what difference could that possibly make?
Ray Madoff:But it had a massive transformation in the way that companies could share their profits with their shareholders.
Ray Madoff:Now instead of issuing dividends, right, they could purchase their shares on the open market, and the effect of purchasing the shares is to boost the value of the stock.
Ray Madoff:And so anybody who didn't need to sell could have their profits by the stock going up in value, and they didn't have to actually pay taxes on it.
Kara Miller:And, essentially, that is why somebody like Larry Ellison from Oracle has so much money.
Kara Miller:He has all these shares that he's had for so long.
Kara Miller:He hasn't liquidated them.
Kara Miller:They've appreciated in value.
Kara Miller:But I would think at some point, okay, you wanna buy your tenth home or your yacht or whatever.
Kara Miller:Don't you have to sell your shares to sort of finance that lifestyle?
Kara Miller:And maybe you don't have to pay the kind of money that people would pay for income tax, which yes.
Kara Miller:Okay.
Kara Miller:That's gonna be a higher tax.
Kara Miller:But wouldn't you have to pay 20% capital gains tax?
Ray Madoff:Yes.
Ray Madoff:So that's what's interesting.
Ray Madoff:Right?
Ray Madoff:You think like, okay.
Ray Madoff:They've got this growing stock, but for most of us, when we have property that's gone up in value, it doesn't mean anything to us until we actually sell it.
Ray Madoff:Right.
Ray Madoff:And that's why the tax law has a rule called realization.
Ray Madoff:You are only taxed when you sell the property.
Ray Madoff:Right.
Ray Madoff:And, otherwise, you're not subject to tax, and we sometimes call that mere paper gains.
Ray Madoff:But if your gains are in the sufficient magnitude, you don't need to sell because people are happy to lend you the money, and that's what these people do.
Ray Madoff:Larry Ellison, Elon Musk, all of these guys are borrowing at huge amounts to support their lifestyle.
Ray Madoff:And that borrowing is entirely tax free, and it's at quite good rates.
Ray Madoff:Because if you think about it, when you're lending to somebody who's given you, you know, $100,000,000,000 of stock, you know you're gonna be paid back.
Ray Madoff:And so everybody's happy to lend to them, and they never need to sell.
Ray Madoff:And moreover, they pass the property onto their kids.
Ray Madoff:Their kids don't need to sell either.
Ray Madoff:So this this perpetual nontaxation of, of, investment profits is one of the reasons that the wealthy have moved themselves, have been able to remove themselves from the tax system.
Kara Miller:I mean, comes back to your point about the notion that that these people have seceded from the tax system.
Kara Miller:It which to me is so incredible.
Kara Miller:Like we have all these programs, Social Security, Medicare, obviously we fund military.
Kara Miller:It sounds to me if you're not paying into the tax system, like, you're not help like, the people with the most money are not really helping out with those programs.
Kara Miller:Is that right?
Ray Madoff:That's right.
Ray Madoff:If they don't pay into the tax system, they are not helping out in the common expenses of the government.
Ray Madoff:And that's really, you know, what we've always thought what it meant to be part of a country.
Ray Madoff:Right.
Ray Madoff:I mean, one of the reasons that, the book is called the second estate.
Ray Madoff:Yeah.
Ray Madoff:And the second estate refers to the aristocracy of pre revolutionary France.
Ray Madoff:And the aristocracy of pre revolutionary France under law were written out of the tax system.
Ray Madoff:That was one of their privileges.
Ray Madoff:They had the right to hunt, carry a sword, and not pay taxes.
Ray Madoff:And minus the right to carry a sword, that's what we have here today in The United States.
Ray Madoff:Our richest Americans have the right to not pay taxes because our tax system no longer captures their wealth where it lives.
Ray Madoff:Now the other place, of course, is the estate tax, and that's a that's a separate story about this.
Ray Madoff:So when we think about inheritances, that's the where people often think, well, that's double tax.
Ray Madoff:Right?
Ray Madoff:It's a it's a very burdensome Right.
Ray Madoff:It's subject to tax to the owner.
Ray Madoff:It's subject to tax again when the person dies.
Ray Madoff:Right?
Ray Madoff:We've had all this double tax rhetoric there.
Ray Madoff:But in fact, our income tax system is drafted to be very broadly encompassing.
Ray Madoff:So the basic rule is gross income includes all income from whatever source derived.
Ray Madoff:And it includes things like if you win the lottery or if you walk down the street and you find something and it's valuable.
Kara Miller:Or if you can know if you walk down the street and you find somebody's $50 bill.
Kara Miller:Report it
Ray Madoff:on your taxes.
Ray Madoff:Okay.
Ray Madoff:Got taxes.
Ray Madoff:Also, if you do a barter exchange.
Ray Madoff:Right?
Ray Madoff:Somebody fixes your car in exchange for someone else fixing their sink, you're supposed to pay income taxes on those things, very broadly encompassing.
Ray Madoff:However, if you receive 10,000,000 or even a $100,000,000 by gift, life insurance, or inheritance, it is not only entirely free of tax.
Ray Madoff:There is no obligation for the person to report it.
Ray Madoff:They don't even have to tell you.
Ray Madoff:Nobody's business.
Ray Madoff:You don't don't worry.
Ray Madoff:We don't care.
Ray Madoff:And so it is written out of the income tax system, and some of it is justified by the fact that we believe it is captured by the estate tax.
Ray Madoff:However, due to this transformation that has happened over the past thirty five years or so, this is no longer the case.
Ray Madoff:This change came about as a result of 18 wealthy families who got together to try to to pursue the repeal of the federal estate tax.
Ray Madoff:And some listeners might remember this.
Ray Madoff:It was in the time of George w Bush, and he called it the death tax.
Ray Madoff:That's when the death tax rhetoric was created.
Ray Madoff:Right?
Ray Madoff:It's a, it's a tax that that, it's an unfair double tax that hurts family farms and businesses.
Ray Madoff:Right?
Ray Madoff:Right.
Ray Madoff:And and it was probably the most effective campaign of its kind in history.
Ray Madoff:I am amazed how many people will still say to me, even people from all political persuasions, well, oh, the estate tax, isn't that an unfair double tax that hurts family farms and businesses?
Ray Madoff:Right.
Ray Madoff:It kinda rolls off people's tongue, which is shocking.
Ray Madoff:You know?
Kara Miller:Right.
Kara Miller:Well, you make this point that that that it was such they got Frank Lunce, who's like, you know, the classic wordsmith.
Kara Miller:And he's he has this quote, and I'm not gonna say it right.
Kara Miller:But it's basically like, once we kinda get you in your heart
Ray Madoff:Yeah.
Kara Miller:And really convince you that, like, what?
Kara Miller:This is a tax at your death and, like, to family far like, once we get you at your heart, like, you can throw any amount of
Ray Madoff:That's right.
Kara Miller:Sort of statistics at this that you want, and people are not gonna be swayed.
Kara Miller:And and they rolled out farmers to testify about this.
Kara Miller:And you're saying, like, some of those farmers, it didn't it wouldn't even help them.
Kara Miller:And meanwhile, the real people, like, the heirs and heiresses to the Mars candy fortune are behind this, and these are the, like, the real people who benefit.
Ray Madoff:Absolutely.
Ray Madoff:The most effective person they had was a Chester Figpen who was the grandchild of sharecroppers, and he was this tremendously powerful he had a Christmas tree farm in, I think it was somewhere like Alabama, somewhere in the South.
Ray Madoff:And he was the face of the estate tax repeal movement, and and people loved it.
Ray Madoff:They wanted to call it the Chester Thigpen bill.
Ray Madoff:You know?
Ray Madoff:He was so effective.
Ray Madoff:He was saying, all I want is to pass the Christmas tree farm on to my children so that they too can carry the Christmas tree farm business on.
Ray Madoff:Right.
Ray Madoff:And the estate tax is gonna take away my right to do it.
Ray Madoff:Now, of course, this was completely false.
Ray Madoff:First of all, the estate tax has all sorts of provisions that provide for no for interest free loans and stretched out periods and special valuation rules to protect family farms and businesses.
Ray Madoff:Right?
Ray Madoff:And those could have been fortified to make them stronger, but that wasn't really what they wanted.
Ray Madoff:They were using them as a cover.
Ray Madoff:When Chester Thigpen died, it turns out he wasn't subject to the tax.
Ray Madoff:He had been misled to think that he was subject to the tax.
Ray Madoff:Right.
Ray Madoff:But he wasn't because his estate was much smaller than what the exclusion amount was.
Ray Madoff:But what's interesting about this tax and this campaign is that what it had a very small, tangible, to see success, which was that under George w Bush, he enacted this ten year plan in which the exemption amount would gradually increase from a million to 2,000,000 to 3,500,000.0.
Ray Madoff:And then in one glorious year, there would be no estate tax at all in the year 2011.
Ray Madoff:And George Steinbrenner died that year, and nine other billionaires died that year.
Ray Madoff:And they all passed their wealth entirely free of estate taxes and free of income taxes.
Ray Madoff:However, at the 2011, the tax came back.
Ray Madoff:And but it had been so it was supposed to come back with a $1,000,000 exemption, but it had been so effectively challenged that under even though Barack Obama was the president, he thought, you know what?
Ray Madoff:I'm gonna just increase the exemption even more so that we can get rid of this issue because I know this issue is hot button.
Ray Madoff:So he raised the exemption to 5,000,000 thinking, well, I've taken care of all the family farms and businesses.
Ray Madoff:Right.
Ray Madoff:But, of course, that wasn't the case.
Ray Madoff:Under Trump, he raised it to 10,000,000, and now most recently, he's raised it again to 15,000,000.
Ray Madoff:However, this is not the real story of the estate tax.
Ray Madoff:Because, of course, for people with billions of dollars, a $15,000,000 exemption is
Kara Miller:not gonna do anything for them.
Kara Miller:What
Ray Madoff:happened that was so much more effective is that everybody became so afraid of this estate tax issue that congress stopped doing its job.
Ray Madoff:And since 1990, thirty five years, congress has not enacted a single provision to close a loophole in the estate tax.
Ray Madoff:And as a result, the tax is a tax that stands in name only, and it is just, you know, filled with loopholes and exemptions for people to create crats and crats and grats and grats and q person, q dots, all the things I teach my students about.
Kara Miller:But we do, like they're these acronyms, right, for, like, I have no idea what, but ways of sheltering your money.
Kara Miller:Exactly.
Ray Madoff:Yeah.
Ray Madoff:And they are so effective that even though the wealthiest 1%, which is the group we're trying to get Right.
Ray Madoff:Currently own $50,000,000,000,000.
Ray Madoff:As of the 2024, they owned $50,000,000,000,000.
Ray Madoff:And we can put this number into context in a minute, but that's a very big number.
Ray Madoff:The entire amount collected by the estate and gift tax and the generation skipping tax, all those taxes, 30,000,000,000.
Ray Madoff:30,000,000,000 is an amount that Elon Musk has gained and lost in a single day, and that was all the estate tax collected.
Ray Madoff:And so it was less than one half of 1% of the total federal revenue collected, from the government that year.
Ray Madoff:I mean, it's tiny.
Ray Madoff:And so what I argue in the book is that the estate tax is really serving now as a cover for the rich rather than as a tax, and it's causing more harm than good.
Ray Madoff:One of the interesting issues is about why we've stopped hearing people complain about the death tax.
Ray Madoff:Right?
Ray Madoff:Where's the rich saying that it's an immoral double tax that hurts family farms and businesses?
Ray Madoff:They're very quiet about this now.
Ray Madoff:And that's because they are served better by preserving a tax that makes it look like they pay taxes taxes rather than had they rather than actually getting rid of the tax, which would reveal all the ways the income tax system lets them get off scot free.
Kara Miller:So I wanna get to like what do you what do we do about this in a minute.
Kara Miller:But one of the really striking things I found in the book is that people often think about the tax system as being progressive.
Kara Miller:In the sense that like, you know, if you make $60,000 as a teacher, you you pay us in taxes than a surgeon.
Kara Miller:Right?
Kara Miller:I mean like people know that the more you make, the more you pay.
Kara Miller:That's kind of how it works.
Kara Miller:And you really make this point that it is not as progressive as we think.
Kara Miller:So for example, let's say you make a $170,000 a year.
Kara Miller:You pay the same amount in payroll taxes as somebody who makes a million dollars a year.
Kara Miller:And I think it's and I don't mean the same percentage, I mean the actual same amount.
Kara Miller:And I think it's worth stopping for actually a minute and talking about payroll taxes, because I think about, oh, you know, this year, what bracket am I gonna be in when like I pay my taxes?
Kara Miller:Right?
Kara Miller:People are in the teens.
Kara Miller:They're in the twenties.
Kara Miller:They're in the thirties.
Kara Miller:Whatever.
Kara Miller:What I do not think about is on top of that, most of us, I think this is right, correct me if I'm wrong, are paying payroll taxes.
Kara Miller:And obviously, not Mark Zuckerberg because he made him a dollar.
Kara Miller:Right?
Kara Miller:But we're paying payroll taxes, and those have been very slowly creeping higher.
Kara Miller:And this has nothing to do with the money you pay in April.
Kara Miller:You wanna just talk about that for a minute?
Ray Madoff:Yes.
Ray Madoff:Exactly.
Ray Madoff:Payroll taxes, are called the hidden taxes because people don't even know that they're taxes.
Ray Madoff:They they're disguised in so many ways.
Ray Madoff:There's a wonderful episode of friends from many years ago where Rachel gets her first paycheck, and she says, who's FICA, and why is he getting my money?
Ray Madoff:Right.
Ray Madoff:Right.
Ray Madoff:Because people don't recognize that it's a tax.
Ray Madoff:It doesn't appear the same way regular taxes appear.
Ray Madoff:It's also called a contribution.
Ray Madoff:A contribution which makes it seem like maybe you're saving for your own retirement.
Ray Madoff:Right.
Ray Madoff:But in fact, what payroll taxes are are actual taxes, just the same as income taxes, that are used to pay a current expense of the government.
Ray Madoff:And that current expense are the Social Security and Medicaid benefits of current retirees.
Ray Madoff:So it is a tax like any other, but it is hidden.
Ray Madoff:It's hidden by these terms FICA and FUDA, and it's also hidden from an employee's perspective because an employer for people who work for others, the employer pays the other half pays half of the 15.3%.
Ray Madoff:But economists agree that even though the employer is the one writing the check, it really is a cost borne by employees who would otherwise make higher salaries.
Ray Madoff:And and so and for people who are self employed like so many people are these days, right, gig workers and all sorts of people who don't get a paycheck but earn their own money, they pay the full 15.3%.
Ray Madoff:And this is a tremendously burdensome task tax on working Americans.
Ray Madoff:Now, you know, the Social Security system is amazing and important, so I'm not I'm not saying anything against the Social Security system.
Ray Madoff:But we have to be mindful that it is truly a tax because when we look at the tax, it is a regressive tax so that somebody who earns a million dollars pays the same tax as somebody who earns a $176,000, which is the cutoff.
Ray Madoff:People sometimes justify that because, to some extent, lower wage workers get a higher percentage of their contributions in their retirement benefits.
Ray Madoff:Right?
Ray Madoff:But this is this is not make up for the fact that people who make more than a $170,000 entirely stop paying into the Social Security system.
Ray Madoff:And the problem is that we're now threatening you know, we've lots of politicians are saying, well, we have to end Social Security.
Ray Madoff:So that's not gonna do anything for people who are paying it now.
Ray Madoff:They're using these deficits in the Social Security system to say that the system doesn't work, whereas this could be fixed by simply raising the cap a little bit.
Kara Miller:But I think what you know, the point one of the points you make that's I think it's so important and that you've really been getting at here is that the people who fund the government, again, whether it's defense, Social Security, I don't care what it is, FEMA needs to do a cleanup, the people who fund the government are working Americans.
Kara Miller:And that it sounds like it goes all the way from like city bus drivers to, you know, surgeons.
Kara Miller:Yeah.
Kara Miller:But the people who do not fund the government are these incredibly rich people who I mean, they work, but that's not how they get their money.
Kara Miller:And it it seems incredibly unfair even to the surgeon.
Kara Miller:Right?
Kara Miller:That, like, Larry Ellison's not paying any taxes, basically.
Ray Madoff:Absolutely.
Ray Madoff:It's the high income earners who are paying a lot.
Ray Madoff:But I think because there's so much confusion about taxes, they think that their interests align with the wealthy.
Ray Madoff:They think those are my people more than, you know, the bus drivers and regular workers that we have in this country.
Ray Madoff:But it's not the case.
Ray Madoff:They're they are carrying the burden.
Ray Madoff:And, and that's why I'm working on a piece that's called now, if you are, if you're paying taxes, you're not really rich.
Ray Madoff:That's how you know you're really rich.
Kara Miller:I know.
Kara Miller:That's not how you know you've lost.
Kara Miller:So I I actually have a question about the surgeon or the the VP at the financial firm.
Kara Miller:If payroll taxes are 15%, and you're also paying, let's say, 37%, which I think is the highest rate for income tax, are there are are you saying that, like, people who make a lot of money, 400, 500, $600,000, are those people paying 50% in taxes?
Ray Madoff:Oh, yeah.
Ray Madoff:Yeah.
Ray Madoff:People are paying and then this also when you add state and local taxes, definitely, high income earners can be paying 50% in taxes.
Kara Miller:But people making $400,000,000
Ray Madoff:Yes.
Kara Miller:Are paying, like, 1% in taxes.
Kara Miller:Right.
Ray Madoff:Or not paying taxes.
Ray Madoff:Yes.
Kara Miller:So this leads to, I feel like, the central question.
Kara Miller:How is this possible?
Kara Miller:Whether you're a Democrat or Republican, I don't care.
Kara Miller:Almost nobody makes $400,000,000, and almost everybody makes, let's be generous, under a million dollars a year.
Kara Miller:So how has this not united, I don't know, everybody, pretty much, except for maybe a thousand people, saying, you know, yeah, there's a lot going on in politics now.
Kara Miller:We don't agree on, I don't know, climate change and guns and whatever.
Kara Miller:But I think we can agree that we don't wanna be paying a third of our money or half of our money in taxes while rich people I mean, really rich people.
Kara Miller:Yeah.
Kara Miller:Like, just literally as in the case of Larry Ellison live on their own island.
Kara Miller:I think he owns, like, the sixth largest island in Hawaii.
Kara Miller:And just don't like, as you say, have seceded from taxation.
Ray Madoff:Yeah.
Ray Madoff:I think that's absolutely right.
Ray Madoff:First of all, nobody would knowingly accept a system in which the richest Americans are written out of the tax system.
Ray Madoff:That's not what we believe in.
Ray Madoff:We believe in a system of progressive taxes.
Ray Madoff:We might not like taxes, but we understand the nature of that system.
Ray Madoff:And I think the reason that this has come to pass is because the public is both reluctant to talk about taxes because it's they've been led to believe that it's too complicated for them to understand or that the rich people, they get out of it by they're using their high priced lawyers, and they'll always beat the system.
Ray Madoff:But this isn't true.
Ray Madoff:They're getting out of it because we have these very large ways that we have stopped taxing investments and inheritances.
Ray Madoff:So one thing is that there is a a a sort of an intimidation factor, I think, that's at work.
Ray Madoff:But the other thing is there's also been a bit of a, what I'll call, a disinformation campaign.
Ray Madoff:And that is because there has been a marketing of certain statistics that make it seem like the richest Americans are paying taxes.
Ray Madoff:And one statistic that we hear a lot is the most recent version of it is the top 1% pay 40% of the income taxes.
Kara Miller:Okay.
Kara Miller:Okay.
Ray Madoff:Top 1% pay 40% of the income taxes, and 40% of Americans pay no income taxes at all.
Kara Miller:That second one I've heard a lot.
Kara Miller:I I've always thought the very poorest people, they don't really pay taxes.
Kara Miller:But, yeah, just probably both of
Ray Madoff:those myths.
Ray Madoff:So both of the okay.
Ray Madoff:So and that sounds like all you need to do is hear that statistic, and you think, okay.
Ray Madoff:I don't have to worry about the fairness of the system.
Ray Madoff:And if you read things like The Wall Street Journal opinion pages, they'll have articles that say we're already soaking the rich, and they bring out this statistic.
Ray Madoff:Okay?
Ray Madoff:And you sometimes hear other wealth reporters saying this.
Ray Madoff:However, you have to drill down just the tiniest bit.
Ray Madoff:Because when they say the top 1%, what they're referring to are those in the top 1% of income earners.
Ray Madoff:Those in the top 1% of income.
Ray Madoff:You're talking about your high paid your high paid employees, people who make a lot of income, they are carrying the lion's share of the income tax burden.
Kara Miller:So the richest Americans Yes.
Kara Miller:You make, say, a dollar a year, I'm guessing.
Kara Miller:If you make a dollar, you're spending the highest.
Ray Madoff:Yeah.
Ray Madoff:Being one the highest 1% of income.
Ray Madoff:And in fact, the richest Americans are just as likely to be in the 40% of Americans that pay no taxes because they don't have to pay taxes on either their growth in assets or in their inheritances.
Ray Madoff:And so that's where things you know, it gets confusing for people.
Kara Miller:Right.
Kara Miller:Okay.
Kara Miller:So let's talk about where we go from here.
Kara Miller:One thing that you say is that this is an unsustainable system.
Kara Miller:When the people with basically huge pools of capital
Ray Madoff:A third of our country's wealth.
Ray Madoff:Right.
Ray Madoff:A third of our country's wealth is owned by people.
Ray Madoff:They have $50,000,000,000,000 Right.
Ray Madoff:And they don't need to pay taxes.
Ray Madoff:They might choose to pay taxes for one reason or another.
Ray Madoff:Maybe they choose to take a big salary, but they don't need to pay taxes.
Ray Madoff:That's the point.
Kara Miller:So it seems like so tell me why that's unsustainable, and what happens if we just, like, keep going on with the people who have the most money not really paying tax.
Ray Madoff:Yeah.
Ray Madoff:Because, of course, the biggest problem is the growing national debt.
Ray Madoff:Right?
Ray Madoff:Now the national debt the interest on the national debt is more than we pay for national defense.
Ray Madoff:I mean, this has never happened before.
Ray Madoff:For a long time, the growing national debt was not a huge problem because interest rates were low, And so, you know, it was practically 0%, you know, 1%.
Ray Madoff:So we could maintain those costs, and the money was being spent in ways that truly fueled the economy.
Ray Madoff:Now the money is being spent just to cut taxes for more and more rich people.
Ray Madoff:And, and so, but yet we have this growing debt because we're not getting enough money.
Ray Madoff:And I think that as the public becomes aware, more and more aware about wealthy Americans not paying taxes, there will be a tipping point.
Ray Madoff:There will be a tipping point where tax fairness matters.
Ray Madoff:I mean, I think that, honestly, I think that both parties, are to blame.
Ray Madoff:I think the Democrats are to blame, in part for this too by focusing on people who make incomes of more than $400,000.
Ray Madoff:Like, you might hear that.
Ray Madoff:Right?
Ray Madoff:There was a sort of a, there there were a number of proposals that were that were good and would have addressed the problem of wealth holders, but there were things like you know, I I I think that living in the Northeast, I know people.
Ray Madoff:I have former students who, you know, they make $400,000.
Ray Madoff:These are not rich people.
Ray Madoff:They can they're basically because they the housing is so expensive in the Northeast as it is in other parts of the country.
Ray Madoff:If you have childcare, it's tremendously expensive.
Ray Madoff:That's even before kids get to college.
Ray Madoff:You know, this this is not these these people are not sitting pretty.
Ray Madoff:Right.
Ray Madoff:They are counting their money and trying to get by as as crazy as that sounds.
Kara Miller:No.
Kara Miller:No.
Kara Miller:I mean, they're not probably flying first class.
Kara Miller:They're definitely not flying private.
Kara Miller:I mean, they're not doing any of those fancy things.
Ray Madoff:No.
Ray Madoff:Absolutely.
Ray Madoff:So I think that, you know, when Democrats were pushing about increasing taxes, people over 400,000, I think that helped helped send some message to the high income earners that they were like the high wealth owners.
Ray Madoff:Right?
Ray Madoff:I think we have to separate, understand who belongs in in what camp and and, and address it, and address it that way.
Ray Madoff:The other thing I think we need to do is we need to clean up our tax system.
Ray Madoff:Right now, part so part of the confusion of things is that we have this payroll tax Mhmm.
Ray Madoff:That's hidden, the income tax that everyone sees, but it's highly progressive and people don't realize about how easy it is for the wealthy to avoid income.
Ray Madoff:Right.
Ray Madoff:And then this estate tax, which as we talked about raises very little money, but it has a very big presence in people's minds because of this history of this very powerful campaign about the estate tax being a a double tax that hurts family farms and business.
Ray Madoff:You know?
Ray Madoff:Right.
Ray Madoff:Huge numbers of the population think they're subject to the estate tax that simply aren't.
Ray Madoff:And and there's just a lot of confusion about it.
Ray Madoff:So I that's why I think the first thing that we need to do to clean things up is to actually and maybe paradoxically repeal the estate tax.
Ray Madoff:The estate tax is no longer collecting revenue.
Ray Madoff:It is simply providing a cover for the rich, and it and it serves also as a way of allowing these deep imperfections in our income tax system.
Ray Madoff:Right?
Ray Madoff:So we have things like, you know, the nontaxation of gains, and there's something called step up in basis of death, which some people might have heard about.
Ray Madoff:It's a way that all the gains are washed away at death for heirs.
Ray Madoff:All of these provisions are sometimes justified because, well, we have this burdensome estate tax, but the estate tax isn't actually being paid.
Ray Madoff:And so let's get rid of the estate tax, and now let's look at, well, what would a fair system be?
Ray Madoff:So when when Jeff Bezos goes for having stock worth nothing to stock worth a $160,000,000,000, should we say that that should never be subject to tax?
Ray Madoff:It's one thing to say and to say that, well, there's problems taxing it yearly.
Ray Madoff:And and I don't know whether you wanna get into that conversation
Kara Miller:or not.
Kara Miller:I certainly I mean, I certainly do remember Elizabeth Warren.
Kara Miller:I mean, she has been out there very vocally saying we need to sort of understand.
Kara Miller:I think, like, even have an inventory of, like, do people have art?
Kara Miller:Do people have a $100,000,000 in the bank?
Kara Miller:Whatever.
Kara Miller:I guess my question to you is, if you were, you know, king of the world Yeah.
Kara Miller:How would you how do you tax this stuff?
Kara Miller:Because people do have fancy lawyers and figure ways to sort of move around it.
Ray Madoff:Yeah.
Ray Madoff:I actually am very concerned about trying to adopt a wealth tax, or there's another one called, like, mark to market where you tax the annual gains.
Ray Madoff:And the concern comes from the fact that if we have annual taxes on the growth of wealth or on total wealth, wealthy people are going to be incentivized more than ever to take their assets away from easy to value assets like publicly traded stock and move them into difficult to value assets.
Ray Madoff:A huge amount of country's wealth is held in things like partnership interests, but they're not just single partnerships.
Ray Madoff:They're partnerships owned by partnerships owned by partnerships owned by, like, many levels deep.
Ray Madoff:Right?
Ray Madoff:And our IRS, extremely underfunded, has very little capacity to effectively monitor this on an annual basis.
Kara Miller:And I'm guessing the recent layoffs do not help their ability to monitor this.
Kara Miller:Right?
Kara Miller:Okay.
Ray Madoff:Exactly.
Ray Madoff:And so I think that annual taxes are going to invite too much too much attention by tax planners
Kara Miller:Okay.
Ray Madoff:To actually be effective and could result in a serious effect on our stock market, which so many Americans rely on.
Ray Madoff:But as wealthy move out of the publicly traded stock to privately held interest, more difficult to value privately held interest.
Ray Madoff:So I would be concerned about that.
Ray Madoff:But I think a better rule is a rule that was, proposed by here's a very unlikely pair.
Ray Madoff:Richard Nixon and Barack Obama.
Ray Madoff:Sure.
Kara Miller:Why not?
Kara Miller:And
Ray Madoff:and a similar rule was actually adopted in Canada.
Ray Madoff:Okay.
Ray Madoff:And this rule says that not only will you tax gains when property is sold, but you will also pay taxes on gains when property is either given away or passes on at death.
Ray Madoff:Whenever the person gives up ownership of property, then we should tally the gains and impose taxes.
Ray Madoff:And there's a lot of value about that rule because one of the things that it does is that it imposes taxes on the person who earned those profits.
Ray Madoff:Right?
Ray Madoff:Who bought the stock, watched to grow up.
Ray Madoff:They now get rid of it.
Ray Madoff:Fine.
Ray Madoff:Pay the tax and let the recipient start again with their new basis after the donor has paid the tax.
Ray Madoff:That's a cleaner system and one that would ensure that this tax free growth doesn't doesn't go on forever.
Ray Madoff:The other important thing is about inheritances.
Ray Madoff:So the estate tax, as I mentioned, is not doing its job, and we have to get rid of it.
Ray Madoff:But once we get rid of it, we need to bring gifts, inheritances, and life insurance back into the income tax system where it belongs.
Ray Madoff:Remember that person that found the $50 bill on the street?
Ray Madoff:Yeah.
Ray Madoff:They pay taxes.
Ray Madoff:Well, so too should the person who gets a $10,000,000 life insurance.
Kara Miller:But what's incredible is that right now, if you inherit $10,000,000 Yeah.
Kara Miller:I think in most states, you pay in, like that's your that's what you get that year.
Ray Madoff:Right?
Ray Madoff:And you get 10 get million.
Kara Miller:The whole thing
Ray Madoff:Tax free.
Kara Miller:No payroll tax.
Kara Miller:No nothing.
Kara Miller:Well, the person the the plumber is getting, like, all this tax and that tax and all sorts of taxes.
Kara Miller:Absolutely.
Ray Madoff:Yeah.
Ray Madoff:Okay.
Ray Madoff:Absolutely.
Ray Madoff:And nobody likes taxes, but we need to do a fair apportionment of the taxes.
Ray Madoff:Yeah.
Ray Madoff:That's the other thing.
Ray Madoff:When we have conversations about tax policy, people who are opposed to reforms, they often say, oh, well, you just want a big government or something like that.
Ray Madoff:It doesn't have to be spent on a big government.
Ray Madoff:It can be spent reducing the tax liabilities of other Americans.
Ray Madoff:So this is not about the size of the government spending.
Ray Madoff:It is about the apportionment of the government spending amongst the Americans.
Ray Madoff:Right.
Ray Madoff:How do American taxpayers divide the cost of government, whatever that size of government is?
Ray Madoff:And that's why I think both Democrats and Republicans should and probably do support a fair tax system.
Ray Madoff:Even if they disagree on the size of government, I think we should all agree that a fair tax system is something that we are entitled to.
Kara Miller:So finally, I wonder about your sort of your concern long term.
Kara Miller:So you called this book the second estate after like this sort of fancy group of people in France who didn't have to pay taxes.
Kara Miller:There was an op ed I remember, I think from twenty years ago, in New York Times by by the scholar Jared Diamond, and he talked about driving around Los Angeles, this image always stuck with me, and seeing gated communities where, you know, people drank bottled water, sent their kids to private school, had private security.
Kara Miller:And he worried greatly, first about that divide, because it's like you don't even live in the same country because you don't know anything about the schools, or maybe even the transportation system if you fly private or whatever.
Kara Miller:But I I think there's also a line in there where he said something like someday the walls may not be high enough.
Kara Miller:And and you've seen the rise of populism on both the right and the left.
Kara Miller:Yeah.
Kara Miller:I wonder what your fears are if we're just like, whatever, status quo, nobody's paying attention.
Kara Miller:That's cool.
Kara Miller:Let's just keep this going.
Ray Madoff:Honestly, I can't live in the fears.
Kara Miller:It's okay.
Ray Madoff:It's too intimidating.
Ray Madoff:I just have to assume that as people learn about things, there will be change.
Ray Madoff:You know?
Ray Madoff:I just have to believe in that because the alternative is too scary.
Ray Madoff:We're on a we're on a path of of divide in this country that is unsustainable.
Ray Madoff:I think that's something that everybody can agree on.
Ray Madoff:But I can't help but believe that, you know, things are on a pendulum.
Ray Madoff:We've seen it.
Ray Madoff:We've seen it swing both ways, and I I just think there's gonna be opportunity for things to swing back.
Ray Madoff:But what I want is that when it swings back, I want us to make smarter choices.
Ray Madoff:And I think some of the choices out there, things like we have to doubly burden billionaires.
Ray Madoff:Billionaires shouldn't exist.
Ray Madoff:All of that type of stuff.
Ray Madoff:I think that is the wrong way to go because it sends a false message.
Ray Madoff:It makes it seem like they're already being burdened, and we need to double burden them.
Kara Miller:Right.
Ray Madoff:No.
Ray Madoff:The problem is that they are not doing anything.
Kara Miller:Right.
Ray Madoff:And and so we need to be brought into the same system that all of us are operating under.
Ray Madoff:And then we can have one that that we can, you know, we can all share and keep up.
Kara Miller:Ray Madoff is the author of The Second Estate from University of Chicago Press.
Kara Miller:She's a professor at Boston College.
Kara Miller:Go get the second estate.
Kara Miller:It's great.
Kara Miller:You may throw it across the room, but it's fantastic.
Kara Miller:Ray, thank you so much.
Kara Miller:I really appreciate it.
Ray Madoff:Thank you so much for having me, Kara.
Kara Miller:Please subscribe to It Turns Out on YouTube and like us.
Kara Miller:It helps other people discover the show, or you can hear us on Apple Podcasts or Spotify.
Kara Miller:At our website, you can sign up for our weekly newsletter.
Kara Miller:I write it myself, and I'll let you know which researcher is coming up and what they're working on.
Kara Miller:Again, we're at itturnsoutshow.com.
Kara Miller:Thanks to the folks who worked on this show, including Matt Purdy and Nate Richards.
Kara Miller:Thanks also to the Alfred P.
Kara Miller:Sloan Foundation for their generous support, and thanks to you for listening and watching.
Kara Miller:Talk to you next time.
