Larry Summers’s campaign to replace Federal Reserve Chairman Ben Bernanke wasn’t doomed by any of the typical doubts about a potential Fed chief. Senate Democrats weren’t worried that Summers was too tolerant of inflation or insufficiently committed to quantitative easing. In fact, they weren’t worried about his opinions on monetary policy at all.

Summers fell because at least five Democrats on the Senate Banking Committee doubted his bona fides as a bank regulator. But even that doesn’t get at the whole truth. Summers really fell because those Senate Democrats — and many other liberals — don’t trust the Obama administration’s entire approach to regulating Wall Street. For all the talk of Summers’s outsized personality and polarizing past, he really lost because he was a stand-in for Obama.

Most clashes between the White House and liberals are disagreements over what’s politically possible rather than what’s optimal policy. On health care, the White House would have liked a public option but couldn’t find the votes. On the stimulus, the White House would have liked a bigger bill but didn’t see a way to pass it through Congress. On taxes, the White House would have liked more revenue but wasn’t willing to go over the fiscal cliff to get it. Liberals often disagree with the tactical compromises the White House makes to pass legislation, but the underlying philosophical divide is narrow.

The conflict over financial regulation is different. There, the White House really does disagree with its liberal base. The Obama administration didn’t oppose breaking up the big banks on tactical grounds. It opposed breaking them up because it thought that was bad policy. The administration would have opposed doing it even if Democrats had 80 votes in the Senate.

“When there was an opportunity to strengthen financial regulation in 2009 and 2010, the administration was less than enthusiastic,” says Simon Johnson, an economist at the Massachusetts Institute of Technology who has been a strong advocate of tougher financial regulations. “They didn’t support the Brown-Kaufman amendment to break up the big banks. Larry Summers isn’t  a fan of the Volcker rule. There was no interest in bringing back Glass-Steagall.”

That’s why the White House sales job for Summers didn’t work. White House aides argued that Summers would be a more effective regulator than Fed Vice Chairman Janet Yellen. They pointed out, fairly, that he’d been engaged in regulatory issues much longer, and he had begun calling for stronger financial regulations back in 2008. They emphasized that he had played an active and constructive role in the Dodd-Frank Act.

But liberals don’t share the White House’s definition of effective regulation. To them, Dodd-Frank is, at best, a beginning rather than an end — and not even that good of a beginning. On financial regulation, the left — and some on the right, such as Sen. David Vitter or Federal Deposit Insurance Corp. Vice Chairman Tom Hoenig — don’t just want to regulate Wall Street. They want fundamentally to change it.

Liberals want to see the biggest banks broken apart so they’re easier to oversee and less of a threat to the financial system if they go bust. They want to reinstate the Glass-Steagall Act, splitting investment and commercial banking. They doubt that Wall Street’s megabanks are worth the risk they pose to the economy, and they find the sector’s return to big profits and big bonuses obscene.

“I’m looking for a lot of things in a Fed chair, but one is really what they’re going to do about this concentration in bank power,” said Democratic Sen. Sherrod Brown, who serves on the Senate Banking Committee and, along with Vitter, has introduced a bill that would break up the biggest banks. “Seventeen years ago the six largest banks made up around 18 percent of GDP. Now it’s more than 60 percent.”

The Obama administration simply disagrees that this concentration is, in and of itself, a problem. “There is this conflation of pain and suffering with effective regulation,” said Michael Barr, who served in the Treasury Department from 2009 to 2010 and was a lead architect of Dodd-Frank. “When you push hard, for example, on the Glass-Steagall argument, you don’t get a lot of conviction from people that a combination of commercial and investment banking caused the financial crisis. It’s really about an uneasiness that enough was done to demonstrate to the financial sector that they can no longer do the things that led to the financial crisis.”

The Obama administration responded by forcing banks to hold more capital and giving regulators more power to supervise and, if necessary, dissolve systemically important financial firms. But liberals simply don’t believe that will work.

“The proof that these banks are still too big to fail are the advantages they have in the capital market,” Brown said. “There’s no reason they can borrow money at significantly less cost if the capital markets didn’t believe we’d rescue them. The fact that these banks are so big means they’re too big to fail. It means they’re too big to manage. And there’s little question that they’re too big to regulate.”

The left questions more than the Obama administration’s approach to regulating Wall Street. Many think the White House is, at the least, intellectually captured by the financial industry. “I think this administration, as is typical for presidents in this town, has too much Wall Street influence,” Brown said. “I think that’s pretty clear.” Summers’s repeated stints consulting for banks and hedge funds didn’t comfort liberals worried that the White House listens too closely to its friends in finance.

Yellen, the current front-runner for Fed chair, has never worked for Wall Street, but there’s little evidence that she shares liberals’ zeal for financial regulation. In 1997, she stood with the rest of the Bill Clinton administration in backing the repeal of Glass-Steagall. Nothing she has said since suggests she holds a vastly different outlook from her colleagues of that era.

But that’s mostly because she hasn’t said very much on financial regulation in recent years. And for liberals, a Fed chair who may be more inclined than the Obama administration to resize Wall Street is preferable to one who has definitely embraced the White House’s approach to financial regulation.