The spring’s high-profile Capitol Hill hearings with the heads of Facebook, Twitter and Google should have been a chance for lawmakers to demonstrate to the American public why Congress is such an important institution.

Since early 2017, concerns about how tech giants were securing and using the personal information of hundreds of millions of social media users had escalated to alarm. Still, despite bipartisan agreement on the growing need for regulations to better protect privacy and prevent social media platforms from being used to spread lies and hate speech, lawmakers haven’t lived up to their oversight task. Months later, they’ve made no significant headway — no bills tackling the issue have received markups or are moving on the floor.

The hearings were perhaps more useful as an illustration of the decline of policy expertise on the Hill, and how Congress has increasingly relinquished its policy-crafting responsibilities to the executive branch and private sector.

Lawmakers at the hearings appeared to have a poor understanding of social media fundamentals and were unprepared to offer policy solutions.

In April, when Orrin G. Hatch, the retiring octogenarian Republican senator from Utah, asked Facebook CEO Mark Zuckerberg at a joint hearing of the Senate Judiciary and Commerce committees how his company made money since it did not charge user fees, the 34-year-old company founder responded with a noticeable smirk: “Senator, we run ads.”

Sen. Lindsey Graham, Republican from South Carolina, asked Zuckerberg to supply his office with “some proposed regulations.” Graham is expected to lead the Judiciary Committee in the upcoming Congress.

Lawmakers’ questioning of Zuckerberg left outside viewers, including many technology writers and new media experts, perplexed. “It would be cool to see Zuckerberg questioned by people who know how Facebook works,” Dan Pfeiffer, a former top communications adviser in the Obama White House, wrote in a tweet that received close to 45,000 likes.

Since the 1990s, changes in the dynamics of Congress and how it does business have resulted in a significant loss of its capacity to perform oversight and push through complex legislation. And that has dangerous implications for the constitutional system of checks and balances and for average Americans’ ability to have their interests represented in some of the most important policy debates of the day, say more than a dozen current and former lawmakers, ex-Hill staffers, congressional experts and historians interviewed for this story.

“It’s really important that we do more to maintain the institution of Congress and those institutions around Congress that provide expertise,” said Illinois Democratic Rep. Daniel Lipinski. “Congress has really lost, unfortunately, a lot of the ability that we have to get good information, good advice when making policy.”

Additionally, lawmakers’ apparent abdication of much of their legislative responsibilities has prompted consecutive Democratic and Republican administrations to enact agendas via executive orders. That dynamic has resulted in a dizzying series of ping-pong policy reversals when an administration of a different party decides to completely retract a predecessor’s regulations.

That Congress is having a difficult time grappling with policy questions on not just social media but many other 21st century challenges did not come out of nowhere. It is the result of a trend that stretches back a generation to Newt Gingrich’s House speakership, and it’s picked up steam in the past decade.

Lawmakers’ lack of productivity and Congress’ loss of intellectual institutional capacity stem from a number of factors: historically high levels of partisanship; a shift of power from committee chairmen to House and Senate majority leaders; suspicion of expert advice that conflicts with ideologically-driven agendas; the election and elevation of inexperienced legislators; reduced legislative branch spending; and the continuous departure of staffers who leave for more attractive salaries in the private sector.

“Anybody who works in Congress will tell you it’s a problem,” said Lee Drutman, a senior fellow at New America’s Political Reform program. “A lot of folks within Congress have been skittish about doing anything about it as they don’t want to be seen as taking more for themselves, and there is a public perception out there that if Congress is terrible we should just starve it of resources. But the reason that Congress is underperforming is precisely because they are starved for resources.”

Republican and Democratic leaders in both chambers aren’t going out of their way to highlight the reasons for institutional decline.

Critics like Daniel Schuman, policy director at the liberal organization Demand Progress, say that’s partly because leadership does not see the benefit in drawing attention to the power consolidated in their own offices. And that discourages many lawmakers from pushing aggressively for an overhaul.

“Dealing with leadership is a Faustian bargain,” Schuman said. “If you play ball, you are going to get opportunities to do the things that you really want to do, but you also find that it significantly limits the things you want to do. If you play a completely outside game, then you can find yourself very marginalized.”

And yet the problems are so widely recognized that Congress earlier this year established a bicameral, bipartisan select committee to recommend ways to improve the budget and appropriations process. Lawmakers also approved funding in this year’s legislative branch spending law for multiple studies into the different factors harming Congress’ legislative abilities.

Longtime congressional analysts see a once-in-a-generation window opening to reverse Congress’ self-inflicted wounds. That’s largely due to House Democrats’ decisive win in the midterm elections and their campaign pledge to provide more oversight of the Trump administration. 

Bolstering the hope for change is significant lawmaker dissatisfaction with the existing, leadership-heavy approach and the emergence of an ideologically broad coalition of good government organizations and think tanks championing the need for improved congressional competency.

“We believe Congress is broken and now is the time to fix it,” said Illinois Republican Rep. Darin LaHood, who has sponsored a bipartisan measure with Lipinski that would form a joint committee to recommend ways to restructure Congress. “We are in much need of reform for Congress and how we operate.”

Advocates on and off the Hill who want to see Congress’ institutional capacity rebuilt say they are cautiously optimistic about the prospects for incremental changes in the 116th Congress that, if handled just right, could lead to more consequential improvements after the 2020 presidential elections. These short-term changes could include guarantees that popular bipartisan committee-advanced bills get floor time under an open rule so long as lawmakers refrain from offering so-called “poison pill” amendments.

“Despite what people read, members of Congress try to be diligent representatives for their constituents. If they go into a public policy debate underequipped, then they are less likely to succeed,” said Bradford Fitch, president of the Congressional Management Foundation, a nonpartisan nonprofit that provides training for Hill staffers. “The person who is usually advocating on behalf of John Q. Public is that member of Congress.”

The long decline

Congress’ last Golden Age is widely defined as the 1950s through most of the 1970s, a period that saw passage of numerous landmark laws such as the 1956 law establishing the interstate highway system, the 1964 Civil Rights Act and the 1969 law laying the framework for environmental policy.

The era was characterized by strong committee leadership, and White House proposals that largely met a responsive Congress. Regular order reigned and the filibuster was only rarely used in the Senate, according to Donald Wolfensberger, a former staff director of the House Rules Committee, now with The Wilson Center, who has written extensively about Congress’ institutional decline.

But concerns that committee chairmen had grown too autocratic, unaccountable and out of touch with the changing direction of the country caused the 1974 wave of post-Watergate lawmakers (who were overwhelmingly Democrats), to push for rules changes. Their efforts were intended to make Congress more transparent and to redistribute power away from committee chairmen and toward congressional leadership and the subcommittees, according to the recent book “The Class of ’74” by John Lawrence. Lawrence served as chief of staff to California Democrat Nancy Pelosi, including when she was speaker from 2007 to 2011.

Among other things, they supported moving toward a more structured rules process in the House. The ruling Democrats had less patience for an open amendment process, which they believed conservative lawmakers leveraged to offer divisive amendments aimed at impeding their civil rights agenda.

That trend continued into the 1990s, but it was the elevation of Gingrich to House speaker in 1995 that really kicked things into gear. The Georgia Republican implemented his campaign platform “Contract with America” with single-minded zeal.

At Gingrich’s behest, the House in 1995 adopted a rules package that drastically curtailed support for much of the chamber’s in-house expertise. It ended funding for the liberal Democratic Study Group, which provided rapid policy analysis of bills, and eliminated the Office of Technology Assessment. Other congressional support agencies also saw their staff sizes significantly reduced.

In 2015, there were 1,969 fewer people in the ranks of the Government Accountability Office, Congress’ internal investigative arm (then known as the General Accounting Office) — a 40 percent decrease from 1993 levels, according to data compiled in May by the Brookings Institution.

At the same time, staffing levels at the Congressional Research Service, Capitol Hill’s personal think tank, declined by 226 people, or 27 percent, while all 143 experts at the Office of Technology Assessment saw their positions eliminated by Gingrich. Only the Congressional Budget Office, which estimates the cost of legislative proposals, has grown — by five people to 235 today.

In retrospect, the elimination of OTA appears especially ill advised given all the problems lawmakers are having today grappling with not only social media but also cybersecurity, and the rapidly growing fields of automation and artificial intelligence.

“One hundred [plus] nerds were just cast on the street,” said Kevin Kosar, who co-directs the nonpartisan Legislative Branch Capacity Working Group, a project of the right-of-center R Street Institute and the liberal-leaning New America think tank, which aims to strengthen Congress.

“This happened just at the dawning of the internet so it was spectacularly bad timing.”

Lawmakers didn’t stop there. Convinced that slashing legislative spending was a politically winning strategy, successive Congresses approved budgets for themselves that didn’t keep pace with the federal government’s expanding size.

During the Obama administration, lawmakers even made cuts of hundreds of millions of dollars to their budget. This made it increasingly difficult to hire and retain the high-caliber staffers needed to handle complex policy challenges, such as climate change, cybersecurity, nuclear proliferation and the reordering of the post-World War II international order.

During a roughly 25-year period when the Code of Federal Regulations grew significantly — from about 140,000 pages to more than 180,000, according to a 2016 study by George Washington University — budget cuts cost the House 1,150 committee staffers, workers who are the supposed engines of congressional oversight and activity.

“You can see now that there is awareness that Congress is no longer capable of doing its job,” said Schuman, who for years has tracked Congress’ institutional decline.

Inexperienced, overstretched

It’s not just that there are fewer staffers. Those on the Hill today are greatly outnumbered and less experienced compared to their counterparts in the executive branch and private sector. That leaves them at a distinct disadvantage when it comes to advising lawmakers and helping shape policy decisions.

“The thing you hear all the time, especially from House staff, is that you have to be a mile wide and an inch deep,” said Aaron Jones, director of congressional relations at The Wilson Center and a former staffer for Kentucky Republican Rep. Harold Rogers. “There are so many issues that a House staffer has to cover that it’s hard to be an expert and to get an appreciable level of knowledge.”

In fact, a 2017 survey by the Congressional Management Foundation of more than 200 senior House and Senate staffers working in lawmaker offices found that only 15 percent were “very satisfied” with their colleagues’ knowledge and ability to adequately support lawmakers’ official duties.

“What was shocking was the stark contrast in some very key areas of where congressional staff felt there was a very strong need and where they felt that institutionally they and the members they served were not getting what they needed, and the implications of that for democracy,” said Fitch of the Congressional Management Foundation.

That same survey found continuous staff turnover on Capitol Hill. In fact, no position had a median tenure of more than four years.

The top reason staffers cited for leaving the Hill: a desire to earn more money.

“Staff salaries have always been low, but the last eight years is when it really started to bite,” said Jones, noting that when he started on the Hill in 2006 it was possible to rack up “pretty good” raises over a short period of time. “Talking to staffers now, they haven’t seen a raise in a couple of years.”

While those salaries have largely remained depressed over the past decade, the cost of living in Washington has shot through the roof. To pay congressional staff a third, or less, of what they could make elsewhere is “just unfair,” said former Virginia Democratic Rep. James P. Moran.

“As a result, the really good ones will move up the ladder, learn everything they can and then they will cash out,” Moran said. “If you have a family, it’s irresponsible not to cash out because Congress doesn’t pay what congressional staff are worth.”

When adjusted for inflation, staff pay has been essentially flat for the last quarter-century, according to Schuman.

The low salaries make it hard for lawmakers to hire experienced experts in most, if not all, of today’s top policy concerns, including the environment, health care, national security and emerging technologies.

A typical member of Congress “is not going to be able to afford a real expert on those things,” Jones observed. “You’re going to be able to get somebody who is learning on the job. A committee may sometimes get somebody who has been in an agency or who has worked in the private sector, but that is because the professional [policy] committees are able to offer more.”

Meanwhile, lack of experience among lawmakers is a big problem too. The average number of years of elected service for both senators and representatives has sharply declined from a decade ago, according to a 2018 CRS report. In the 111th Congress (2009-10), the average amount of experience in the Senate was 13.4 years, and 10.3 years in the House. In the current Congress, those numbers have fallen to 10.1 and 9.4 years respectively.

The incoming Congress will have even less experience following the large number of retirements among GOP House lawmakers and the drubbing the party took in the midterms. The election cumulatively produced the House’s third-highest turnover rate since at least 1974, according to the data-news analysis website FiveThirtyEight.

The shortage of policy experience has particular implications for House Republicans, since current conference rules limit committee chairmen to just three two-year terms. Those limits, as well as the unfavorable re-election climate for Republicans this year, drove eight GOP committee chairmen to choose retirement.

While their replacements in the new Congress have yet to be selected, the next cohort of House ranking members is expected to be significantly less experienced, which experts predict will impair their effectiveness.

“When they come up that fast, they haven’t had the experience of having to craft serious legislation and having to craft any compromises,” said Thomas Susman, a former Senate staffer and longtime congressional analyst. “They haven’t really worked across the aisle.”

Average legislative effectiveness, defined as how successful lawmakers are at moving their bills through Congress, goes up significantly for committee chairmen who hold the gavel longer, according to research by University of Virginia public policy professor Craig Volden and Vanderbilt University political science professor Alan Wiseman.

Weaker policy and oversight

Despite the downward trends in institutional knowledge and experience, lawmakers have demonstrated they can move into high gear when disaster strikes, even though that sometimes leads to rushed and bad policy.

The Sept. 11, 2001, attacks are an obvious example. The terrorist strikes in New York and Washington spurred lawmakers to establish the Department of Homeland Security, pass back-to-back warfare authorizations in 2001 and 2002, and enact the Patriot Act, which granted the government new foreign and domestic surveillance powers.

Since then, split jurisdictions across dozens of committees and subcommittees have made it especially difficult to oversee DHS.

Lawmakers from both parties have expressed a desire to update a pair of Authorizations for Use of Military Force because they do not have an expiration date and because it enables the Pentagon to massively expand the war on terror beyond the perpetrators of the Sept. 11 attacks. And the Patriot Act is detested by liberals and libertarians such as Sen. Rand Paul of Kentucky for its broad surveillance authorities and the excessive secrecy it gives to law enforcement agencies investigating possible terrorism concerns.

Elizabeth Saunders, an associate professor at Georgetown University specializing in the intersection of Congress and foreign policy, is pessimistic that lawmakers can significantly improve foreign affairs oversight unless a Cold War — or armed conflict — breaks out with China.

Given the lack of institutional and political incentives for specializing in foreign policy, it’s difficult to imagine the rise of lawmakers who can command the sort of broad, bipartisan respect for their policy acumen as Georgia Democratic Sen. Sam Nunn and Arizona Republican Sen. John McCain did in their days.

“The payoff of being known as being one of these big foreign policy people has declined,” Saunders said. “It’s just not going to really change much until you give them an existential reason to focus their attention on a big enemy.”

In addressing some more recent crises, such as the spring 2010 Deepwater Horizon oil spill, Congress failed to produce significant legal reforms. In their new book “The Committee,” Bryan Marshall, a political science professor at Miami University, and Bruce Wolpe, top adviser to then-House Energy and Commerce Chairman Henry Waxman, a California Democrat, write that the relevant policy committees worked as intended. They held oversight hearings with testimony from industry executives, and the Energy and Commerce Committee advanced legislation requiring numerous safety changes by the oil industry.

But re-election calculations ahead of the 2010 midterms caused Democrats and Republicans to harden their positions on a host of relevant issues, such as regulating offshore drilling and raising the corporate liability cap for future oil spills. Thus, a grand-bargain law proved unworkable.

Seeing that Congress was not going to decisively act, President Barack Obama used executive orders to institute new safety rules for offshore drilling operations, only to see them rolled back by President Donald Trump in his first year of office.

Washington’s policy response to the oil spill closely captures the 180-degree nature of the major policy developments of the last decade.

Obama significantly altered by executive order and regulation many of George W. Bush’s policies on climate change, immigration, civil rights, criminal justice and national security after a unified Republican strategy of obstruction on Capitol Hill left few prospects to enact a lasting legacy through legislation.

And Trump has easily reversed Obama-era policies, slashing regulations and government oversight of the private sector to a degree not even accomplished during the Bush years.

The only significant achievement of Obama’s tenure that Trump has not been able to undo is the 2010 health care law, a reminder, once again, of the long-term advantages of bipartisan congressional action versus temporary executive fiat.

Opportunities for change

The proposed rules changes House Democrats will consider in January have given outside analysts some cause for optimism, as have several of the measures included in the fiscal 2019 Legislative Branch Appropriations measure enacted in September.

The law requires several studies around the topic of recapturing lost congressional expertise. They include a study on the pay gap between Hill staffers, the executive branch and the private sector and a study on options for reconstituting the Office of Technology Assessment.

Democrats will take back control of the House with the biggest net gains they’ve had since the 43 seats they picked up in 1974 after Watergate, and expectations are high within the caucus that they will immediately begin aggressive oversight of the Trump administration.

Among the proposed rule changes is one that would establish a House select committee tasked with recommending ways to improve congressional operations.

Outside advocates hope the committee will examine questions about jurisdictional conflicts, particularly when it comes to overseeing the Homeland Security Department; whether lawmakers are over-stretched with too many committee assignments; and the effects of decades of policy centralization in the hands of House majority leadership at the expense of the authorizing committees.

Supporters of establishing a select committee say Congress is way overdue for a checkup. Previous joint committees on the organization of Congress, which differ from select committees in including lawmakers from both chambers, were established in 1945, 1965 and 1992.

But creating a committee to recommend changes is one thing. Finding the will to implement them is another.

Changes that independent analysts have long called for have not been undertaken in large part because they are politically difficult. They include shifting power away from the House speaker, allowing the minority party some influence over legislation, and significantly increasing Legislative Branch funding in order to rebuild lost institutional expertise.

“There’s a great tendency to think that organizational change is what is needed, and yet it’s a matter of a will to make the system work,” said Ira Shapiro, a former senior Senate staffer, who published a book earlier this year on institutional breakdowns within the Senate.

Despite the significant hurdles that remain to rebuilding congressional competency, advocates of rebuilding a Congress that is a co-equal to the executive branch in both theory and reality say they are determined to continue the work.

“If you have a process that people don’t feel buy-in into, then you’re just going to have discontent,” said Rep. Mike Gallagher, a first-term lawmaker who was frustrated to learn early as a freshman just how little influence rank-and-file members have over the policy debate. He hopes to see the next House speaker empower committees. “In order to wield power effectively going forward, you have to be willing to give it up and you have to be willing to devolve it down to the committees.”

Earlier this month, the Wisconsin Republican unsuccessfully attempted to convince his caucus to adopt an internal rules change in the next Congress that would allow committee members to elect their own chairmen rather than the Republican Steering Committee, which is dominated by leadership.

“It all boils down to a basic democratic accountability issue, which is in America do you want your laws and your government run by your elected officials or do you want special interests and unseen persons having a disproportionate influence on the laws that get made,” said Kosar, of the Legislative Branch Capacity Working Group.

“In Washington, D.C., knowledge brings with it political power, power to affect policy outcomes,” Kosar said. “And what we have seen over the decades is that lobbyists and special interests, they have stocked up on expertise.”