After Kevin McCarthy’s Thursday announcement he won’t seek the speakership, Republicans are asking themselves who will. And the name everyone keeps coming back to is the one person who issued the first statement saying he wouldn’t go for the job: Paul D. Ryan.

Ryan has mastered the art of turning down the speaker’s gavel, so much so that the former vice presidential nominee and current Ways and Means chairman had a statement out within 20 minutes of the majority leader’s unexpected announcement.

“While I am grateful for the encouragement I’ve received, I will not be a candidate,” the Wisconsin Republican said.

Period. End of story, right?

Not so fast. Speaker John A. Boehner reportedly reached out to the Wisconsin Republican twice in the hours after McCarthy announced his decision. And while the reality is that Ryan has consistently insisted he wants nothing to do with the speakership, he may be the only Republican who could actually unite the conference.

“He’s really the only viable candidate and that’s the challenge that he has,” said Intelligence Chairman Devin Nunes, R-Calif. “Most of the members know that’s really our only path forward at this point.”

Nunes has made a name for himself as one of the most reliably establishment Republicans in the House. But even the most conservative voices in the conference were acknowledging Thursday that Ryan was the natural heir.

The House Freedom Caucus has noted it is sticking with Florida Republican Daniel Webster as its candidate, but even Justin Amash, one of the most conservative voices in the conference, acknowledged Ryan could excite a lot of members.

“I think we all know that Trey Gowdy or Paul Ryan would unite a lot of people, but neither of them has indicated that they are interested in running,” Amash said.

Instead, there have been rumors about the idea of an interim, or “caretaker” speaker. Someone who could bring the conference together temporarily, such as retiring John Kline of Minnesota or Boehner ally Tom Cole of Oklahoma. Or perhaps Lynn Westmoreland of Georgia, who told CQ Roll Call he would be making calls this weekend, but that he didn’t want to commit because he hadn’t consulted with his wife.

Majority Whip Steve Scalise, R-La., who claimed to have the votes locked up to advance to majority leader, hasn’t said what his intentions are. But when CQ Roll Call caught Scalise walking out of the Ways and Means room Thursday and asked if he would run, the Louisiana Republican wouldn’t commit either way.

“Obviously, I got to digest what just happened,” Scalise said.

Of course, Oversight and Government Reform Chairman Jason Chaffetz, R-Utah, instantly becomes a real contender for the position, and McCarthy’s announcement has breathed new life into Webster’s campaign. Asked if he had renewed hope Thursday, Webster told CQ Roll Call he had “a little bit of hope.”

“I didn’t lose today, how ’bout that?” Webster said.

While the HFC continues to support Webster, Amash said what the HFC cares about is process. “And if someone were to run and made those changes before the speaker election, then they have an opportunity to win a lot of votes,” the Michigan Republican said.

But even Amash conceded that under the current conditions, “Gowdy and Ryan are the ones who could consolidate votes pretty quickly.”

Gowdy has been just as emphatic as Ryan that he isn’t interested in leadership, but speculation about the South Carolina Republican hasn’t ceased either. On Thursday, Gowdy and Ryan walked to afternoon votes together, and they were seen talking on the House floor at length. Gowdy did not reveal what he and Ryan discussed, but alluded to Ryan’s family, which he cited as the reason he wouldn’t run for speaker last month.

“You cannot sell somebody on it. You have to persuade them, and the people that persuade you the most in life are people that you think have your best interests in mind,” Gowdy said. “So that’s why you talk to friends, and friends give you honest assessments.”

Gowdy said Ryan knew where he was in life, and he knew how old his children are. “You know the demands of the job,” Gowdy said. “It’s a very holistic conversation. It is not just about what’s best for the conference. It’s also, in some regards, what’s best for your friend.”

Gowdy noted that Ryan still has young children, and that the job of speaker is demanding. But Gowdy knows that Ryan is one of the few people who could bring Republicans together.

Asked if Ryan could unify the GOP conference, Gowdy had quite the response. “Either him or the fella that just spoke to us,” Gowdy said, referring to the Pope, “but he went back to Italy.”

Emma Dumain and Connor O’Brien contributed to this report.

Related: