Sen. Lindsey Graham is coming under fire for his work on a bipartisan immigration bill, with conservative activists sounding an alarm that the South Carolina Republican is advocating “amnesty” for illegal immigrants.

NumbersUSA, a group that advocates a drastic reduction of immigration, sent out “an urgent action alert” to its 31,000 South Carolina members asking them to “turn up the temperature and call the Senator to remind him that South Carolina’s 337,000 citizens who can’t find a full-time job don’t want foreign workers competing with them and don’t want Amnesty,” the group said in a release.

Graham, who is up for re-election in 2014, has been working with a bipartisan group of eight senators that is drafting immigration legislation that it hopes to unveil soon after the recess.

In January, the group released a framework for its proposal, which includes providing a path to citizenship. Some conservatives view those efforts as amnesty, and they oppose allowing undocumented immigrants to eventually become citizens because they believe it will encourage more illegal immigration.

Graham has previously been defiant about his work on immigration, saying he thinks there is a conservative solution to the problem. Publicly, he has remained unconcerned about the possibility of a primary challenge.

The NumbersUSA release was triggered by a Bloomberg News story published Wednesday that quoted Graham as saying, “I don’t feel heat like I used to.” Graham reportedly made the comment at a March 26 news conference with local faith leaders who support the immigration negotiations.

“If you want to talk about immigration, you’re welcome to come down and talk to me. If you want to run ads, spend all the money you want to spend. I’m not backing down,” Graham continued.

NumbersUSA argued that his comment “makes it sound like he will be the puppet of the cheap-labor employers regardless of what the people of S.C. want or need.”

The group noted that South Carolinians needs jobs and that Graham should be working to provide them.

“Just how much heat does he have to feel before he listens to South Carolina’s 337,000 citizens who can’t find a full-time job and don’t want foreign workers competing with them,” the group said.

But Graham also said in the article that he’s not afraid to leave the immigration talks if he thinks it’s a bad deal. “I will do it in a heartbeat,” he said.

Despite some conservative blowback, Republicans have increasingly come out in favor of overhauling immigration and providing a path to citizenship. Noting the significant role the Latino community played in re-electing President Barack Obama, the Republican National Committee has urged support for immigration revisions.

Sen. Rand Paul, R-Ky., also recently backed the idea and was open to providing a path to citizenship.

NumbersUSA also put out an “action alert” against Rand Paul after he made the announcement.