The winner of the budding Senate Democratic primary between Rhode Island Secretary of State Matt Brown and former state Attorney General Sheldon Whitehouse may hinge on grass-roots support from the Latino community. And recent events suggest that support by Latinos — who account for about 9 percent of the Ocean State’s population — is up for grabs.

State Sen. Juan Pichardo (D), who had been Brown’s Providence co-chairman, withdrew his endorsement earlier this month, citing his close relationship with Whitehouse.

His withdrawal followed that of former Brown campaign treasurer Tomas Ramirez. Ramirez opted to remain neutral rather than defect to the Whitehouse camp.

Matt Burgess, spokesman for the Brown campaign, said Brown’s Latino support remains strong.

“The Latino community is critical in this campaign, and we are spending a lot of time reaching out to that community across Rhode Island,” he said. “Matt Brown has a long history of working with Latinos during his time at City Year, running the Democracy Compact [that registered new Latino voters] and as secretary of state.”

Latinos are particularly important to Brown because he established strong links to the community when he founded City Year Rhode Island, a public service organization that encourages volunteerism.

The stakes of the primary jostling are high: The winner of the Democratic primary will take on either Sen. Lincoln Chafee (R-R.I.) or possibly Cranston Mayor Steve Laffey (R), who is considering a GOP Senate bid. Either way, the seat is one of the national Democratic Party’s top targets in 2006.

The question of whether “Viva Brown!” will soar or fall flat is central to the question of whether Brown can succeed in a primary where he faces a wealthier, better-established and more experienced opponent with whom he shares few if any policy disagreements, according to Brown University political science professor Darrell West.

“Matt Brown announced very early, and at that point it wasn’t clear if there would be other Democrats coming in and he immediately signed up people, and then Whitehouse came into the race and was seen as a better candidate, better funded, etc., and so there were some defections,” West said.

A number of activists who had been cited as Brown supporters, including Ramirez, showed up on the host committee list of a recent Whitehouse fundraising letter.

Ramirez has donated $250 to both Whitehouse and Brown and said he would remain neutral during the primary.

“I did not defect from Brown’s campaign to support Sheldon’s campaign,” he said Friday in a posting on the Rhode Island Latino Political Action Committee’s Internet listserv. “I simply chose not to support one candidate over the other because of the good friendship that I have with both Matt and Sheldon. Thus, I made the decision to remain neutral during this ‘primary’ period,” he wrote.

As voters of Hispanic descent grow in numbers and gain political clout, the losses “could hurt Brown in the primary because Latinos are so crucial to a Democratic primary,” West said. Whoever is perceived as being more sympathetic to the needs of the Latino community could get a significant primary boost, he added.

Writing on the same listserv as Ramirez, local attorney Alberto Cardona criticized Pichardo for abandoning Brown and questioned Whitehouse’s commitment to the community.

“Yes it is true that Both Sheldon and Matt are good leaders and are committed to ‘Our State’ and yes it is true that I have in the past supported Sheldon. However!!!! as the old saying goes, ‘Dime con quien andas y te dire quien eres,’” he wrote in Spanish, which roughly translates as, “Birds of a feather flock together.”

Cardona then told the group to “Forget the ‘party line’ politics, and forget the party that seems to forget us.”

In the same discussion, a woman reminded the group that Whitehouse was attorney general when a young Hispanic girl who was not granted protective custody was murdered the same day she was set to testify in a state trial.

West said some in the Hispanic community blamed Whitehouse for the girl’s death and showed their displeasure in the 2002 gubernatorial primary, which Whitehouse lost. But West said many have moved on and that the incident is “becoming ancient history at this point.”

Beyond the ongoing debate within the Latino community, Brown may be facing an uphill battle in early polling and fundraising.

A Brown University poll of 470 Rhode Islanders conducted June 25-27 shows Whitehouse doing better against Chafee in a general election matchup. Chafee led Whitehouse 41 percent to 36 percent, while Chafee led Brown 44 percent to 29 percent. The poll’s margin of error was 4.5 percentage points.

Brown raised more than $500,000 in the first three months of the year, before Whitehouse entered the race.

In the quarter ending June 30, he raised a little more than $300,000 compared to Whitehouse’s $780,000.

Whitehouse began July with more than $1 million in the bank, thanks to a $360,000 donation to himself.

“Whitehouse impressed people with his fundraising, which could hurt Brown in the next quarter,” West said. “It’s early enough that Brown can recover, but first impressions matter a lot, and Whitehouse’s fundraising advantage means a lot at this point.”

“We are very pleased with the campaign. Matt Brown has out-raised Chafee and Whitehouse in 2005 and we are building a strong organization across the state,” Burgess said.

However, a Rhode Island Democratic source who did not want to be identified said Brown should worry.

“It’s very early, but if I was sitting where Matt is sitting right now, I would be concerned,” the source said.

Brown benefited in the first quarter when it seemed that Rep. James Langevin (D-R.I.) might enter the race, the source said. Langevin opposes abortion rights, while Brown (and Whitehouse) support them. Donors who were concerned about the choice issue flocked to Brown, the source said.

In the source’s view, “the bottom has pretty much fallen out of his fundraising.”

Some Democrats worry that the active fight in advance of a September primary could hurt Democrats’ chances.

In the past, Rhode Island Democrats have sometimes moved up the date of their convention at which they endorse candidates, said state party Chairman William Lynch.

“Historically in Rhode Island the late primary has hurt us,” Lynch acknowledged.

Moving up the convention is “something that we’ll consider,” he said, noting that the latest the party would endorse someone is a year from now.