* Around 3,000 protesters rally at Thai PM's temporary HQ
* Death toll from Tuesday's clashes rises to 5
* Army spokesman urges both sides to avoid confrontation
* Court to rule later on legality of state of emergency (Updates with rally under way, quotes from protest leader and army spokesman)
By Amy Sawitta Lefevre and Apornrath Phoonphongphiphat
BANGKOK, Feb 19 (Reuters) - Protesters seeking to oust Thai Prime Minister Yingluck Shinawatra rallied at her temporary office on Wednesday, but the premier stayed away from the potential flashpoint a day after five people were killed in gunbattles in Bangkok.
A senior security official said police would not attempt for now to retake more protest sites, after Tuesday's "Peace for Bangkok Mission" saw the deadliest clashes since the anti-government demonstrations began in November.
Problems continue to mount for Yingluck, after an anti-corruption agency filed charges against her over a soured rice subsidy scheme that has stoked middle class anger and left hundreds of thousands of farmers, her natural backers, unpaid.
Yingluck, seen by opponents as a proxy for her brother, ousted former premier Thaksin Shinawatra, has been working from a Defence Ministry compound in north Bangkok since the protests forced her to vacate her Government House office.
"We came here because we do not want Yingluck to use the Defence Ministry complex any more," Chumpol Jumsai, a protest organiser, told around 3,000 supporters. "We're asking soldiers to stop letting Yingluck use this facility."
The protests are the latest instalment of an eight-year political battle broadly pitting the Bangkok middle class and royalist establishment against the mostly rural supporters of Yingluck and Thaksin.
Bluesky TV, the protest movement's own channel, showed footage of troops guarding the building behind barbed wire. In contrast to Tuesday's face-off with police, the atmosphere was not confrontational and protest leader Suthep Thaugsuban was allowed inside to speak to senior soldiers.
A spokesman for the military, which has said it would intervene if police were unable to maintain security in the capital, earlier appealed for both sides to avoid confrontation.
"Our strategy has not changed and is still to provide support to police," Colonel Werachon Sukondhapatipak told Reuters. "We have no intention of deploying extra troops. If the government needs extra help with security, it has to ask us and so far it has not asked for reinforcements."
The military has remained aloof from the latest crisis, but has a long history of intervening in politics, generally in support of the Bangkok establishment that includes the top brass, royal advisers and old-money families.
Violence flared on Tuesday as police made their most determined effort since the start of the protests to reclaim sites around government buildings occupied for weeks.
The Erawan Medical Center, which monitors Bangkok hospitals, said on Wednesday that one police officer and four protesters had been killed and 65 wounded. The death toll had earlier been put at four.
Police said they arrested more than 180 people for violating a state of emergency declared last month. A court was due to rule later on a challenge filed by protest leaders to the legality of that emergency degree.
National Security Council Chief Paradorn Pattanathabutr told Reuters there would probably be a pause while security forces assessed their options. "We don't expect to reclaim any protest sites further," he said.
Thai politics has been gripped by growing paralysis since Yingluck called a snap election in December.
Disruption by protesters meant voting could not be completed in the Feb. 2 poll, leaving Yingluck at the head of an enfeebled caretaker administration amid uncertainty over when a new government can be installed.
Demonstrators accuse Yingluck's billionaire brother Thaksin of nepotism and corruption and say that, prior to being toppled in a 2006 coup, he used taxpayers' money for populist subsidies and easy loans that have bought him the loyalty of millions in the populous north and northeast.
The protesters, who are still blocking major intersections in central Bangkok, want to suspend what they say is a fragile democracy under Thaksin's control and eradicate his influence by altering electoral arrangements.
Adding to the crisis, a flagship rice programme that paid farmers way above the market rate has proved ruinously expensive and the caretaker government lacks the power to keep funding it.
A state bank had to cancel a loan that might have helped prop up the scheme in the face of a revolt by depositors who began pulling their money out.
Three Government Savings Bank branches in Bangkok contacted by Reuters on Wednesday morning said they were no longer seeing unusual numbers of customers withdrawing funds.
Thailand's anti-corruption body began an investigation last month into the rice scheme and said on Tuesday it was filing charges against Yingluck. She was summoned to hear the charges on Feb. 27. (Writing by Alex Richardson; Editing by Alan Raybould and Ron Popeski)