Confrontational language used by David Cameron and other Government ministers about the need for British Muslims to do more to combat extremism may have hampered efforts to win support within the communities, a report by a respected defence think-tank has claimed.

The Royal United Services Institute (Rusi) report on tackling Islamic State (IS) said "emotive" comments had been "counterproductive" to outreach work with British Muslims.

The report also warned that the military effort in the Middle East against IS may result in a stalemate of "open-ended containment" until the situation in Syria becomes more stable, rather than the US-led coalition's aim of decisively wiping out the jihadist group.

The study highlighted the high cost of the RAF's operations in Iraq, estimating that £60,000 of taxpayers' money was spent for every IS target destroyed - the majority of which are relatively cheap armed pick-up trucks or machine gun positions.

The forthcoming Strategic Defence and Security Review (SDSR) would have to consider the impact of the operations on the RAF, including the surveillance and intelligence aircraft and crew as well as the Tornado jets and Reaper drones carrying out the air strikes.

The Prime Minister hit out at British Muslims who "quietly condone" IS even if they themselves were non-violent in a speech in June and in the wake of the Tunisian atrocity said that Muslims in the UK "need to act" if relatives are seeing extremist preachers or visiting radical websites.

Rusi's director of international security studies Raffaello Pantucci noted that the authorities had reached out to Muslim communities to convince them of the dangers of young men and women travelling to Syria and Iraq, while police and councils were also involved in efforts to counter radicalisation.

But, he added: "Unfortunately, this outreach has been undermined by the use of emotive language to characterise the threat as well as counterproductive accusations that communities are failing to respond to a threat within."

There were elements within the UK's Muslim communities that characterised the British Government as "anti-Muslim" in a "deliberately provocative" way.

"However, these individuals are a minority who speak for few. Ultimately, many in these communities enjoy living in the UK's free and multicultural society and do not want their fellow citizens to die alongside Isis.

"Garnering their support is something that will only be possible in an environment where the government uses less confrontational language."

The far right had taken advantage of the situation to construct a narrative of "Muslims versus others", he added, and "for Government officials also to talk publicly in such terms only appears to breathe life into these narratives".

As well as the concerns about the domestic battle against IS-inspired extremists, the Rusi report examined the military campaign to take on the jihadi group, also known as Isis, in Iraq and Syria.

Rusi's Elizabeth Quintana said that although the strategy pursued by the UK, US and allies was based on "the destruction of Isis as a military force", it may be "more realistic to assume that the strategy will become one of open-ended containment, limited to the degrading of ISIS in Iraq and Syria in order to limit its spread in and beyond the region".

She continued: "In this case, 'success' may be the reduction of the ISIS phenomenon to something that regional players and Western powers can tolerate until the Syrian civil war is resolved."

Research fellow Justin Bronk said that the US, UK and the other coalition nations needed to ensure their presence in the region could be sustained for "many years".

That could include investment in the RAF, particularly spy planes such as the E-3D Sentry airborne warning and control system (Awacs).

"In the UK, this must be taken into consideration during the forthcoming SDSR process when it comes to fast-jet numbers, ISR (intelligence, surveillance and reconnaissance) enablers and upgrades to key platforms such as the E-3D Awacs, which have been over-tasked and underfunded during more than a decade of continuous operations."

The Government insisted the UK was playing a "full role" in the fight against IS - also known as Isil - both militarily and by combating its "poisonous ideology".

A spokesman said: "The UK is playing a leading role in Global Coalition action against Isil, which has seen the terrorists lose 30% of the territory it once held in Iraq.

"We have a comprehensive strategy, tackling Isil militarily, politically and countering its poisonous ideology.

"Militarily, we are focusing our contribution where we can have the most impact - airstrikes in support of Iraqi forces, providing vital intelligence, surveillance and air-to-air refuelling capabilities to the coalition, and in the training of Iraqi security and moderate Syrian opposition forces in the areas they have requested and need - such as countering improvised explosive devices and highly valued weapons' maintenance.

"We are also tackling Isil's finances, the flow of foreign fighters and working to make areas safe for Iraqis to return home to. With around 800 personnel in the region and 150 personnel in headquarter and training roles, and a new Coalition Strategy Communications Cell here which is funding £10 million to help fight the battle of hearts and minds amongst the young Muslims around the world, the UK is playing a full role in this fight."