A potential push to integrate social housing within the city has residents calling for more department support during tough times.
According to NSW BOCSAR data, Boorooma was a suburb where crime spiked with 32 incidents being reported in 2019 compared to 13 the previous year.
Darcy Drive resident Jordan Stewart said an Aboriginal Housing property down the street has caused a lot of neighbourhood trouble in the last 12 months.
"My wife and I have a little girl and we're expecting another child and we've lived in Boorooma for about two years," Mr Stewart said.
"The family living in the house at the bottom of Darcy Drive has brought a lot of trouble and there is always police driving around."
Mr Stewart said the safe reputation, which has attracted families, is being threatened.
He said many residents would support the integration of social housing in established areas if the NSW Department of Family and Community Services listened to their concerns.
"We've got surveillance on our house and our neighbours have just put that in, but as soon as you want to do that, it admits you don't feel safe in your own house," he said.
"People don't have an issue with community housing, but if a family moves in giving people grief, they're given so many chances and the community isn't being listened to.
"People work hard to afford to live here; I have two jobs to pay our mortgage.
"It seems like it doesn't matter how many times we email or call the police, it's like nothing can be done about it," he said.
Mr Stewart said he wants to be able to have a conversation about this issue, without it becoming too sensitive or be shut down.
"I want to make it OK for people to talk about this issue ...a lot of people want to know who we can talk to when issues arise," he said.
"My grandparents are in housing commission and they're lovely people."
The Daily Advertiser contacted the NSW Department of FACS, but they declined to comment.