Crossbench senators Jacqui Lambie, Rex Patrick and Stirling Griff referred leaked comments made by Mr Setka to the police after he warned they would "wear the consequences" for supporting a bill cracking down on unions.

An AFP spokesman said the referral was assessed in accordance with standard protocols.

"During this process, the AFP was unable to obtain sufficient evidence to warrant an investigation," the spokesman told AAP on Friday.

"The AFP considers this matter finalised."

In the leaked recording, the Victorian secretary of the construction union also said the crossbench senators could be expected to be abused on the street in 20 years if they supported the "ensuring integrity" bill.

"Someone is going to point the finger and say, 'There's them f***ers that voted for that bill that f***ed up not just construction workers but all workers in Australia'," Mr Setka said during a shop stewards meeting.

Mr Setka denied threatening the senators, saying MPs needed to "toughen up" and accept the comments as part of regular campaigning.

The comments were also referred to the Senate's powerful privileges committee to see if Mr Setka was in contempt of parliament by blocking the politicians from doing their job.

The bill at the centre of the dispute makes it easier to deregister unions, ban officials and puts a public interest test on mergers.

Senator Lambie, who holds a key vote in the upper house, has signalled she will support the bill unless Mr Setka quits his job at the CFMMEU.

A report on the draft bill is due to be released on Friday.

Australian Associated Press