The racism intensified following his involvement in the public broadcaster's coverage of the King's coronation.

The veteran correspondent was invited to participate in an ABC TV discussion on coronation day, during which he pointed out that the crown represents the invasion and theft of Aboriginal land.

Barely a week goes by in which he and his family are not targeted by racial abuse, Mr Grant said in a column on the ABC News website on Friday, and he needs space from "the stench of the media".

Prime Minister Anthony Albanese said Stan Grant had his respect and he wished the journalist well.

Speaking to reporters in Japan, Mr Albanese said he was not aware of the details of Mr Grant's decision, but cautioned people need to be aware that hurtful comments have been made in the lead-up to the voice referendum.

"You only have to look at one of my social media feeds to see some of the comments that are, quite frankly, completely out of line," he said.

"We can have respect for different views without engaging in vilification and that's important."

Mr Grant has expressed frustration at the ABC's executives, saying not a single one has publicly refuted the lies that have been written and spoken about him.

"I don't hold any individual responsible; this is an institutional failure," he said.

While the ABC's news director Justin Stevens has been a "support and a comfort" according to Stan Grant, the national broadcaster has its own legacy of racism.

In a statement Mr Stevens said the broadcaster stands by Mr Grant and the abuse is "abhorrent and unacceptable".

Threats against Mr Grant have been referred to police, while a formal complaint has also been lodged with Twitter.

The coronation day panel in which Mr Grant participated was an early segment during eight hours of coverage, Mr Stevens said, and resulted in a strong response from some viewers.

The ABC Ombudsman will investigate editorial complaints regarding the panel.

"Responsibility for the coverage lies with ABC News management not Stan Grant, yet he has borne the brunt of a tirade of criticism particularly form sections of the media that target the ABC", Mr Stevens said.

"Reporting on his contribution to the panel discussion has been unfair, inaccurate and irresponsible."

It is not clear how long Mr Grant will step away from the Q&A hosting role for.

Mr Grant said he would not give racists the satisfaction of blaming racism for his decision to take time out.

Rather, he is leaving because the hard truths of Australia's Indigenous people are too big and fragile for an intensely politicised media to handle.

Mr Grant's latest book, The Queen is Dead, argues for a republic, a just settlement with Indigenous people, and an end to the monarchy in Australia.

Australian Associated Press