But the government says the comments are divisive and dishonest.

In his budget reply speech on Thursday, the opposition leader queried the decision to allow 1.5 million people to come into Australia over five years, which he described as the highest intake in the nation's history.

Speaking from Jerrabomberra in NSW on Friday, Mr Dutton said the nation was built on the success of its migrants, but in a housing crisis it had to be managed.

"We've got to recognise that there are a lot of Australians who are really doing it tough," he said.

"People can't find rental accommodation now, and they can't buy a house ... you're bringing 6000 people a week extra in, I just don't understand where those people are going to live."

Treasurer Jim Chalmers said the former coalition government was projecting an even higher migration level before it left office and less investment in housing than that proposed by Labor.

"They're not a lever that the government is pulling - they simply recognise that the students and long term tourists are coming back quicker than anticipated, and fewer Australians are leaving Australia to go and work overseas," Dr Chalmers said in Brisbane on Friday.

"The opposition has in the past said they want the students and tourists to get back quicker, so there's an inconsistency there."

Dr Chalmers said the coalition and the Greens were blocking a $10 billion Housing Australia Future Fund which would provide 30,000 social and affordable homes.

Also as part of his budget reply, Mr Dutton called for JobSeeker recipients to be able to work up to 10 hours a fortnight while receiving the unemployment payment.

"In our country, we've got 438,000 job vacancies and over 800,000 people on unemployment benefits, so we want to get them off welfare and into work," he said.

"It would give them a lot of extra money in their pocket compared to the $40 that the government is offering."

Mr Dutton declined to commit to supporting Labor's proposed $2.85 a day lift in JobSeeker payments.

Prime Minister Anthony Albanese said Mr Dutton had failed to back extra JobSeeker support when he was a cabinet minister.

"This is a budget aimed squarely at assisting vulnerable people through cost of living relief, but also aimed at assisting people in middle Australia," Mr Albanese said in Sydney.

While the opposition backed some budget measures such as tripling the bulk billing incentive and expanding single-parent payments, Mr Dutton said families had received little support.

"The budget hurts working Australians. Worse, it risks creating a generation of working poor Australians," he said.

Dr Chalmers described the opposition leader's budget reply as "divisive, dishonest and backward-looking".

"Peter Dutton thinks you can spend more, tax less and get bigger surpluses - that's the kind of dishonesty at the core of his budget reply," he said.

Responding to the opposition leader's call for small-scale nuclear power, Dr Chalmers said Mr Dutton needed to say which suburbs would host the plants and how they would stack up economically.

Australian Associated Press