It is part of a national trend where the fertility rate has declined in Australia since the global financial crisis, with Victoria last year replacing the Australian Capital Territory as the state or territory with the lowest rate, according to Bureau of Statistics data.
In Victoria, the fertility rate is 1.76 babies per woman - well below the replacement rate of 2.1 babies per woman, the level where the population sustains itself. But despite the birth rate falling to its lowest level since 2005, Victoria's overall population is growing strongly due to migration.
"Some people like to suggest it was due [to the Howard government's baby bonus]; the reality was the upturn in fertility started before the baby bonus was introduced," he said.
One factor, he said, was the booming economy at the time, which gave young people confidence to start a family rather than wait.
That trend is evidenced in Victoria where towns such as Robinvale and Yarrawonga have some of the highest birth rates. Within Melbourne, suburbs such as Pakenham and Truganina have fertility rates well above replacement levels. But in the inner suburbs of Melbourne, the fertility rate is on average just 1.3 babies per woman - lower even than Japan, with its shrinking population. It is particularly low in Parkville, St Kilda, East Brunswick, Docklands and East Melbourne.
Mothers also tend to be older in the inner suburbs, with a median age of 33.6 years old compared with 31.6 for the state as a whole.
Ellen Kwek, 34, an architect from North Carlton, has just had her second child, a daughter named Zoe, with a gap of four years from her son.
"Because I had my first child when I was about 30, I didn't feel in any particular hurry to have the second one so soon after ... this is about when it seemed right for us," she said.
Ms Kwek said along with her partner Mick she had always thought of having a maximum of two children. Her career was a consideration as she wanted to avoid taking "another chunk" out of it. If they were to consider a third child, housing costs and size would be an issue.
"I don't think we could imagine having another child in the same size house we have," she said.
Professor Hugo said despite Australia's fertility rate declining to 1.88 births per woman, it is still at a level envied by countries such as Singapore, Japan or much of Europe. Fertility rates of 1.3 or 1.4 babies per woman create economic problems from declining populations and how to fund large numbers of retirees.
"What it means is that ageing, although we talk about it as a significant factor, it is not nearly as dramatic [an] impact as it has been in places like Japan, or in a place like China, where they'll suffer some really significant problems," he said.