They found when it comes to fish, females don't find males with big genitals any more attractive than those with normal or smaller genitals.
"Our findings show the size of male genitals has no effect on their attractiveness, success in reproduction, or their ability to swim and move around in the water," said researcher Professor Michael Jennions, from the ANU Research School of Biology.
The findings contradict two previous ANU studies, which found that larger penis size had a positive relationship with fish paternity success, and human research which found women rate men with a larger penis as more attractive.
Professor Jennions said the latest study, published in Nature Communications, would lead to a greater understanding of the evolution of genitals. Male genital size varies hugely among species.
The research involved studying male genital size of mosquitofish, Gambusia holbrooki, which has a penis-like structure known as a gonopodium. The normal male gonopodium is equal to about 30 per cent of the mosquitofish body length.
However, the team selectively bred male mosquitofish for eight generations to create some fish with larger gonopodia, and some with smaller gonopodia.
The males with different size genitals were then allowed to freely compete to mate with females. The researchers then used DNA paternity testing to see which males were more successful at fathering offspring.
"To our surprise, we found the size of the gonopodia made no difference to which fish successfully became fathers," said Dr Megan Head also from the ANU Research School of Biology.
The research used a sample size of 173 males and 165 females and paternity tested over 2,250 offspring.
Mosquitofish have live offspring rather than lay eggs. They are considered a feral pest in Australia after they were introduced in the 1920s in a failed attempt to control mosquito populations.
The research has been published in the Nature Communications journal.