Hello.
First of all I have not worked in Australia so I am not common to their laws. I suggest you investigate in the law regarding restraints and wheelchairs. At this point you have to assume the people you have asked are right: In fact breaks are a restraint. To use an example of the USA, a country where everyone is suing everyone is not helpful, neither to use the UK as an example (where health and safety has gone crazy, likely because it follows the USA as if it is part of it).
This is not a case of common sense but of law(s). If you have not seen on the television any adverts from lawyers about blame and claim (as is common in the USA and UK) you might understand that freedom is higher regarded in Australia than taking risks which comes with an increase of freedom.
Makes sense?