While gold and silver do not attract GST (as long as it is of the right purity and from a recognised manufacturer like ABC Bullion) they are treated by the ATO like any other investment asset.
As a result, your purchases and sales of precious metal will have tax consequences and you should seek professional advice on how to report any gains or losses on your income tax return.