I purchased an annual pass for the AGO a couple of years ago, and while that pass was active, I just loved dropping by to revisit particular sections, and eventually familiarized myself with every single floor's floor plan.
They have a pretty expansive European collection (although not too many sculptures), as well as two floors dedicated to contemporary and modern art. The poodle art up on the fifth floor is... interesting to say the least. They also have an amazing collection of ship models in the basement.
Nowadays, I usually visit when there is an exhibition that I'm interested in, and after the exhibits I can still steal away for an amble in favourite sections.
I do hope they have a Matisse exhibit sometime in the future!