Casa Loma was beautiful, and the best part about visiting is that it's included in the Toronto Pass! If you're taking public transit, be ready to be winded hiking up a crapton of steps like the low life peasant you are.
I could honestly have wandered around Loma for hours more than we did, but they were closing early for a wedding when we visited. They provide you with self guided tour boxes, which was super nice, but my friend and I didn't really dedicate ourselves to using them aside from like a couple of places that we found the need to.
The two towers are poorly setup for present day crowds, but obviously it wasn't built for that originally. The small spiral staircases we're a pain to come up and down because there was a flood of people trying to do the same thing at the same exact time. They really should've had a staff member at the bottom and top cutting of streams of people, to work more efficiently.
The Norman Tower is open-aired and has an amazing view, but don't waster your time on the Scottish Tower; you're just in another enclosed room which was a big let down for all the stress it took to actually get up there.