This was a nice stop for us. It was a little out of the city which resulted in the most expensive Lyft ride of our trip but it was totally worth it.
First off, it's pretty crazy that a guy a little over a hundred years ago built this castle-like thing. The structure itself and everything in it was amazing. The back-story was less than exciting but the things were interesting to see anyway.
The audio tour was nice but we ended up cutting a lot of them short and only once or twice listening to the "additional" information they offered at the end of the main narration.
Although old, a lot of the design and appointments still seemed decadent and interesting. My favorite was the guest room that had its own bathroom AND sitting room. My house guests are lucky if they don't have to share the bathroom with my cat!
Although it takes some literal legwork, I definitely recommend the climb up to the tower. You're rewarded with a great view of the city. It's no CN Towers view but it gives you a different, more calming perspective.
We mostly enjoyed Casa Loma for the daydreaming it allowed us to do from their beautiful back patio. We imagined living in something like this, laughed at ourselves and went on our merry way.