This place is rated highly and I can see why. The only detractors were the service and maybe the price.
Service: I'm going to go straight to service because although the young lady was nice initially, her attitude changed completely after seeing a standard tip. Maybe she was expecting a handsome tip with the level of explanation she had about the oysters but her body language and attitude was a 180 when she saw a regular 15% tip. It could just be my philosophy but I tend to tip 15% before tax - which is about 13% after tax. Her explanation was also quite bland and it was obvious she didn't actually know very much about the oysters they were serving.
Food: good - fresh oysters with lots of variety. The mains and appetizers were not nearly as extravagant as many of the reviews here would allude to. They were OK but not anything to write home about.
Venue: good - ambience is 5/10 in terms of how loud it was. Lighting was adequate but on the dark side. I saw lots of dates happening on a Saturday night. Parking is easy on the street and costs about $3/h which is pretty good for DT Toronto.
Price: average - about $3-5/oyster on a regular night which is to be expected. Other items were average priced.
I will be back but I'm in no rush. There are tons of other oyster joints in Toronto that I'd check out before having a preferred place. So far, Pearl Diver would rank somewhere in the middle of the pack.