I used to live in the Annex and we'd go to By The Way regularly for brunch. I've never gone for other meals but brunch.
In the summer the patio is great. It's big and the sun shines right on it. In the winter, the restaurant gets very crowded and there are frequent waiting lines.
The food is pretty good. I really like the different eggs Benedict. They use an amazing dill sauce that just keeps you coming back.
Service is ok. Usually pretty friendly but it does take some doing to get their attention for requests and the bill.
I now go to By The Way when friends are in town or I simply want good egg Benedict.