It's a must to get the feel for Montreal. Suggested things to do (all within reasonable walking distance):
Norte dame basilica and square
Walk along river's north bank
Visit any of several museums
Strolling: lots of courtyards and monuments in the neighborhood that help fill in the city's history
Bypass as many horse carriages and souvenirs shops as possible. Unless that's your thing, they have really popped up a lot.