I don't ride real fast to and from work, but I like to keep a good pace. I'm definitely not part of the slow bike movement. I bike fast enough to catch up to another biker every once in awhile on my commute, and I hesitate to pass them. I feel kind of bad zipping by. Maybe I've got a faster bike, maybe I'm in better shape and I don't want to show them up. So instead I'll usually slow down for a bit, then turn off on a side street and take an alternate route.
It's the bikers in regular clothes that I don't like to pass - if they look like they're on a training ride, I'll zip on by without any qualms and give a small wave or 'morning'.
It's funny, but once in awhile I get passed by someone from out of nowhere, and I don't like it. I'm tempted to keep up with them and engage in some CAT 6 racing. Or if I notice someone behind me, I'll kick it up another notch in an effort to not get passed.
The guy in the picture above - I slowed down and debated about passing, or turning off, or maybe even being sociable (me?) and make a little small talk. Eventually, I turned off at the next intersection. It seemed rude to pass going uphill.