I suppose any exception that you're gonna hide from the public should be logged.

If you have a web site which attempts to send an email and just tells users that it can't in the event of the catch half of a try/catch wrapping the attempt to send mail being used, you probably should still be doing something with the exception instead of just swallowing it. This is a time to log!