Google has been updating Hangouts a lot lately in an attempt to make it less terrible. It's definitely improving, but still has a way to go. With all the updating we've missed a few little changes along the way, and this is a particularly useful one—Hangouts will switch between the speaker and earpiece for voicemail depending on whether or not you've got the phone to your ear.
This appears to have been added in v4.0 as the v3.3 builds Google was on before didn't have the feature—that version played via the earpiece only. Now you can just start a voicemail and place the phone to your ear to listen via the earpiece (based on the proximity sensor). Pull it away, and the sound switches to the speaker. You can even go back and forth during a really long message (ugh).
I tested this on a few phones and it seems to work without issue. Some of you have probably noticed this feature before, but just so we're all on the same page, this is pretty cool. Anything to make listening to voicemail more tolerable.