There's a subtle change to the Android voice input system, and you probably already have it on your phone without realizing. Some time ago, users started reporting that a delete button had appeared on the dictation keyboard, and now we're seeing it too. The most likely cause is an update to the Google app, which contains the voice settings.
It appears this behavior is triggered by the v4.8 update to the Google app. That version started rolling out in late June. It takes time for these updates to reach everyone (most don't sideload APKs), and it's not a major change. That's probably why we just started hearing about it more recently.
The delete button lets you remove entire words (not individual letters) when using voice input. This saves you from going back to the regular keyboard just to correct something. We've seen this on a variety of devices, so it shouldn't matter what you're using.
- Sir Plew