Google has been hard at work on updated media controls on Android 11, which not only moves the controls to the Quick Settings bar (as opposed to being displayed as a notification), but also adds an easily-accessible menu for switching audio outputs. The latter feature might arrive in Chrome soon, according to work being done in the open-source Chromium project.
Chrome already has a media control button that appears when audio or video is playing in any tab, allowing you to play or pause whatever is playing without switching tabs. It's a bit basic, but still better than having to hunt down which tab is playing something (or giving up and just muting your computer).
Current media controls in Chrome
Google is now adding a bar below the controls for switching the audio output, similar to the device picker in Android 11 Beta 2. The change is currently only available in the Dev and Canary channels (Chrome 86), and you have to enable a flag. If you paste chrome://flags/#global-media-controls-seamless-transfer in the address bar, flip the highlighted switch to 'Enabled,' and restart the browser, the audio device controls will appear.
Media controls in Chromium 86 build
The functionality isn't fully implemented yet — on a raw build of Chromium from today (v786829), a new section in the media controls appear, but there is only a single blank dropdown menu. Thankfully, the Chromium bug page for the feature includes a concept design for what the finished interface should look like. Audio devices will be listed as buttons, with a dropdown button on the bottom-right (presumably an overflow menu).
Concept screenshot for final design
It's not clear yet if you'll be able to set a different output device for each media player, but the fact that each notification has its own control bar seems to point in that direction. Although the flag is present in Chrome 86, it's too early to guess if it will be part of the final release, or if it will be pushed to a later version.
- Chrome Story