Nearly every phone sold in the last few years has a 4G LTE radio, but when you place a call it's still falling back to traditional 3G technologies. The all-IP voice technology known as VoLTE (voice over LTE) is still in the early days, but AT&T is beginning its rollout this very month on May 23rd.
AT&T is referring to this feature "HD Voice," which is accurate, but a bit confusing. HD Voice can also refer to the non-LTE wideband calls deployed by both T-Mobile and Sprint. AT&T is the first to do HD voice with LTE, though. This standard will support simultaneous data and voice over the IP network, but both parties will need to be using a VoLTE-enabled phone and be in a supported service area. The Galaxy S4 Mini is the first supported device, but VoLTE will hit other phones later.
The initial test markets later this month will be in Illinois, Indiana, Minnesota, and Wisconsin. More locations will be added over time. You might not like voice calls any more, but at least they'll sound better soon.