Though your ISP may tell you that you have a huge amount of upload and download bandwidth, the Mbps speeds they say you have are usually not sustained speeds, and are only good from your internal network to their local network and back. Once your data leaves their local network, speeds can vary and even drop by quite a bit.
Your data travels though multiple network carriers to get form you to us. And high network traffic or network maintenance taking place somewhere in that path can cause a delay in your data reaching us. Which in turn can cause buffering in your stream. The same type of issue can happen in the network path between us and a listener's player.