Ping Freezing on Servers?

Sometimes I join a server and after a few minutes of playing the ping freezes at a random number and the players freeze and are unresponsive. I had a friend with me in a server and they said the server was working fine when one of these freezes happened. Does anyone know what causes this and how to fix it? This happens in 9/10 of the servers I have joined.