I have encountered this problem and it seems to coincide with my ISP providing sub-par service, with high latency and dropped packets, as measured by the tool at http://myspeed.visualware.com/

Voipo had me run a ping test that repeatedly pinged the Voipo server. When things were bad, the ping times bounced up and down and sometimes returned errors.