Think I found the problem, Carbonite.

My ping time went from the times in my earlier post to 500+ ms again, noticed heavy outbound traffic in the router and disabled Carbonite for 10 min, ping times dropped immediately.

I'm going to work on QOS and prioritizing traffic in the morning to see if either choking Carbonite back or giving the Voipo ports higher priority works best.

Might be worth adding Carbonite / Mozy to your troubleshooting list.