It's important for everyone to also note their broadband connection speed here as well. Someone on DSL will have more issues than someone running on 6+ mbps.
I agree that download speed is a significant factor. In fact, my guess is that it is the single most important factor. Other factors are probably CDNs, router type, modem type, wired vs wireless, distance from router if wireless, other active devices on the LAN, and any number of other small factors. I definitely don't discount or doubt the experiences of others. If someone says they are having a particular issue, I imagine they are. However, I am hesitant to just chalk it up as a Hulu problem. With my 25 Mbps service, I am not seeing these issues on my PS3s, Rokus or iPad. If it was just a Hulu problem, I would think it would show up fairly uniformly across all devices. The most difficult thing in this new streaming environment is the infinite number of factors that can contribute to the end user experience.