Stewie Speaks...

Okay, so I don't understand something. Who can hear and UNDERSTAND what Stewie from Family Guy says?!?!? I've been watching for the past year or so, catching all the reruns on TBS and Cartoon, but it seems like Brian, and some random people can understand him. What gives? What do they have in common that they can understand this child. It is confusing me!!! Make the pain stop...make it stop.