Is Sad Music Actually Sad?

Be it Elliott Smith or Queen, classical or dub step, there's usually a clear understanding that some songs are sad, and some songs are happy. But what is it about the music that makes us feel these feelings we're feeling?