Raising one eyebrow conveys a lot of concepts, all of them unpleasant, including but not limited to contempt, sarcastic compassion and disapproval. The master of that is certainly my mother, that likes to do it without one word of comment. She just looks, raises, lowers, and changes the subject.
My SO has been eyebrow-raised a couple of time, and deeply resents it. I, being the son of my mother, have managed to develop:
  1. 1" thick skin, not pierced by mere eyebrow waggling.
  2. Ability to raise eyebrows (developed with a bit of exercise in front of a mirror).
I guess that your family environment really does influence you deeply.