"You again" is quite a familar expression, however does it implies a negative connotation?
You are correct. "You again" does have a negative connotation associated with it. This phrase implies that he or she has seen you earlier but is not happy that he sees you again. You can also hear it in the tone of her voice. Keep in mind, however, that it doesn't always have to be a negative phrase. The phrases "Good to see you again" or "Will I see you again?" do not negatively connotate. I recommend just paying attention to the entire sentence if you ever hear this phrase again. Ex: You again? What are you doing here? Ex: Oh, no. Not you again.