So... he is trying to say that he won't fall for any nasty provocative?

Rebecca

Payton says this here just to indicate that he's not going to listen to the woman or let what she says get in the way of his conversation or goal. Jessica asks Payton if he knows what time it is to indicate that he shouldn't be there because it is very late and he responds in this way to show that he doesn't care about what she thinks or what she has to say.