Is it rude to say “off you go”?

Rebecca

No. "Off you go" is not considered rude. "Off you go" is a polite way to dismiss someone if you are planning to stay and he is planning to leave. It's just a polite way to ask someone to leave.