Does "walk into" mean "visit" here? Is it common to use like this?

Rebecca

"Walk into" here does literally refer to the moment you enter, or "walk into", someone's home. He is describing the feeling of smelling muffins baking when you enter someone's home, which is a nice smell and makes their home seem inviting.