From the wording of the question, it is the change of domains, not the change of clothes. Shaatnez would indicate that you would have to remove your garment immediately, not necessarily leave the house. I still maintain my original answer from a pasuk in Nasso, that a Zav/Tzarua has to leave the Machane.
