Avram > Absolutely, whole communites do and should operate on this premise! I am not so haughty that I can say that I do all my mitzvos lishma. Only our greatest tzadikim reached that level. Why stop at clothes? Why not do away with anything that helps us overcome the yetzer hara?
That is a good question. I don’t know where the border is. Quoting a charedi Rav: if you do halakha, you can rely on Sh’A to give you an answer. The danger of doing chumros – there is no S’A to rely on, and ein l’davar sof.
But, as I said before – I am not necessarily telling people to change their behaviors, I am just suggesting to be respectful to those who do not follow your ways. Maybe here is the issue: If you are drinking coffee in beis midrash, you know you are doing a kullah, you are not looking down at those who are not drinking. [Once a rav stopped drinking in beis midrash when the class was finished. The student suggested learning another blatt to finish the drink. The Rav responded “I am drinking to learn, not learning to drink ].
BUT When you accept a chumrah/neder to protect yourself from yetzer hara, yetzer hara turns around and makes you feel that your chumrah makes you holier – and even others less so. I-forgot-who (R Yishmael?) says in Nedarim that he only once ate from a korban of a nazir who was really sincere….
So, when a whole community takes this upon themselves, the result could be looking down at others.