" It is not surprising that what we wear in the Presence of the Blessed Sacrament is different from what we wear in our own homes or swimming. This is simply having proper decorum; the Church does not call man to wear a suit and tie to bed or constantly in his own home. Nor would it be sensible for him to swim in such an outfit.
Charity, decorum, and Christian decency demand that man appears well dressed for the occasion. The difference here is not simply “cultural” or “situational,” but a call to be charitable in our decorum, decency in our actions and dress, and humility as Catholics. Note how Police Officers are dressed while on duty, or Nuns in their habits and priests in their collar (and cassock, for Traditional priests). "

― , Catholic Modesty: What It Is, What It Isn't, and Why It's Still Important