Caring about what you wear is just caring
Are some people who care about fashion small, shallow people, with questionable values? Probably. But I doubt they number more than the general average, and probably no more than any other focused group (sports fans, art collectors, Republicans).
The people I’ve met who take a lot of care in their personal appearance do so not to the detriment of other areas of their lives, but as evidence that they apply the same level of care to those areas. Care is care. This is not to say that an obsessiveness with style never indicates a narcissistic personality, but, again in my experience, this seems the exception.
A man who carefully coordinates his tie, belt, and socks, is, in my mind, more likely to care about standards of quality when building a car, designing a bridge, or drilling my teeth.