Agreed and Benedict XVI certainly has a refined aesthetic appreciation of music, art and literature, so it makes sense that it would extend to fabrics. It's just that as a priest his wardrobe is fairly limited and somewhat specialized, not to mention that clerical garb doesn't change much. Knowing the best tailors of ecclesiastical "threads" in Rome is not quite the same as knowing the latest trends or current "in" designers in fashion.
The better quality priests' clerical garb and liturgical vestments are usually made from wool, silk, linen or cotton. As you probably know, Italy also makes some of the finest textiles in the world, including an amazing spectrum of wool from the lightest summer weight wool to a thick cashmere used in jackets. That is pretty cool about the cloth you made being crafted into a jacket for your wife. Bravo!
My guess is a lot of his preference had to do with people who knew how to make clothing for someone of his size.