One question might be whether this relic were the result of a later miracle rather than actually being the original shroud. With all the chaos that the early church underwent, it would have been easy for the original shroud to have been destroyed. It doesn’t take much exposure to the elements to destroy linen.
No, actually Linen is very tough material. It was not uncommon for linen bed clothes to last for generations. Often the linens were major portions of the inheritance of a family's wealth. This was when a single bed sheet could represent the hard work of several tradesmen, growers, spinners, fullers, weavers, dyers, bleachers. A cloth like the Shroud, over 14 feet long, woven on a wall loom, could take a skilled weaver several weeks worth of labor to create.