Toward the end of the novel, Javert chose death by suicide as preferable to having to face any more of Valjean's abominations. The sewage swim was probably the straw that broke the camel's back and represented the death of civilization. What was there left for Javert to live for?
He was also someone who recognized the depth of his own sin, his need for forgiveness and grace. The movie does not sufficiently record, as does the book, the ongoing dilemma of being a fallen human being, prone to sin, regardless of his repeated efforts to establish a new and reputable life.
Javert chose death by suicide as preferable to having to face any more of Valjean's abominations.
Javert is like the older brother in Jesus' parable of the prodigal son, angry that his reprobate younger brother is welcomed back into the family by his father. Javert knew only the justice of God, and little of his mercy. He could not understand how Valjean, the one you describe as "a pervert, a thief and a guy who swam in raw sewage," could prosper under a just God. But Valjean knew who he was and despite repeated failures, at heart he humbly accepted Christ's sacrifice on the Cross as the payment for his own sin.
Swimming in raw sewage is an apt metaphor for humanity. As the apostle Paul explained, even the best of what we do is like "filthy rags," and of his own not inconsiderable accomplishments as one who faithfully served The Law, he considered it all "dung" (Phil 3:8).
Javert could not bring himself to accept that same grace, because he saw himself as righteous through the performance of his duties under the law. And when he did recognize his failure on this account, he could only imagine death and shame as the only outcome. That is the mindset of the Pharisee. It is extraordinarily bleak and it is a only a pale, bloodless imitation of God's true justice. It may well be, as you put it "the social glue which provided whatever stability existed in French society," but it is not a glue that will hold for eternity.
The Law brings death and the Spirit brings life. (2 Cor 3:6)The same grace granted to Valjean was available to Javert (as it is available to all of us), and had he accepted that, he would have most certainly had much to live for.