Augustine was writing in Rome's late imperial and multicultural phase. Rome's multiculturalism would have been something close to a given. It's less clear what to say about North Africa, where Augustine came from. It was more provincial than Rome, but it's difficult for a layman to say what life was like there.
The universal church that Augustine championed was not identical with the empire or with multiculturalism. No more than those in the European nations who for centuries acknowledged a universal church supported multiculturalism or a world state. While proclaiming the union of all believers, Augustine would have taken a dim view of empire itself, since its demands for universality could conflict with those of the church. But that idea of the church did grow out of the example of the Roman empire and its aspirations to universality. To my way of thinking it's a tough call. There is something in the circumstances of Augustine's time which acts to subvert the point that Williamson wants to make. But in the end it's more a partial than a complete subversion.
Look at the 17 centuries of Western History since Augustine and you behold a creative tension between the universal and the national or particular. Our multiculturalism is to some degree an return to the universalism of the Roman ecumene or community. But it's also a blow to that creative tension.
Whether it's wildly wrong, or imaginatively spot on, Williamson's article does get one to think. Liberalism has been seen as Christianity minus Christ or God. I don't know if that's true or fair, but it does seem as though the Christian vision and secular multiculturalism are at least potentially at odds. They share the universalist ideal. But political or economic universalism may undercut or negate religious universalism, because it provides a different basis for unity. Ultimately, political and economic institutions will displace religion as the creators and maintainers of the ideal of unity and community. A religious community, however universal it aspire to be, will never embrace as much of the physical and demographic world as the One World of politics or economics, though it may embrace much more of the metaphysical and spiritual world.
But does believing in the spiritual equality and community of all people mean inviting them all here? A 4th or 5th Century Roman might have had to deal with what we are trying to cope with. A 4th or 5th century North African would have had fewer problems of that sort than his Roman counterpart, or than us.
Here is one more thought: in addition to the universalisms you mention--religious, on the one hand, and economic and political on the other, there is also moral universalism--the idea that we owe duties to all persons (even slaves and foreigners, according to Epictetus, and even to our enemies, according to Jesus).
This notion was already present in the teachings of the Stoics long before Augustine, and is a part of the Christian notion of agape or universal love.
Universalism was, indeed, a part of classical liberalism, but seems to have died out in current liberalism, which sees itself as a loosely associated coalition of Blacks, gays, feminists, Chicanos, et al., who share mutual interests but who owe duties only to others within their own subgroup.
Contemporary liberalism has in this way betrayed the universalistic ideal.
This ideal is also woefully absent in much of contemporary conservatism (e.g., in most of the posts on this site).