I believe environmental issues are the primary concern. They push the economics to make it unprofitable.
By the term "environmental issues," I meant the process of obtaining permits to build a new refinery. If the economics for a new refinery looked more favorable, then oil companies would work harder on obtaining the needed permits, which they are not doing.
As for the cost of environmental facilities required as a part of a new refinery design, that is an interesting question. One may argue that about 25% of the investment cost of a new refinery is associated with facilities required strictly for environmental compliance. Taking that assumption, then the cost of a new refinery without environment facilities might be $1.875 Billion.
Interest charges would be about $131 Million per year, or about $1.88 per barrel. At an average refinery margin of $2.35 per barrel, after paying the yearly interest charges to borrow the money to build it, it would take 57 years to recover the capital. Then the refinery owner could start pocketing a profit. Not a very attractive proposal.
The real problem with refinery profits in the US is the amount of spare refinery capacity built in places like Korea and the middle east, as a strategic investment for those countries. That government subsidized spare capacity competes directly with new grass roots capacity which might otherwise be built in the US. The only economic protection for the US capacity is the current high tanker rates for transporting refined products into the country.
One might argue that refineries built somewhere else in the world do not require so much environmental investment, but that "savings" is largely offset with the higher general construction costs.