Given that trolls were made from stone by Morgoth, in mockery of Ents, it seems unlikely that the Olog-hai would be "twisted" Ents or Entwives. Were this possible, Morgoth would have done so already, just as he had done to elves to produce orcs.
Rather, it seems to me that if Trolls are "awakened stone" in the same manner as Ents are awakened trees, and if Trolls in this respect are similar to the stone watchers in Mordor (as described in ROTK), that is to say, stone animated by a spirit of some kind such that it takes on fleshly form, then it seems to me unnecessary to posit any relation to Ents at all, except for the Trolls being a kind of mockery of Ents on Morgoth's part.
Now, as to Olog-hai: what distinguishes them from Stone-Trolls is their ability to endure sunlight, in similar manner that Uruk-hai and Orc-Human halfbreeds can endure sunlight. It seems more likely to me that the Olog-hai are simply the product of a breeding program, together with magical spells, which counteract or negate the fatal weakness of the original Trolls: the fact that they turn into stone when exposed to sunlight.
It seems to me that the Olog-hai are perhaps Trolls interbred with orcs and/or humans, such that they retain the size, ferocity, and strength of trolls, but also are smarter and no longer turn into stone in the sunlight. Perhaps some kind of magical spells are also required to negate the trollish weakness in sunlight, but I see no need to posit any Entish connection.
A twisted Ent would still be tree-ish, not troll-ish. And you could not crossbreed trees with trolls or other animals; you could crossbreed trolls with orcs or humans, because, although in origin of stone, they take on the form of humanoid animal life, not the vegetable form of tree life.