Have you ever heard of an "Adhesion Contract?" That is where one person has all the power in a bargaining position and the other party is free to pay what is demanded or walk away. Such contracts are controlled and rightly so.
A person would certainly be harmed if the drug company used its bargaining power viv-a-vis the consumer. The flaw in the Libertarian argument is their failure to recognize there is a vast difference in bargaining power among various persons, firms or entities. Imagine you personally going to a giant drug company and trying to bargain their price down for a drug that would save your five year olds life.
Why don't YOU imagine being told that EVERY 5 year old with this affliction will die because the drug that could save them was never developed. There was some promising research but the investment was simply too high to proceed.
See how that works?