if you don't have the right to refuse business... then you're a slave.
whether or not i perform my work / services is entirely at my discretion. you can claim discrimination all you want, it doesn't matter. unless American business people are slaves forced to perform their tasks, it's entirely voluntary
While I agree with your central belief, there are circumstances where it becomes murky, at best.
For example, should a sports franchise be allowed to refuse entrance to a paying customer?