As a pilot I would much rather have a thought driven system do things like load an approach, change a freq, pull up a checklist. The distractors and hazards to aviation aren’t associated with proper manipulation of the controls as much as the things that take your attention away from manipulating the controls. Getting stuck in a finger fire screwing up loading an approach or selecting a freq is way more dangerous. It would be a lot easier to fly with your hands and think “load the RANGER 3 Arrival.” It is also a lot easier to tactily feel performance in the flight controls than wonder if you are thinking correctly.
The old “pilot workload” argument always seemed to be debated the loudest by those with the least skin in the fight. Designers (limited by current technology) and pencil necks (whipping the budgets) against those who actually flew the end product. One of the best all around discussions and reviews I was part of was the joint AF and MDC design teams for the cockpits of the C17. It just seemed that the level of cooperation was better balanced (healthier?) for the scope of the work involved.