That physical screening test you mention is not difficult, any decent swimmer can go over twice that speed without any real difficulty. I’m not surprised that the actual training program is far more demanding. I would be concerned if that were NOT the case actually.
See 47, and let me add to your comment. The BUD/S screening physical is hard but not too hard, deliberately. A lot of sailors are coming from a destroyer or sub etc. To start them off on day one at a third-month pace (in any activity, running, swimming etc) would mean that the sailors who just spent months in shore duty working out all day would have way too much of an advantage, but really proving nothing but that they started in better shape. The run/swim/o-course times decrease weekly, while the calisthenic reps increase. It’s more mental than physical, if by physical you mean run/swim times. It’s the heart and the mindset that cound much more. And not having an phobias or weaknesses. They must be well rounded. A olympian stud with an achilles heel will NOT make it.