Being in the same division since like always was a factor too, for many years you had to win the division to make the playoffs. For years after that the NFL’s post season format didn't favor divisional rematches...
still it's odd.
True, non-division winners have a hard time getting in the playoffs, and at some points in NFL history nearly no chance. But generally if the teams are good they find a way, my Steelers face a division opponent almost every year they get in the playoffs, heck we even got to stomp on the Browns back in 02.