Depends on the era. There were a handful of times the southern kingdoms invaded. You can tell by the noses on the statues.
I was about to send similar information to wbarmy. Northern rulers periodically sailed/rowed south to keep the southerners in their place with warfare. However, with weak leaders or chaotic conditions the south sometimes took over. I guess floating north was easier.