They don't, actually. They're basically leaves floating in the stream of atmospheric winds, with the added "Beta Effect" of Coriolois which, all things being equal, continuously pushes them north.
Your statement that 'they don't' is counter to both my education and experience. It won't necessarily drive them long term, but it can have significant impact on short term direction especially when differential water/temperature profiles exist.