The patent holder only controls the seed produced pursuant to his patent. The fact that the marketplace recognizes some patented seeeds to be markedly superior to seeds previously in commercial use testifies to the value of free markets.
Constitutionally patents are on very weak legs when applied to plants [IMO], as it says To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries; but plants aren't 'inventions' nor are these modifications 'discoveries' -- add to that the 'perpetual copyright' the Supreme Court basically handed Disney means that these GMO patents may realistically never expire.