Is there much of a speed difference between the two?
Not until you get Grand Central Dispatch aware apps that can parallel process on multiple cores... then watch out. Things you want to do will be done before you start!
The AMD Radeon 5850 in the 27" iMac is faster than the fastest graphics card for the Mac Pro, the last generation Radeon HD 5870. Frankly, the selection of graphics cards for the MacPro sucks.
The i7 version of the 27" iMac is a very nice system. Frankly it is cheaper than any PC with similar specs. I had to go to a Cyberpower to match the price and even then, the monitor wasn't as a good (its a ViewSonic VA2626WM Flat 26" (25.5" Viewable) TFT Active Matrix XGA LCD Display 1920x1200).