Finns are actually related closer to Estonians and Hungarians ( Magyar )than the other Scandanavians. The language base all three share is Finno-Ugric.
Finns are hard to pigeonhole.
Linguistically, they _are_close to Estonians and Hungarians, but, at least as indicated by Y-Chromosome haplogroup frequency, they are genetically closest to the Saami, Lithuanians and Estonians. They are fairly close to Swedes and Norwegians, but closer to Indian Uttar Pradesh Rajputs than Hungarians.
Look here.
They are also closely related to the Mari,a tribe of the original people from what is now called the "Mari Republic" in Russia.
Finnish & Hungarian are indeed part of the same languistic family, but I once read that the two are about as similar as English and Iranian, which are both Indo-European languages. Finnish & Estonian are indeed very close. Also, Finland was rather heavily influenced by Sweden for centuries, and its elite was partly Swedish, I think. For example, the name of Finnish composer Jan Sibelius sounds Swedish, not Finnish.