I did the same thing when I was their age. (Of course, we didn't have instruments and never did.)
My son saw "That Thing You Do" (with Tom Hanks...no flames please) when he was about 4 and started drumming on upside down pots and cans..... I bought him a "starter" drum set and I was very impressed with what he could do.
This year we bought him a full set and he is amazing - I'm getting private lessons set up for him this month! When the teenage boys next door piss me off, I open my son's windows and let him play all he wants....... !