But he wasn’t comparing Darwin’s house to historical landmarks, he was comparing it to religious shrines.
Hardly an original comparison; such comparisons are made every day about the places I mentioned. Not only are such comparisons made, they are afar more apt. Washington is full of Grecian temples housing statues of culture heroes. A US President declared Gettysburg to be consecrated and hallowed ground. Traditional cult activity takes place at Graceland -- processions, offerings made to the dead, the sale of mementos, etc.
Why, then, is Darwin's house alone among these examples a literal religious shrine?
(Aside: I'm glad you noticed he was only comparing it to religious shrines. Think back to middle school English: What do you call a comparison of two unlike things?)