They said the thought the house was abandoned. Then why did they knock on the door?
They did not actually think it was abandoned. They went on a fishing expedition and they apologized only because they could not find anything or were unable to plant anything.