I think the offense was that as everyone knows, monkeys and apes like bananas, and, of course, as everyone who is a person of color also knows, negroes are perceived as just semi-evolved monkeys and apes, hence the offense.
we were recently on vacation.
I was behind a tour group of black people. they each had a banana. I guess they did not get the memo.
( I grinned from ear to ear as i passed them)