The Meaning of the Word
Unfortunately, very few of the people who have advocated for equality and diversity, at any time in history, have understood it broadly enough to encompass the entire human species. When the wealthy slaveowner Thomas Jefferson wrote that it was self-evident that all men were created equal, he was referring to white males who owned property. Back in those days, it was dangerous radicalism just to suggest that kings were not divinely appointed to rule over the commoners.
When the early feminists sought the vote, how did they persuade men to support their cause? If you thought they just gave eloquent speeches about the value of diversity in a flourishing democracy, you'd be wrong. Rather, some of them deliberately appealed to racial prejudice, arguing that white women's votes would help to ensure Aryan supremacy.
When the schools began to accept children of all races, students were taught that they should appreciate the diversity of different cultures. Then they took the bus home to segregated neighborhoods and eagerly watched TV shows about cowboys shooting "savage" Indians.
At present in the United States, one of the two major political parties often campaigns on a strategy of getting votes from bigots by stirring up prejudice against blacks, gays, Hispanics, Muslims, and anyone else they think won't vote in large enough numbers to make a difference. (Yes, I know there are decent and moral Republicans out there, and I respect you guys, but y'all seriously need to clean up your party and throw out the racists and the gay-bashers.)
The trouble with the word "diversity" is that there is no clearly defined consensus view of what it should mean. Many folks think of diversity as including all people who are just a little bit different from themselves; but as for those they see as significantly different, well, it doesn't even occur to them that the word might apply in that context.
No matter how much our society broadens its view of diversity, there's always farther to go. In the future, even if we reach a point where the humanity of every person is respected, there will still be more questions to address. For example: As we learn to communicate more effectively with animals, should we treat them as entitled to self-determination? If technology advances far enough to allow us to build self-aware androids, what civil rights will they have? Will we need anti-discrimination laws to protect cyborgs? If we encounter sentient aliens, how should we interact with them? Some day, we may have to deal with these issues in real life, rather than just entertaining ourselves with them in science fiction.
Bigotry rarely is as self-evident as one might expect.