I refuse to apologise on behalf of the phenotypical expression of my genetic material

Algerian man in restaurant: You look Algerian, actually.
Me: No no. I'm half southeast Asian.
AM: You're sure you aren't part Algerian?
Me: Yes.
AM, looking disbelievingly at me: Hm...I'll have to get you tell my fortune later.
Me: ?!

You could replace "Algerian" in that story with just about any nationality or ethnic group on Earth and I will have experienced it previously. Okay, you couldn't replace the part about the fortune-telling. Are mixed-race people renowned for their psychic powers? If so, how come no one told me earlier? I could have made my fortune as Madame Nanila by now. Curses.

Here are the groups to which people have not only insisted I must belong, but gotten upset with me to some degree for not belonging to them, over the years.

• Moroccan
  
• Thai
  
• Jewish
  
• Spanish
  
• Vietnamese
  
• Samoan
  
• Indonesian
  
• Indian
  
• Japanese
  
• Mexican
  
• Italian
  
• Cuban
  
• Unmixed white

I don't mind being difficult to pigeonhole. I do mind, however, when people get peeved because I don't fit into the preconceived identity box they made for me. How on Earth is that offensive?