My aunt (an immigrant who moved to England as a child and has consequently completely lost her Irish accent) occasionally enjoys abruptly terminating the "those immigrants" conversations by emphatically reminding the speaker she's one. I'll have to ask her if she's had much of the "Oh but I don't think of you as an immigrant" response.