i think the doorman in my building fears/hates me

As a foreigner living in a neighborhood with almost no other expats (there are certain neighborhoods here that are heavily populated by foreigners), people in my neighborhood often have no idea what to do with me. One of those people is the doorman in my building. Granted, we did not get off to a great start as I first met him when I somehow managed to drop my keys down the elevator shaft in the middle of the night, thus rendering me unable to enter my apartment. He went down to the basement and retrieved them, and then we got to talking. After hearing me speak for a minute he asked where exactly I was from (my accent is a bizarre mix of Peruvian, Spanish, Colombian, and Argentine Spanish). I told him that I was from the US, he responded with an "ahhhhh" and a head nod, and we both went our separate ways.

Over the next few weeks we exchanged pleasantries as I walked in or out of the building. But sometimes (when he was feeling extra friendly?) he would shout random words in English at me as I left the building. A few weeks ago I was heading out to work, and I said something like "have a nice day" to him. He replied by shouting "WAAAAAATERRRRR" at me as i headed out the door. This is not the first time this has happened to me. Sometimes if people know that I speak English, they shout random English words at me. I think they do it just to let me know what they know, kind of to show off their knowledge.

Anyways, today I was sitting on the subway, heading back from work, when the doorman got on the train. He walked by me, we said hi, he moved to a different part of the car. We both knew that we were heading back to the same building (a 3 block walk from the subway station). Would we walk together? When the train stopped at our station the doorman bolted out. Walking up the stairs to exit the station he looked back once to check that I was still behind him, then continued to rush on, even running through a bit of light traffic to get ahead of me. When I got back to the building he was here. He greeted me with a "HEEEEY! I saw you in the subway today". I wanted to reply "Yea, I saw you too, I also saw the back of your head as you ran away from me!", but I didn't. I just smiled and moved along. This man hates me or is deeply afraid of conversation with me. I think tomorrow when I get home from work I might just stand by the door with him and be like "Sooooo whats up with you lately?". See what happens...