Does this remove any mapping to backspace?
:noremap <bs>
? No, it just opens a menu.
So, basically the issue was that backspace key is defined as delete key in my system?
Well, it's called ASCII delete key, but it's a backspace. A third possible id for backspace. Not sure why it works like this for you, I'd expect it to depend on terminal.