There may be some problems with using both emacs and vi on some terminals. A few terminals have no escape key (ESC) so you may need to type Ctrl-[ to get ESC.
If software flow control exists, then the ^S command in emacs will freeze the display. The ^Q command will unfreeze the display. One fix is to map this to another key-press by configuring emacs that way. Some terminals have meta keys although you may need to setup the terminal to create a meta key.
vi and Cursor-Keys
Vi uses the esc-key as a command to exit insert mode. Unfortunately for most terminals the arrow-keys send an an escape sequence (starting with the ESC character) to the host. Vi must distinguish between these two meanings of ESC. A smart vi (such as vim if configured for it) is able to detect the difference by noting the time between the ESC and the next key. If it's a short time, then it's likely that a cursor key was pressed. Use "help cursor-keys" in vim to find out more.
Here's another way to fix this. On VT terminals the left-arrow-key may be either set to send ESC [ D or ESC O D. The other arrow keys are similar but use A, B, and C instead of D. If you're having problems, choose ESC [ D since the "O" in the other alternative could be interpreted by vi as a command to "Open a line". The "[" should be interpreted by vi to mean that an arrow-key has been pressed. ESC [ D will be sent provided "Cursor Key Application Mode" has not been set. ESC [ D is normally the default so everything is seemingly OK. Except that many termcaps contain a string (not the init string) which sets what you want to avoid: "Application Mode". Editors may send this string to the terminal when the editor starts up. Now you are in trouble.
This string has the termcap code "ks" (smkx in terminfo) meaning enable the function (and related) keys (including the arrow keys). An application enables these keys by sending the "ks" string to the terminal. Whoever wrote the termcap reasoned that if an application wants to enable these keys, then they should be put into "Application Key Mode" since this is an "application", but you don't want this.
The Linux console has no "ks" string so you can't fall into this trap at the console. For other terminals you may need to edit the termcap (or terminfo) or use another termcap entry. You need to change not only the "ks" string but also the termcap definitions of what they send: kd, kl, kr, ku. Then run tic to install it.
For vim (vi iMproved) there is a way to set it up to work OK with ESC O D (so you don't need to edit termcap): See vim help for "vt100-cursor-keys". You may run "gitkeys" and then press your cursor keys to see what they send but they may be set to send something different when you're in an editor.