Safer terminal

As much as we strive towards having the whole server fleet be treated as an indistinguishable herd of cattle, there are still times when taking care of pet-servers is necessary. One such time is when the infrastructure is still undergoing a cloud migration, or when a particular server really is a special darling.

In such cases, any additional feedback to the operator helps in avoiding mistakes. In this post you’ll see approaches that you might use to enable such visual feedback. The end goal is to immediatelly visually discern that one has entered a potentially dangerous environment, one where any mistake might be of grave consequences.

Most of the time, an operator’s tool of choice is a terminal. We’ll focus on iTerm2, a terminal on macOS. But even if you’re not an iTerm2 user, the following tips might inspire you to apply something similar in your chosen terminal.

iTerm2

Triggers

The prerequisite for Triggers is to ‘Install Shell Integration’. You’ll find it in iTerm’s menu bar:

Then, you can enable Automatic Profile Switching. But, since installing APS on remote hosts is often impractical or inadvisable, you can work around this by creating a so-called Trigger.

First create a new profile, say “Production”, and under “Advanced” tab add a Trigger with the following configs:

• Regular Expression: ^(\w+)@([\w.-]+):.+\$ (amend if necessary)
• Action: Report User & Host
• Parameters: \1@\2
• Instant: checked
• Enabled: checked

Then head back to the APS setting and insert hostnames of your production servers. Finally, for this new profile set a distinct visual feedback under the “Colors” tab - white-on-red would be fitting.

Badges

Another, or additional, option is to set a badge, per iTerm pane, when a particular command gets executed. You can wrap the existing command for switching your infrastructure-related profiles and have the wrapper also execute iTerm’s SetBadgeFormat command. Here’s a sample:

declare -a important_profiles=("prod" "stag")
if [[ "${important_profiles[*]}" =~ "$profile" ]]; then
  echo -e "\x1B]1337;SetBadgeFormat=$(echo "Careful: ${profile}" | base64)\007"
fi

Regex matching on longer lines

In case the part of the log that you want to match on has (too) many logical lines, you should increase the following iTerm limit: Preferences > Advanced > Number of screen lines to match against trigger regular expressions.