Terminal tips for web developers

Many see the terminal as a scary beast and try to avoid it as much as they can. In reality it is a powerful tool for any *nix user and the more you know it the more you get addicted to it. The followings are some terminal tips you may find useful during your web develop.

There is nothing advanced or new here, it’s a post for personal reference. All tips work on both Mac and linux.

1. Keep SSH connections alive

I often keep SSH connections open all the day long, but the remote server or the local router may kill the connection after some minutes of inactivity to save resources. To avoid to be disconnected you may tell the ssh client to send a keep alive signal every X seconds.

All you need is to locate your ssh_config file (usually in /etc) and append the following lines at the end of the file (you may need root permissions)

ServerAliveInterval 600
ServerAliveCountMax 3

ServerAliveInterval tells the ssh client to send a signal every 600 seconds. The ServerAliveCountMax command makes the client retry 3 times before giving up (after 1800 seconds).

2. Transfer files through SFTP

I often need to grab one file from a remote server. I could open a SFTP client such as Filezilla, establish a connection, navigate to the directory where the file is located and download it OR I could just throw a single command in the terminal

sftp user@server:/path/to/file

Replace user with the user name you login with and server with the IP or domain of the remote server. If you have the SSH server on a not-standard port you can pass the server port as follow:

sftp -oPort=2222 user@server:/path/to/file

3. Dump remote mysql database locally

To dump a remote database you don’t need to login to the server, mysqldump the database and then transfer the file locally. You can do everything on the fly from the terminal

ssh user@server "/usr/bin/mysqldump -u user -p password database" | dd of=/where/you/want/the/dump.sql

You may need to change the path to mysqldump.

Note that you are actually using the remote mysql to dump the db, so you don’t risk incompatibilities between local and remote mysql server versions.

4. Duplicate a remote site

Note: if misused rsync can lead to unexpected results and data loss! Take your time to understand its syntax and experiment from and to safe locations before going live.

Rsync is one of the best things terminal has to offer. On the Mac is installed by default, on linux you probably need to get it (sudo apt-get install rsync, on Debian).

If you need to grab a whole site (or any directory) from a remote server all you need to do is

rsync -azq -e ssh user@server:/path/to/remote/directory /local/directory/

a option tells rsync to recursively copy all files keeping permissions, owners and timestamps intact (so you have a specular copy of the remote files). z compresses files during the transfer to save bandwidth. q makes things quiet, no output on screen. The first times you can replace q with v (verbose) to see what rsync is doing.

Note that the path to the remote directory is without the trailing slash.

Say you want to backup /var/www/matt to /Users/matt/Sites/matt the command you need to issue is

rsync -azq -e ssh user@server:/var/www/matt /Users/matt/Sites/

You can obviously synchronize the changes you make locally back to the server.

rsync -azq --delete -e ssh /Users/matt/Sites/matt user@server:/var/www/

We added the --delete option that removes files from the remote server if they have been deleted locally.

If you get a rsync: command not found you probably need to install rsync on your server.

You can also selectively exclude some files from the synching process. Create a new text file and put something like the following inside:

.DS_Store
.FBCLockFolder
._*
*bak
*~

This ensures that nasty Mac files and backups were not replicated to the server. You may add any filename and wildcard to the text file, one entry per line. Save the file as exclude.txt and tell rsync to use it:

rsync -azq --delete --exclude-from /path/to/exclude.txt -e ssh /Users/matt/Sites/matt user@server:/var/www/

Adorable, isn’t it?

If you have directory names with spaces wrap the path with single quotes and escape spaces with . E.g.: '/Users/matt/Sites/Path With Spaces'

5. Use remote server as a proxy

Sometimes you may have the necessity to test a website from a different location or IP address (say to test IP aware functionalities). You can easily use your remote server as proxy through SSH.

ssh -D 8080 -Nf user@server

You have just created a proxy on port 8080. To use it on Firefox open Preferences, go to Network tab under Advanced options and click the Settings… button. Select Manual proxy configuration, uncheck Use this proxy server for all protocols and enter 127.0.0.1 Port: 8080 under SOCKS Host.

On Safari the proxy configuration is under Preferences > Advanced > Proxies. Check SOCKS Proxy and enter 127.0.0.1 : 8080 under SOCKS Proxy Server. Other browsers have similar settings.

Remember that the proxy is running as a background process. You can see it in action by issuing the following command:

ps aux |grep ssh

When you don’t need it anymore you have to kill it from a task manager, or from the terminal with something like killall ssh. Obviously you will have to reset the browser as well.

2 thoughts on “Terminal tips for web developers”

  1. another tip for searching command in bash: hit cotrol+r and start typing part of a command you entered, it will magically reappear, hit ctrl+r again to search another match

Comments are closed.