So, why *NIX and why do you need to go to the console?
According to the statistics of W3Techs, Unix is used by 68% of all the websites whose operating system they know. This means that if you are web developer, your code is most probably running on a Linux server. And at least you need to know how to configure and debug your code on Unix and Linux systems. Let's find out what you need to know to feel comfortable in the command line.
The Basics
The basic *NIX command consists of three components:
- command or program to run
- options to alter or specify the behavior of the command
- arguments or input data that is needed to run the command
For example, if you need to get a list of
files in the directory /var/www, you need to run the command ls with the argument /var/www. To add the size of files to the output, you need to add the -s option, and the final command will look like this:
ls -s /var/www
I/O Redirections and Pipelines
Many *NIX commands use text input and output which you can operate with, and the great feature of this is that you can send the output results of the command to a file using redirect, or even pass the output of one command to the input of another command using the pipelines. For example, we can output the command from the previous example to a file:
ls -s /var/www > /var/www/files.txt
This command will create or erase file
/var/www/files.txt and output a list of files in the /var/www directory.
Here is a list of standard I/O redirections and pipelines:
-
>Redirect output from a command to a file on disk. The file will be erased and overwritten. -
>>The same redirect, but appending the output file. -
<Get input to command from a file. -
|Pass the output of one command to the input of another command. -
teeBoth redirect output to a file and pass it to the next command in the pipeline.
The Main Commands
To get manual pages for a command, run
man. Manual pages follow a common layout and may include name, synopsis,
description, and usage examples. This will display the documentation
for the chmod command:
man chmod
To execute some commands like saving
configurations or rebooting processes, you need to run them as
the super user. To do this, you need to prepend sudo to your command:
user@server:/var/www$ chmod 777 log chmod: changing permissions of ‘log’: Operation not permitted user@server:/var/www$ sudo chmod 777 log sudo chmod 777 log
If you need to execute a bunch of
commands as a super user, you can use su, or switch user command.
user@server:/var/www$ su Password: root@server:/var/www#
Note: To save the security layer and
avoid accidental execution of objectionable commands, do not use
sudo and su without any purpose.
Into the Real World
Basic Navigation
There are three main commands to navigate in the file tree:
-
pwdto print the name of the current working directory -
cdto change directory -
lsto list directory contents
Here is an example of using these commands with the output of terminal:
user@server:~$ pwd /home/user user@server:~$ cd /var/www user@server:/var/www$ ls -alF total 16 drwxr-xr-x 5 root root 4096 Jan 22 09:45 ./ drwxr-xr-x 14 root root 4096 Jan 22 09:38 ../ drwxr-xr-x 2 root root 4096 Jan 22 09:45 html/ drwxr-xr-x 3 root root 4096 Jan 22 09:45 log/ drwxrwxrwx 1 user user 442 Mar 24 12:22 testing/
Searching Files
There is the find command to search for
files in a directory hierarchy. This command is very powerful and can
search for files and directories by name, access permissions, date, and size.
Find all directories with the “logs”
name in the /var/www directory using the -type option:
find /var/www -type d -name logs
To search PHP files in the current
directory, add the -name option:
find . -type f -name "*.php"
Find files with defined permissions
using the -perm option:
find . -type f -perm 0777 -print
Find all files which are greater than 500MB:
find / -size +500M
Of course, you can combine all of those
options in one command, and this is only the basics of the find command, which
is a very powerful tool for searching files. Use manual pages to get more
information.
Manipulating Files and Folders
There are five main commands for manipulating files and folders in *NIX system:
-
touchis used to change timestamps on existing files and directories, but also this is the easiest way to create new file -
mkdirto make directories -
cpto copy files and directories -
mvto move or rename files and directories -
rmto delete files and folders
The next example will
create a file index.html, copy this file to the new directory in
/var/www, and remove the source file.
root@localserver:~# touch index.html root@localserver:~# mkdir /var/www/newdir root@localserver:~# cp index.html /var/www/newdir/ root@localserver:~# rm index.html
Another
great command is ln, which is designed to make links between files.
The command ln is often used to create a symbolic link for
enabling a virtual host:
sudo ln -s /etc/apache2/sites-available/newvirtualhost.com.conf /etc/apache/sites-enabled/ newvirtualhost.com.conf
Change Access Permissions
To change the file owner and group, use
chown. Don't forget to grant ownership to the apache user when you are
creating a new virtual host of your web application:
sudo chown -R www-data:www-data /var/www/newvirtualhost.com
Sometimes cache or log directories of
your application must be writable for all users so you need to change
access modes to 777 with the chmod command. Add the -R option to add permission to all nested
files and folders.
sudo chmod -R 777 /var/www/private/cache
If you just want to make a file
executable, use chmod with the +x option.
sudo chmod +x /var/www/private/backup.sh
File Reading
To view files in the console, you can use the cat command. With cat, you can concatenate files' contents using extra
parameters, and you can also use mask in filenames.
cat /etc/apache2/apache2.conf cat /etc/apache2/apache2.conf /etc/apache2/ports.conf cat /etc/apache2/mods-enabled/*
But the cat command will get you
confused very fast, because it shows output in raw format without
any paging—so it is inconvenient to use with log output. To get
a filter for paging through text one screenful at a time, you should
use the more or less commands, which are much of a muchness.
less /etc/apache2/apache2.conf cat /etc/apache2/mods-enabled/* | less
Another useful command is tail, which is
created to output the last part of files. This command is perfect to look through log histories. By default, this tail command prints the last 10
lines, and you can change this number using the -n parameter.
tail /var/log/apache2/error.log tail -n 25 /var/log/apache2/error.log
But if you have, for example, a bunch
of log files, you need something more powerful to make a proper
search. Something like grep—a program that reads from standard
input, tests each line against a pattern, and writes to standard
output the lines that match this pattern. By using it in combination with cat
and pipelines, you will get what you want.
If you want to filter text-lines of
output, you can use the grep command:
grep notice /var/log/apache2/error.log cat /var/log/apache2/*.log | grep "shutting down"
As you can see, grep is good for using
in pipelines. In this example, the last command will output all lines
containing the “shutting down” string from log-files.
File Editing
If you want to edit text files in console mode, you can use one of the three most popular text editors:
- GNU nano, a small and friendly default text editor, which is a perfect choice for basic tasks
- Vim, an improved programmers' text editor, which is most powerful, but complex for beginners
- mcedit, a full-featured windowed editor from Midnight Commander, which is easy to use but not installed by default on *NIX systems
Compare them and make your choice:
nano /var/log/apache2/error.log vim /var/log/apache2/error.log mcedit /var/log/apache2/error.log
Archiving
Sometimes you need to back up or compress some data on your server.
The most common archiving utilities are tar
and zip. Notice that the zip command may not be installed on
your server by default.
You can create an archive with the following commands:
tar -zcvf archive-name.tar.gz directory-or-file-name tar -jcvf archive-name.tbz2 directory-or-file-name zip archive-name.zip directory-or-file-name
If you want just to see a list of files
in the archive, you can use the -l option for both tar and unzip:
tar -ztvf archive-name.tar.gz tar -jtvf archive-name.tbz2 unzip -l archive-name.zip
Or extract some source files:
tar -zxvf archive-name.tar.gz tar -jxvf archive-name.tbz2 zip archive-name.zip
Schedule Tasks
If you want to schedule your scripts to
run periodically, you need to use the Cron utility, which is driven by a
cron table—a configuration file that specifies the shell commands to
run periodically on a given schedule. And the command to maintain
cron tables is crontab.
Calling crontab with option -l will
show your cron table.
Also, the -u option is provided to specify
the name of the user whose crontab is being used. If you are going
to run tasks of your web application, it is better to edit crontab
for user www-data.
user@server:~$ sudo crontab -lu www-data # m h dom mon dow command */5 * * * * php5 /var/www/yii do/tasks >> /var/www/tasks.log 00 15 * * 1-5 /var/www/backuper.sh
In this output, you can take a look at an example of a cron table. As you can see, every line is scheduled by minute, hour, day of month, month, and day of week. Every field may be an asterisk, which means every value of the field. Also you can use sets and ranges using commas and hyphens. Following a range with a slash specifies skips of the number's value through the range. In this example, the first command will run every five minutes, and the second command will run from Monday to Friday at 15:00.
To edit this list, run crontab with the -e
key instead of -l. The cron list will be opened in your default editor.
Use the -r option to clear the cron list.
Performance Monitoring
Command top shows system summary
information and provides a dynamic real-time view of running
system processes. Press Shift-M to sort processes by memory usage, or Shift-P to sort by CPU usage.
top - 21:33:02 up 308 days, 21:24, 1 user, load average: 0.00, 0.01, 0.05 Tasks: 87 total, 1 running, 86 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.3 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 501800 total, 471348 used, 30452 free, 49672 buffers KiB Swap: 4194300 total, 56192 used, 4138108 free. 149488 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16269 www-data 20 0 348592 38884 12044 S 0.0 7.7 0:02.42 php5 26533 www-data 20 0 409516 38488 24312 S 0.0 7.7 1:00.04 php5-fpm 1076 mysql 20 0 887824 32748 1616 S 0.0 6.5 276:46.59 mysqld 862 syslog 20 0 256612 31428 368 S 0.0 6.3 32:45.88 rsyslogd 18901 root 20 0 105632 4316 3248 S 0.0 0.9 0:00.04 sshd 25393 www-data 20 0 87356 4312 1564 S 0.0 0.9 4:46.92 nginx 27846 memcache 20 0 328464 3828 252 S 0.0 0.8 1:04.30 memcached
To display the amount of free and used memory
in the system, use the free command. Add the -h option to display output fields
in human-readable format.
user@server:~$ free -h
total used free shared buffers cached
Mem: 490M 453M 36M 23M 46M 140M
-/+ buffers/cache: 265M 224M
Swap: 4.0G 54M 3.9G
Another useful command is df, which is a
command to report file system disk space usage. You can call it with the -a option to show all the file systems of your server. Also, don't
forget to add the -h option for human-readable format.
user@server:~$ df -ah Filesystem Size Used Avail Use% Mounted on /dev/vda1 20G 6.3G 13G 34% / udev 235M 4.0K 235M 1% /dev tmpfs 50M 344K 49M 1% /run
Command Line History
You can use the !! command to repeat the previous command, or use sudo !! if you forgot to run a command with
sudo.
user@server:/var/www$ chmod 777 log chmod: changing permissions of ‘log’: Operation not permitted user@server:/var/www$ sudo !! sudo chmod 777 log
If you forgot the syntax of commands or are feeling lazy about typing a large command query, you can use history to
display your command history. It is good to combine this command with
strings filter commands like grep, tail and others to find exactly what
you want.
history | tail history | grep crontab history | egrep -i 'ssh|ftp'
Conclusion
Using the console is not rocket science. Unix and Linux systems are easy to understand and use because of their simple design and good documentation. I hope this article will make you pretty comfortable with the command line and bring you to the next level of managing your web applications with the command line.
If you have any questions or you want to share your favourite console commands, don't hesitate to leave a comment below the article.
Comments