Linux Commands Every Architect/Developer Should Know
Linux Commands Every Architect/Developer Should Know
#1 – Process Monitoring
top – To view CPU & memory usage information. _General guidance is Load Average should be similar to the number of cores for the system not under stress._You can also use newer version htop command for more interactivity.
Usage Example:
top -u <user> #user specific processes Press M: sort by memory, P: sort process list by CPU usage, V: Forest view, R: for reversing the order htop -u <user> #user specific processes
- ps – Classic command to view list of running processes – Linux variants have different parameter options. Below examples are the most useful:
ps auwwx #list all the processes with wider format & easy to remember ps -ef | grep 'java' #to filter specific process ps -ef | awk '{print $1}' | sed '/UID/d' | sort | uniq -c | sort -nr #count running process under each user
- kill – Classic command for killing a running process. killall is useful to kill all the processes by a name.
kill -l #to list all available signals kill -9 <pid> #SIGKILL signal to kill the process killall top #kill all top processes; you can put any process name
- lsof– List of Open Files by the process – very useful when debugging resources consumed by a process particularly when everything in Linux is a file. Note that it might not be installed by default in some variants (e.g. CentOS) and you need to install “lsof” package.
lsof -u <username> #list of files opened by a user lsof -c <process> #list of files opened by a process named lsof -p <pid> #list of files opened by PID lsof -i #files opened by network lsof -i :443 #to find the process/service listening on a port
- fuser – shows the PIDs of processes using the specified files or file systems in Linux.
You need to install “psmisc” package before using this command – see installation section below for instruction as per Linux variant.
fuser 80/tcp #find the process/service listening on a particular port by running the command fuser <filename> #to find out file being used by the
#2 – File Operations
- tail – for viewing the last part of files/logs, most commonly used parameter is “tail -f”
- find – for searching the files. Commonly usage examples – click here:
find . -name "*.log" #to find the log files in current directory & sub-directory find . -type f -name "*.java" #to find all java file types find . -type f -perm 0777 -print #to find files with 777 permissions find / -type f -perm 0777 -print -exec chmod 555 {} \; #to find files with 777 permissions and replace with 555 find / -size +50M -size -100M #to find files between 50-100MB size
- grep – for searching pattern within file. Click here for different examples.
grep -i "linux" *.log #case insensitive search within log files grep "REGEX" *.log #use any REGEX grep -w "word" *.log #check for full word grep -c "word" *.log #count the number of words matched grep -v "word" *.log #invert the match to display non-matching grep -r "word" *.log #to search recursively in all folders grep -A 5 -i "word" *.log #display 5 lines after the match
- Others: uniq sort diff cut ncat sed awk
#3 – Network Monitoring
- tcpdump – for analyzing network packet level details. Useful for packet-level inspection, detecting denial of service attacks by inspecting large packets or source, debug the source & destination generating traffic, etc.- tcpdump not port 22 and not port 25 (exclude SSH to avoid unnecessary info)
Usage examples:
tcpdump udp #for capturing specific protocol dump; find list of protocols in /etc/protocols) tcpdump -c10 -i eth1 -n -A port 80 #dumps in ASCII format with (-A) with specific port and exit after receiving 10 packets (-c)) tcpdump -l | tee dat #Make stdout line buffered. Useful if you want to see the data while capturing it tcpdump -i eth0 host 10.122.19.222 #capture for specific IP address tcpdump -s 0 -A -vv 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420' #capture only HTTP GET or POST only
- Note: If not installed, install “tcpdump” package (e.g. for Alpine Linux: apk add tcpdump). Click here to read the troubleshooting article by RedHat and see 20 different examples by clicking here.
- netstat – Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships
- traceroute – print the route packets trace to network host
- ss – utility to investigate sockets (very useful to check socket stats)
- iperf – utility to check network bandwidth between two systems. Click here to read the most useful link found for iperf.
- Others: strace, mtr, dig, sar, ifconfig (prints ip address)
#4 – Disk & Memory Monitoring
- du / df – du (disk usage of the set of FILEs, recursively for directories), df (report file system disk space usage)
- free – to find out free and used memory
- iostat – for CPU stats and I/O stats for devices & partitions
- mount – to mount a file system
- fdisk – to manipulate disk partition
Installing Package
Below commands are example of installing packages for different variations:
$ yum install <package> # RHEL/CentOS $ apt install <package> # Debian/Ubuntu $ apk add <package> # Alpine Linux $ dnf install <package> # Fedora
Other Useful Commands
- alias (for bash) – useful for creating shortcuts to frequently used commands
- tree – shows a visual representation of the files in a directory
- watch – run any command at regular intervals and displays the output
- truncate – shrink or extend the size of a file to the specified size. For example, the below command will free up capacity quickly:
truncate -s 0 filename
Security:
- Check Certificate Expiry and other details using openssl command
openssl x509 -in my-cert.pem -noout -issuer -subject -dates
Top 10 commands from bash history:
cat ~/.bash_history | sort |uniq -c|sort -nr|head -n 10
- systemctl****– to control the running system and other services
- Miscellaneous: mpstat, pmap, kill, iostat, vmstat, chkconfig, uptime (shows how long system is running), pidof, cal (shows calendar), zip, unzip, ssh, whatis (locate the binary), whereis (one-line man page), finger (short dump of info about a user), w (current user info), chown (change ownership of file), chmod (change permission of files & directories), locate (to find a file), ping (to check connectivity)
To conclude, these commands/utilities help not only to troubleshoot but to understand the functions of Linux operating system. Even though you are using application performance monitoring (APM) tools, as a software architect you should know these expand the full-stack knowledge.
Linux Useful Websites
- Linux.org – for discussion, tutorial, and references
- Linux.com – by Linux Foundation for latest news, articles & updates
- Linux Foundation – community-driven site focused on Opensource
- man7.org – Linux Online Manual
- Linux Commands Cheatsheets
- Explain Shell – Explains each part of the command (useful when trying to understand complex commands used together with parameters)
- nixCraft/cyberciti – for Linux sysadmins
- Linux Today – for latest news on Linux