Linux commands cookbook

50 Most Frequently Used UNIX / Linux Commands
http://www.thegeekstuff.com/2010/11/50-linux-commands/

Write a Simple Script in Shell

Write a for loop.

$ for pod_name in $(kubectl get pods -l app=swag-worker-test -o jsonpath={..metadata.name}); do; kubectl delete pod $pod_name; done
pod "swag-worker-test-67fffcdd5-5hgf3" deleted
pod "swag-worker-test-67fffcdd5-h8jgg" deleted

# you could also write multiple lines
for pod_name in $(kubectl get pods -l app=swag-worker-test -o jsonpath={..metadata.name}); do
    kubectl delete pod $pod_name
done

Write a while true.

# using trap and wait will make your container react immediately to a stop request
$ bash -c "trap : TERM INT; sleep infinity & wait"
# or
$ bash -c "while true; do echo 'I am alive!'; sleep 3600; done"

#!/bin/bash
while true; do echo 'I am alive!'; sleep 3600; done

ref:
https://stackoverflow.com/questions/31870222/how-can-i-keep-container-running-on-kubernetes

Set Environment Variables from a File

$ export $(cat .env | grep -v ^# | xargs)

ref:
https://stackoverflow.com/questions/19331497/set-environment-variables-from-file

Switch to Another User

# the latter with "-" gets an environment as if another user just logged in
$ sudo su - ubuntu

Clear the Content of a File

$ echo -n > /var/log/nginx/error.log

ref:
https://unix.stackexchange.com/questions/88808/empty-the-contents-of-a-file

Pipeline stdout with xargs

$ find . -type f -name "*.yaml" | xargs echo
./configmap.yaml ./pvc.yaml ./service.yaml ./statefulset.yaml

$ find . -type f -name "*.yaml" | xargs -n 1 echo
./configmap.yaml
./pvc.yaml
./service.yaml
./statefulset.yaml

$ find . -type f -name "*.yaml" | xargs -n 2 echo
./configmap.yaml ./pvc.yaml
./service.yaml ./statefulset.yaml

$ redis-cli KEYS "*-*-*-*.reply.celery.pidbox" | xargs -n 100 redis-cli DEL

ref:
https://shapeshed.com/unix-xargs/

Set a Timeout for any Command

$ timeout -t 15 celery inspect ping -A app:celery -d celery@$(hostname)

Run a One-time Command at a Specific Time

at executes commands at a specified time. You may need to install the "at" package manually.

# install
$ sudo apt-get install at

# start
$ sudo atd

# list jobs
$ atq

$ at 00:05
at> echo "123" > /tmp/test.txt

$ at 00:00 18.1.2017
at> DPS_ENV=production /home/ubuntu/.virtualenvs/dps/bin/python /home/ubuntu/dps/manage.py send_emails > /tmp/send_emails.log

Press Control + D to exit at shell.

ref:
https://www.lifewire.com/linux-command-at-4091646
https://tecadmin.net/one-time-task-scheduling-using-at-commad-in-linux/

Pass Arguments to bash when Executing a script Fetched by curl

$ curl -L http://bit.ly/open-the-pod-bay-doors | bash -s -- --tags docker

ref:
https://stackoverflow.com/a/25563308/885524
https://github.com/vinta/HAL-9000

Change a File's Modify Time

$ touch -m -d '1 Jan 2006 12:34' tmp
$ touch -m -d '1 Jan 2006 12:34' tmp/old_file.txt

ref:
https://www.unixtutorial.org/2008/11/how-to-update-atime-and-mtime-for-a-file-in-unix/

Delete Old Files under a Directory

$ find /data/storage/tmp/* -mtime +2 | xargs rm -Rf
$ find /data/storage/tmp/* -mtime +2 -exec rm {} \;

ref:
http://stackoverflow.com/questions/14731133/how-to-delete-all-files-older-than-3-days-when-argument-list-too-long

Append String to a File

# append
$ echo "the last line" >> README.md

# replace
$ echo "replace all" > README.md

Rename Sub-folders

$ for f in */migrations/; do mv -v "$f" "${f%???????????}south_migrations"; done

ref:
http://unix.stackexchange.com/questions/220176/rename-specific-level-of-sub-folders

List History Commands

$ export HISTTIMEFORMAT="%Y%m%d %T  "
$ history

Make Permission For A File Same As Another File

$ chmod --reference=file1 file2

Find Computer's Public IP

$ wget -qO- http://ipecho.net/plain ; echo

ref:
http://askubuntu.com/questions/95910/command-for-determining-my-public-ip

Compress and Uncompress Files

$ tar czf media-20151010.tar.gz media/
$ s3cmd put media-20151010.tar.gz s3://goeasytaxi/

# decompress
$ tar -xzf media.tar.gz

$ sudo apt-get install zip unzip
$ zip -j -r deps.zip spark_app/src/deps/
$ zip -r hourmasters.zip hourmasters/
$ scp -r -i ~/hourmasters.pem ssh [email protected]:/home/ubuntu/hourmasters.zip ~/Desktop/

# decompress
$ unzip stork.1.4.zip
$ gzip -d uwsgi.log.*.gz

$ gzip dps.201701171200.sql
$ gunzip dps.201701171200.sql.gz

Count File Lines

$ wc -l filename.txt

$ wc -l *.py

Find Files by Name or Content

$ find / -name virtualenvwrapper.sh

# 在現在的資料夾裡的全部檔案中搜尋字串，會自動搜尋子目錄
$ find . | xargs grep 'string'

$ find . -iname '*something*'

$ find *.html | xargs grep 'share_server.html'

# 搜尋當前目錄及子目錄下的含有 print() 字串的檔案
$ grep -rnw "." -e "print()"

$ grep -rnw "." -e "print()" --include=\*.py

ref:
https://stackoverflow.com/questions/16956810/how-do-i-find-all-files-containing-specific-text-on-linux

Find Directories by Name

$ find . -type d -name "*native*" -print

ref:
https://askubuntu.com/questions/153144/how-can-i-recursively-search-for-directory-names-with-a-particular-string-where

List Files by Date

$ ls -lrt

List Files Opened by a Process

$ lsof | grep uwsgi

$ lsof -i | grep LISTEN
$ lsof -i -n -P | grep LISTEN

Extract Content from a File

$ cat uwsgi.log | grep error

Display Contents of All Files in the Current Directory

$ grep . *
$ grep . *.html

List Used Ports

$ netstat -a

# TCP
$ netstat -ntlp | grep uwsgi

# UCP
$ netstat -nulp

Ping a Port

$ curl -I "10.148.70.84:9200"
$ curl -I "192.168.100.10:80"

$ sudo apt-get install nmap
$ nmap -p 4505 54.250.5.176
$ nmap -p 8000 10.10.100.70
$ nmap -p 5672 10.10.100.82

$ telnet 54.250.5.176 4505

ref:
http://stackoverflow.com/questions/12792856/what-ports-does-rabbitmq-use

Show Network Traffic and Bandwidth

$ tcpdump -i eth0

$ sudo apt-get install tcptrack
$ tcptrack -i eth0

ref:
http://askubuntu.com/questions/257263/how-to-display-network-traffic-in-terminal

List Running Processes

# show all processes
$ pstree -a

# also show pid
$ pstree -ap

# 列出前 10 個最佔記憶體的 processes
$ ps aux | sort -nk +4 | tail

# 列出 mysql 相關的 processes
$ ps aux | grep 'worker process'
$ ps aux | grep uwsgi

# 樹狀顯示
$ ps auxf

# 搜尋 process 並以樹狀結果顯示 parent process
$ ps f -opid,cmd -C python

Kill Processes

# 列出目前所有的正在記憶體當中的程序
$ ps aux

# 匹配字串
$ ps aux | grep "mongo"

# 幹掉它
$ kill PID

# kill all processes matching a name
$ sudo pkill -f runserver

Store User's Input as a Variable

$ read YOUR_VARIABLE_NAME

$ read name
# you type: vinta

$ echo $name
vinta

ref:
https://canred.blogspot.tw/2013/03/read.html

Show Terminal Size

$ stty size
$ echo $LINES && echo $COLUMNS
59 273

ref:
https://stackoverflow.com/questions/263890/how-do-i-find-the-width-height-of-a-terminal-window