General Information

Finding text and files

• grep -R "text" Find specific text in files in current directory and child directories
• grep -rnw '/path/to/somewhere/' -e 'pattern' Find specific text in files in specified directory
• sed -i 's/original/new/g' file.txt Replace especific characters/words in a file
• find ./ -type f -exec sed -i -e 's/Foo/Bar/g' {} \; Replace all instances of the word "Foo" for "Bar" in the current directory and children directories.
• find my_root_dir -depth -exec rename 's/(.*)\/([^\/]*)/$1\/\L$2/' {} \; Rename all files and subdirectories to lowercase
• find / -type f Get a list of all directories and files
• find / -type f | wc -l Count the number of all directories and files
• find /usr -iname "wallpaper.jpg" Find all files with specific text

Used and available space

• df -h Disk usage by partition
• du -shc * Disk usage by directory and files
• du -hc Disk usage by directory and files (recursive, detailed)
• du -sch .[!.]* * |sort -h Total detailed and sorted usage, included hidden files and directories

CPU and Load Average

Load Average is the amount of processes running and/or waiting to be run in the queue. Depending on the number of CPU cores/processors in the system, it may be overloaded or not.

CPU cores

• nproc
• lscpu
• grep 'model name' /proc/cpuinfo | wc -l

Load Average

• uptime
• cat /proc/loadavg
• top

Based on this example output:

load average: 1.00, 0.40, 3.35

Number of jobs in the run queue or waiting for disk I/O averaged over 1, 5 and 15 minutes. The interpretation on 1 core vs 4 core is:

• Last minute: 100% usage of the run queue on a 1 core processor, 25% usage of the run queue on a 4 core processor
• Last 5 minutes: 40% usage of the run queue on a 1 core processor, 10% usage of the run queue on a 4 core processor
• Last 15 minutes: 100% usage of the run queue and 235% overloaded (waiting for disk I/O) on a 1 core processor, 83.75% usage of the run queue on a 4 core processor

Text manipulation commands

grep, awk, sed, cut, sort, diff, head, tail, uniq, fmt, pr, tr, more, less

awk

df -h | awk '{ if ($6 == "/") print "Disk usage: " $3"/"$2 }'
free -h | awk '(NR==2){print $3"/"$2}'

Realtek RTL8812AE 802.11ac wireless network card

By default, Linux cannot find Realtek RTL8812AE 802.11ac drivers and uses instead rtl8821ae.

In order to use the proper drivers, you have to download them and copy them into /lib/firmware/rtlwifi/:

wget https://github.com/lwfinger/rtlwifi_new/blob/master/firmware/rtlwifi/rtl8812aefw.bin
wget https://github.com/lwfinger/rtlwifi_new/blob/master/firmware/rtlwifi/rtl8812aefw_wowlan.bin
cp rtl8812aefw* /lib/firmware/rtlwifi/

Jobs and Processes

Start a process in the background

• With output from stdout and stderr

$ gatsby develop &

• Without output from stdout and stderr

$ gatsby develop 1 >/dev/null &

• Bring process to foreground: fg
• Move running process to the background: Ctrl + z
• See running processes: top
• Kill a process: kill -9 PID

Configuration files

• Global: /etc/profile
• User: ~/.bashrc

After making changes to file you need to reset the changes by executing following command:

# source /etc/profile
OR
# source ~/.bashrc

Graphics

Exit graphical environment

• Ctrl + Alt + F1

sudo service lightdm stop

Go back to graphical environment

sudo service lightdm start

Users and Groups

• List users: cat /etc/passwd
• List groups: cat /etc/groups
• Create user: sudo useradd -m -s /bin/bash newuser
• Create password: sudo passwd newuser # if user is not input, it will change root password
• Add user to group sudo: sudo usermod -aG sudo newuser
• Edit sudo file: sudo visudo
• Create group: sudo groupadd mygroup
• Delete user: sudo userdel myuser
• Delete group: sudo groupdel mygroup
• List groups a user belongs to: groups user
• List groups a user belongs to: id user

Disks and Filesystem

Boot loader

• efibootmgr: Modify the UEFI Boot Manager by creating and destroying boot entries, changing the boot order, and more.

Partition tools

• fdisk: Allows you to create, view, delete, and modify partitions on any drive that uses the MBR method of indexing partitions
• cfdisk: The same as fdisk, but it provides a slightly fancier user interface with menus one can browse with arrows
• sfdisk: The same as fdisk, but designed to be scripted, enabling administrators to script and automate partitioning operations
• gdisk: Allows you to create, view, delete, and modify partitions on any drive that uses the GPT method of indexing partitions
• parted: Allows you to modify existing partition sizes, so you can easily shrink or grow partitions on the drive.

Filesystem stats

• df: Displays disk usage by partition
• du: Displays disk usage by directory; good for finding users or applications that are taking up the most disk space
• iostat: Displays a real-time chart of disk statistics by partition
• lsblk: Displays current partition sizes and mount points
• blkid: Display information about block devices, such as storage drives

In addition to these tools, the /proc and /sys directories are special filesystems that the kernel uses for recording system statistics

Filesystem tools

• mkfs: Create different types of filesystems
• mount: Temporarily mount a filesystem to a Linux virtual directory
• umount: Unmount a filesystem
• eject: Unmount a filesystem and remove the partition
• chattr: Change file attributes on the filesystem
• debugfs: Manually view and modify the filesystem structure, such as undeleting a file or extracting a corrupted file
• dumpe2fs: Display block and superblock group information
• e2label: Change the label on the filesystem
• resize2fs: Expand or shrink a filesystem
• tune2fs: Modify filesystem parameters

How to list all disks and partitions

• lsblk
• sudo fdisk -l

How to list all devices and drivers

lspci -v | less

How to delete a partition

$ sudo fdisk /dev/sdb
> d
> w

How to create a partition

$ sudo fdisk /dev/sdb
> n
> w

How to format a partition

$ sudo umount /dev/sdb3
$ sudo mkfs.ext4 /dev/sdb3
or
$ sudo mkfs.vfat /dev/sdb3
or
$ sudo mkfs.ntfs /dev/sdb3
$ sudo eject /dev/sdb

How to remove write protection of usb drives

$ sudo hdparm -r0 /dev/sdX

How to find out hardware installed

• BIOS: $ sudo dmidecode -t 0
• Motherboard: $ sudo dmidecode -t 2
• Processor: $ sudo dmidecode -t 4
• RAM: $ sudo dmidecode -t 17
• Max RAM allowed: $ sudo dmidecode -t 16
• More info at https://www.linuxtechi.com/dmidecode-command-examples-linux/

Create bootable pendrive from iso image

1. Insert pendrive in USB port
2. Check what is the USB pendrive device/partition: sudo fdisk -l or lsblk
3. Unmount the device/partition: umount /dev/sdc1 (in my case, sdc1 is the USB pendrive device partition)
4. Create bootable pendrive: sudo dd bs=4M if=my_iso_image.iso of=/dev/sdc status=progress oflag=sync

In step 4, you must navigate where the iso image is before issuing the command.

How to create an image file for partition backup

sudo dd if=/dev/sdb5 of=/media/cheo/TOSHIBA_HDD/Lubuntu_backup.img oflag=append conv=notrunc,sync,noerror status=progress

The command above does the following:

• Create an image file of the sdb5 partition and copy it into an external usb hard drive (TOSHIBA_HDD)
• You can name the backup image file as you please. I chose Lubuntu_backup.img, but you can name it as you wish. However, BE SURE to use the file extension .img
• oflag=append conv=notrunc copy the file in the hard drive without using the whole hard drive (otherwise the whole hard drive will be wiped out and all the original data in the hard drive will be lost)
• conv=sync,noerror prevents the operation to halt if an error occurs
• status=progress shows the progress of the operation

Note that if the partition to backup is 50GB, but the data in the partition is only occupies 20GB, the whole 50GB of the partition will be backed up

How to restore a partition with a backup image file

sudo dd if=/media/cheo/TOSHIBA_HDD/Lubuntu_backup.img of=/dev/sdb5

How to add a newly installed operating systems into GRUB2

1. If not already installed, install os-prober
2. Run: sudo grub-mkconfig -o /boot/grub/grub.cfg or update-grub

More info: https://wiki.archlinux.org/index.php/GRUB

How to prepare a disk for EFI System

$ gdisk /dev/sdX
Command (? for help): o
Command (? for help): n
Partition number (1-128, default 1):
First sector (34-1331166, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-1331166, default = 1331166) or {+-}size{KMGTP}: 500M
Hex code or GUID (L to show codes, Enter = 8300): ef00
Command (? for help): p
Command (? for help): w

Edit GRUB2 boot options

Edit /etc/default/grub and make intended changes. Then run:

$ sudo grub-mkconfig -o /boot/grub/grub.cfg

How to access system on Boot Rescue mode

grub> ls
grub> ls (hd0,gpt2)/
/etc /media /boot /usr /var ...
grub> set prefix=(hd0,gpt2)/boot/grub
grub> set root=(hd0,gpt2)
grub> insmod /boot/grub/i386-pc/linux.mod
grub> linux /boot/vmlinuz-4.15.0-99-generic root=/dev/sda2
grub> initrd /initrd.img
grub> boot
grub-mkconfig -o /boot/grub/grub.cfg

Make a bash script executable

$ chmod +x mybashscript.sh

• Add an alias to the bash script by editing ~/.bashrc and appending this line:

alias <new name>='/home/<full path to script>/mybashscript.sh'

• Apply the changes in the system

$ source ~/.bashrc

• Instead of creating an alias, you can move mybashscript.sh to /usr/local/bin

List of data recovery distros

• System Rescue CD
• Parted Magic (paid subscription)

How create a System Rescue CD bootable USB

• Download the latest SystemRescueCd ISO image from the download page
• sudo mkdir -p /mnt/iso
• sudo mount -o loop,exec Downloads/systemrescuecd-x86-5.2.2.iso /mnt/iso
• Plug in your USB stick and wait 5 seconds to allow enough time for the system to detect it
• Unmount the USB stick if auto-mount is enabled or if it was already mounted
• cd /mnt/iso
• sudo bash ./usb_inst.sh
• In the dialog that opens*, navigate the correct USB device with the up and down arrow keys, press space bar to select it and press enter
• cd
• sudo umount /mnt/iso

*If you are not using xterm-256color, the dialog won't open and the terminal will throw: Error opening terminal: xterm-256color

How to create a Parted Magic bootable USB

• Download the latest Partes Magic ISO image from the download page
• Plug in your USB stick and wait 5 seconds to allow enough time for the system to detect it
• Unmount the USB stick if auto-mount is enabled or if it was already mounted
• Run dd if=/path/to/partedmagic.iso of=/dev/sdx in a shell where sdx is the USB stick

List of disk partition command-line utilities

• fdisk
• parted
• gdisk
• cfdisk

List of data recovery command line tools

• ddrescue
• testdisk

Compress/Decompress directory

• Compress: tar -zcvf mydir.tar.gz mydir
• Decompress: tar -zxvf mydir.tar.gz

ddrescue

ddrescue is a tool designed for cloning and recovering data. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying to rescue the good parts first in case of read errors, to maximize the recovered data.

To clone a faulty or dying drive, run ddrescue twice. First round, copy every block without read error and log the errors to rescue.log.

# ddrescue -f -n /dev/sdX /dev/sdY rescue.log

Second round, copy only the bad blocks and try 3 times to read from the source before giving up.

# ddrescue -d -f -r3 /dev/sdX /dev/sdY rescue.log

Now you can check the file system for corruption and mount the new drive.

# fsck -f /dev/sdY

More info: https://wiki.archlinux.org/index.php/disk_cloning

Goodies

• bash-completion: bash utility

• bc: CLI calculator

• galculator: GTK+ scientific calculator

• xfce4-screenshooter: screenshot utility

• gnome-screenshot: screenshot utility

• ristretto: image viewer utility

• transmission-gtk: torrent client

• leafpad: text editor

• nethogs: CLI network processes monitor)

• vifm: vi type file manager

• obs: screencasting software

• screenkey: key press screencasting

• sensors: CLI temperature monitoring

• darktable: raw photo editing

• rawtherapee: raw photo editing

• nodejs: javascript library

• gatsby-cli: reactjs framework

• textlive-full, texmaker, latex2html, pandoc: text manipulation

  Suckless

• dwm

• dmenu

• dwmstatus

• st (terminal emulator)

• feh (image viewer)

• ranger (file manager)

• vifm (file manager)

• zathura (pdf reader)

• lynx (text web browser)

• simplescreenrecorder (screencast)

• groff

Microsoft Windows

Clean and format pen drive in Windows with Diskpart (Windows 10)

1. Launch CMD
2. diskpart
3. list disk
4. select Disk 2 *
5. clean
6. create partition primary

*On step 4, choose the correct disk number for your pen drive

Assign a letter to a drive

Sometimes, when plugging a hard drive or usb drive, Windows does not assign a letter and the drive is not shown in the Explorer. This is how to fix it:

• Launch 'Computer Management' -> Storage -> Disk Management
• Look for the drive without a letter -> Right click -> 'Change drive letter and path' -> Add -> Assign the following drive letter

How to create a bootable Windows 10 USB pendrive from Linux

1. Download Windows 10 from here. Then follow these commands:
2. Insert USB pendrive (in this example in /dev/sdd)
3. Issue these commands:

$ sudo add-apt-repository ppa:nilarimogard/webupd8

$ sudo apt update

$ sudo apt install woeusb

$ sudo woeusb --target-filesystem NTFS --device Downloads/Win10_1809Oct_English_x64.iso /dev/sdd

LaTeX

Installation

$ sudo apt install texlive-full texmaker

Note that texlive-full installs all packages available for texlive (~5GB). You may want to install just texlive-base and only some other packages that you will use.

Tips

• Add linkable table of content:

\usepackage{hyperref}

• Add Japanese text support:

\usepackage{CJKutf8}
\begin{document}
\begin{CJK}{UTF8}{min}
日本語
\end{CJK}
\end{document}

• Add Yen currency character:

\def\yen\\{{\setbox0=\hbox{Y}Y\kern-.97\wd0\vbox{\hrule height.1ex
width.98\wd0\kern.33ex\hrule height.1ex width.98\wd0\kern.45ex}}}
\begin{document}
The  cost is 108\yen\\
\end{document}

• Auto generate table of contents:
  • Add the line \tableofcontents between \begin{document} and \newpage
  • Each section must be preceded by \newpage:
  • For the new changes to take affect, you must comment and uncomment \tableofcontents:
    1. %\tableofcontents
    2. Run
    3. \tableofcontents
    4. Run

GRUB v2

The configuration file is /boot/grub/grub.cfg, but you shouldn't edit it directly. This file is generated by grub v2's update-grub(8), based on:

1. The script snippets in /etc/grub.d/
2. The configuration file /etc/default/grub

To configure GRUB "v2", you should do any or both of these:

• edit /etc/default/grub, then run update-grub
• modifying the snippets in /etc/grub.d/

Also check the GRUB2 page for more detailed configuration instructions, ideas and suggestions.

SysVinit vs Systemd

If chkconfig is not working, you can replace it with update-rc.d. These are the equivalents:

Regular Expressions (RegEx)

• Any one character -> .
• Any number of previous (including 0) -> *
• Any number of previous (non including 0) -> +
• End of line -> $
• Beginning of line -> ^
• Any non white space character -> \S
• Any white space character -> \s
• Optional -> ?
• Any lowercase letter -> [a-z]
• Any uppercase letter -> [A-Z]
• Any letter -> [A-Za-z]
• Any number -> [0-9]
• Validate IP address: grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)" file.text

iPhone

Transfer files from iPhone to Linux

1. Install ifuse

$ sudo apt install ifuse

2. Make a directory to connect and store iPhone files

$ mkdir ~iPhone

3. Connect your iPhone to the PC
4. Run ifuse over ~iPhone directory

$ ifuse ~iPhone

5. Launch your favourite file manager
6. Navigate to your iPhone directory and browse DCIM directory to get the videos and photos.

Disconnect

$ sudo umount ~iPhone

View HEIC image files

$ sudo apt install heif-gdk-pixbuf

Open file with file viewer

libimobiledevice

$ sudo apt-get install usbmuxd libimobiledevice6 libimobiledevice-util
$ sudo apt install ifuse
$ idevice pair
$ usbmuxd -f -v
$ ifuse ~iPhone