Watchman Utility Linux

Watchman watches files and directories for changes and triggers actions when specific changes are noted.

  • Watchman can recursively watch one or more directory trees (we call them roots).
  • Watchman does not follow symlinks. It knows they exist, but they show up the same as any other file in its reporting.
  • Watchman waits for a root to settle down before it will start to trigger notifications or command execution.
  • Watchman is conservative, preferring to err on the side of caution; it considers files to be freshly changed when you start to watch them or when it is unsure.
  • You can query a root for file changes since you last checked, or the current state of the tree
  • You can subscribe to file changes that occur in a root

Reference:
https://facebook.github.io/watchman/

How to block network traffic by country on Linux

As a system admin who maintains production Linux servers, there are circumstances where you need to selectively block or allow network traffic based on geographic locations. For example, you are experiencing denial-of-service attacks mostly originating from IP addresses registered with a particular country. In other cases, you want to block SSH logins from unknown foreign countries for security reasons. Or your company has a distribution right to online videos, which allows it to legally stream to particular countries only. Or you need to prevent any local host from uploading documents to any non-US remote cloud storage due to geo-restriction company policies.

Instructions and more details from this site:

http://xmodulo.com/block-network-traffic-by-country-linux.html

 

Linux Commands

Here are some useful Linux commands, from time to time they come in handy – see the man pages for more details and full syntax of each command.

Bash shortcuts:

CTRL + U – Cuts text up until the cursor.
CTRL + K – Cuts text from the cursor until the end of the line
CTRL + Y – Pastes text
CTRL + E – Move cursor to end of line
CTRL + A – Move cursor to the beginning of the line
ALT + F – Jump forward to next space
ALT + B – Skip back to previous space
ALT + Backspace – Delete previous word
CTRL + W – Cut word behind cursor
Shift + Insert – Pastes text into terminal

Pausing commands and running commands in the background:

CTRL + Z – Pauses an application
fg – Returns you to the application

fg & bg commands:
Your shell keeps a table of currently executing jobs and can be displayed with jobs command. You need to use bg command to restart a stopped background process. The fg command moves a background job in the current shell environment into the foreground.

Before you start using fg command, you need to start couple of jobs on your system for demonstration purpose. Type the following commands to start jobs:

xeyes &
gnome-calculator &
gedit fetch-stock-prices.py &

Finally, run ping command in foreground:

 ping google.com 

To suspend ping command job hit the Ctrl-Z key sequence.
Finding jobs that are running or suspended in background

@Type the following jobs command:
$ jobs -l

The output of the jobs -l command shows the following job running in the background:

[4]+ 6138 Stopped ping google.com

How do I bring a background ping command job to the foreground?

To bring a background or suspended process called ping to the foreground, enter:
$ fg %4

OR
$ fg %ping

How to Find and Kill Running Processes in Linux with pidof:

pidof firefox

Use nohup To Run Commands After You Log Out Of An SSH Session:
The nohup command is really useful if you use the ssh command to log onto other machines.
I always connect to the Raspberry PI via ssh from a laptop. If I started downloading a large file on the Raspberry PI without using the nohup command then I would have to wait for the download to finish before logging off the ssh session and before shutting down the laptop. If I did this then I may as well have not used the Raspberry PI to download the file at all.

To use nohup all I have to type is nohup followed by the command as follows:

nohup wget http://mirror.is.co.za/mirrors/linuxmint.com/iso//stable/17.1/linuxmint-17.1-cinnamon-64bit.iso&

Running A Linux Command ‘AT’ A Specific Time:
Imagine you want to run that same command at a specific point in time.
The ‘at‘ command allows you to do just that. ‘at’ can be used as follows.

at 10:25 PM sat
at> cowsay ‘hello’
at> CTRL + D

The above command will run the program cowsay (a configurable talking cow Turns text into happy ASCII cows, with speech balloons) at 10:25 PM on Saturday evening. The syntax is ‘at’ followed by the date and time to run. When the at> prompt appears enter the command you want to run at the specified time. The CTRL + D returns you to the cursor. There are lots of different date and time formats and it is worth checking the man pages for more ways to use ‘at’.

Apropos: Search man pages for keywords.

 apropos ssh 

Chage: Change days between password changes.

 chage -M 60 -W 14 keith 

Chattr: Change file attributes, even prevent root from deleting a file!

 chattr +i file1

Cksum: Check a file’s checksum value for integrity or generate a checksum value – md5sum and sha1sum commands do the same but with increased hashes and are recommended over cksum. Best to always us sha1sum or greater.

cksum file1
md5sum file1
sha1sum file1

Fsck: Check and repair a Linux file system.

 fsck /dev/sda1 

Last: Shows who logged in last.

 last 

Lastlog: Shows the recent logins of all users.

 lastlog 

Lsattr: Lists file or dir attributes

 lsattr /var/www 

Lshw: Lists hardware  of the system. * may require installing from your distros repos.

 lshw 

Pidof: Show the PID of a running process e.g. SSHD

 pidof sshd 

Testparm: Check the syntax of SAMBA configuration.

 testparm 

Vimtutor: VIM tutorial of the vi/vim editor.

Which: Displays the full path of a command, e.g. find the path to sshd executable.

 which sshd 

Who: Show who is logged in.

 who 

Watch: It will run a command repeatedly, displaying its output. This allows you to watch the program output change over time. By default, the program is running every 2 seconds.

[email protected]:~$ watch -d ls -l

CTRL+D: If you want to logout of the terminal any time quickly instead of typing exit, you can simply press Control+D key.

[email protected]:~$ CTRL+D

Last: Last logins with time and date stamps.

 last 

sudo !!:  will run the previous command
Example if you run a command that requires root access, you can simply issue “sudo !!” which will now run that same command with sudo root access.

 apt-get install apache2
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root? 
 sudo !!
sudo apt-get install apache2
Reading package lists... Done
Building dependency tree
Reading state information... Done 

NOTE: Don’t forget to add a space between the sudo and !!

Look: The look utility displays any lines in file which contain string as a prefix. If file is not specified, the file /usr/share/dict/words is used, only alphanumeric characters are compared and the case of alphabetic characters is ignored.

 look engine
engine
engine's
engineer
engineer's
engineered
engineering
engineering's
engineers
engines 

Note: if on a server you will need to install a word dictionary such as words “yum install words” on CentOS/RH or wbritish on Debian/Ubuntu “apt-get install wbritish”.

pwgen: generate pronounceable passwords

Generate a random password with upper and lower case values including numbers securely random with at least 20 characters:

 pwgen -cns1 20
hbuqHaE3pUgePGXQ2k3R 

tac: concatenate and print files in reverse

The reverse of cat if you like, here is an example of a file named cars, when using cat we see:

 cat cars
1       BMW
2       Porsche
3       Corvette
4       Ferrari
5       Mercedes Benz
6       Citroen
7       Volvo 

When using tac we see the reverse:

 tac cars
7       Volvo
6       Citroen
5       Mercedes Benz
4       Ferrari
3       Corvette
2       Porsche
1       BMW 

tree: list contents of directories in a tree-like format

Sometimes helps rather than looking at ls outputs, especially with many directories, very easy but powerful utility.

 tree
.
├── cars
├── colours
├── scripts.tar.gz
├── test1
│   └── file1
├── test2
│   └── file2
├── test3
│   └── file3
├── test4
│   └── file4
└── vimcustom.sh 

The Linux ls command: is handy to list out files, directories, permissions and links. However there is a lot more to the ls command, here some handy extended ls commands that provide greater detailed information about your Linux computer.

List block devices

lsblk

See USB devices

lsusb

View distro details

lsb_release

View CPU information

lscpu

View PCI information

lspci

Hardware details

lshw or lshw --short

Device drivers listing

lsmod

List open files

lsof

List file locks

lslk

View file attributes

lsattr

Searching and extracting data: – multiple utilities and tools

  • find – search for files in a directory hierarchy
  • grep – print lines matching a pattern
  • less – file perusal filter for crt viewing opposite of more
  • cat – concatenate files and print on the standard output
  • head – output the first part of files
  • tail – output the last part of files
  • sort – sort lines of text files
  • cut – remove sections from each line of files
  • wc – print newline, word, and byte counts for each file

CAT – the cat command:

Display multiple files using cat

cat file.txt logs.log

Display contents of file with line numbers:

cat -n file.txt

Copy the contents of one file to another file:

cat file.txt > text.txt

Suppress repeated empty lines:

cat -s file.txt

SED – the powerful sed command

Use sed to find and replace text in files:

Use s/ to substitute the found expression mac with linux as follows:

sed 's/mac/linux/g' system.txt

To update file pass the -i option:

sed -i 's/mac/linux/g' system.txt

/g is global and is often used in VIM when replacing all instances, it means all matches
will be replaced (Globally). /gI will catch all cases e.g. uppercase and lowercase.

sed -i 's/mac/linux/gI' system.txt

uniq – The Linux utility “uniq” can report or delete repeated lines in a file.

cat file.txt
Usage: uniq [OPTION]… [INPUT [OUTPUT]]
Usage: uniq [OPTION]… [INPUT [OUTPUT]]
Filter adjacent matching lines from INPUT (or standard input),
Filter adjacent matching lines from INPUT (or standard input),
writing to OUTPUT (or standard output).
writing to OUTPUT (or standard output).

Delete repeated lines using uniq

uniq file.txt

Usage: uniq [OPTION]… [INPUT [OUTPUT]]
Filter adjacent matching lines from INPUT (or standard input),
writing to OUTPUT (or standard output).

display number of repetitions for each line

uniq -c file.txt

Avoid comparing set number of initial characters
skip the first 4 characters in each line before comparing

uniq -s 4 file.txt

case insensitive

uniq -i file.txt

There are hundreds of different examples that can be made for the above utilities, read the man pages and play around with them on a test system or Google the command with a specific function that you want to achieve.

ddrescue and ddrescue-GUI

ddrescue and ddrecue-GUI for data recovery or to retrieve corrupted data.

ddrescue:

GNU ddrescue is a data recovery tool. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying to rescue the good parts first in case of read errors. Ddrescuelog is a tool that manipulates ddrescue mapfiles, shows mapfile contents, converts mapfiles to/from other formats, compares mapfiles, tests rescue status, and can delete a mapfile if the rescue is done. Ddrescuelog operations can be restricted to one or several parts of the mapfile if the domain setting options are used. The basic operation of ddrescue is fully automatic. That is, you don’t have to wait for an error, stop the program, restart it from a new position, etc.

Install into Ubuntu / Debian:

 sudo apt-get install gddrescue 

More info and Download:
https://www.gnu.org/software/ddrescue/

ddrescue-GUI:

DDRescue-GUI is a program designed to make it easier to use GNU ddrescue (A Command-Line data recovery tool). It provides a simple graphical method for using ddrescue. This is designed to be as user-friendly as possible so users new to Linux can use ddrescue easily.

DDRescue-GUI is a simple GUI written in Python 2 designed to make the data recovery tool, ddrescue, easier for beginners to use. It’s designed for Linux, and more recently Apple OS X, as KDiskRescue appears to be abandoned, with the last update in 2006. DDRescue-GUI is desgined to look native on almost all Desktop Environments, so whichever one you use, it should look familiar.

More info, download and install:
https://launchpad.net/ddrescue-gui

 

Last logon time of user in Windows

Using ‘Net user’ command we can find the last login time of a user. The exact command is given below.

 net user username | findstr /B /C:"Last logon" 

Example:
To find the last login time of the computer administrator

 C:\> net user administrator | findstr /B /C:"Last logon"
Last logon 6/30/2010 10:02 AM
C:> 

For a domain user, the command would be as below.

 C:\>net user john /domain | findstr /C:"Last logon"
Last logon 9/18/2013 10:18:41 AM 

Reference:
[[http://www.windows-commandline.com/last-logon-time-of-user/]]

Test the spammyness of your email

As the name suggests this free service will test the spammyness of your email – https://www.mail-tester.com/

From their FAQ:

We’re the guys from MailPoet and AcyMailing.

We needed a cheap, simple and efficient way to quickly test the quality of our own newsletters.

We simply built on our own tool. Now we’re sharing it for free via our web-interface and enable you to include our tests in your own app and whitelist our service by creating an account.

We’re geeky email software engineers. We can let you imagine how such people look like.

https://www.mail-tester.com/

How To Verify SSL Certificate From A Shell Prompt

How do I verify and diagnosis SSL certification installation from a Linux / UNIX shell prompt? How do I validate SSL Certificate installation and save hours of troubleshooting headaches without using a browser? How do I confirm I’ve the correct and working SSL certificates?

http://www.cyberciti.biz/faq/test-ssl-certificates-diagnosis-ssl-certificate/

Nginx Error_log Configuration

Nginx serves millions of applications on a daily basis, and those millions of apps generate errors. Guess what? So do yours. Some of them are critical, while others are just notices. The important thing here is to get access to those errors to debug and have a clear view of what’s happening with your web applications.

In this article, you will explore the Nginx error_log configuration and how Nginx handles the types of errors that can be logged into the files.

http://www.nginxtips.com/nginx-error_log-configuration/

Backup and Restore permissions in Linux

You can use getfacl and setfacl to take a backup of permissions from a directory and then restore those permissions back if required at a later date.

The following directory of /perms contains these permissions:

/perms/
|-- [-rw-r--r--] permfile1.txt
|-- [-rw-r--r--] permfile2.txt
|-- [drwxr-xr-x] subperm1
| |-- [-rwxr-xr-x] permfile1.txt
| `-- [-rw-r--r--] permfile2.txt
`-- [drwxr-xr-x] subperm2
|-- [-rw-r--r--] permfile1.txt
`-- [-rwxr-xr-x] permfile2.txt

To Backup this directory and sub-directory permissions you can run:

# getfacl -R /perms > perms_bu

The contents of the perms_bu file will contain all the permission details for both directories and files. Should any file or directory permissions be changed or modified in the future, we can restore as show below with the perms_bu file

To restore the permissions:

# setfacl --restore=perms_bu

NOTE: When you restore the permissions be sure you are in the relevant path area e.g. for the above example run the restore from the root area /

Reference:
Linux Man pages or http://linux.die.net/man/1/getfacl