Colorize log files with ccze tool

This is CCZE, a fast log colorizer written in C, intended to be a
drop-in replacement for colorize (http://colorize.raszi.hu).

CCZE is a robust and modular log colorizer with plugins for apm, exim, fetchmail, httpd, postfix, procmail, squid, syslog, ulogd, vsftpd, xferlog, and more.

Installation:

# yum install ccze (Red Hat/CentOS)
# apt-get install ccze (Debian/Ubuntu)

Usage:

tail -f -n 50 /var/log/syslog | ccze

Output is really nice as shown here:

blog-cczetool

 

 

 

 

 

Reference:
https://github.com/cornet/ccze
http://freecode.com/projects/ccze/

 

Put /tmp on tmpfs

On Dustin Kirkland’s blog http://blog.dustinkirkland.com/2016/01/data-driven-analysis-tmp-on-tmpfs.html he wrote up a piece on running /tmp on tmpfs and I’m sold on it!

He writes:
– Put /tmp on tmpfs and you’ll improve your Linux system’s I/O, reduce your carbon foot print and electricity usage, stretch the battery life of your laptop, extend the longevity of your SSDs, and provide stronger security.
– In fact, we should do that by default on Ubuntu servers and cloud images.
– Having tested 502 physical and virtual servers in production at Canonical, 96.6% of them could immediately fit all of /tmp in half of the free memory available and 99.2% could fit all of /tmp in (free memory + free swap).

The benefits of /tmp on tmpfs are:

  • Performance: reads, writes, and seeks are insanely fast in a tmpfs; as fast as accessing RAM
  • Security: data leaks to disk are prevented (especially when swap is disabled), and since /tmp is its own mount point, we should add the nosuid and nodev options (and motivated sysadmins could add noexec, if they desire).
  • Energy efficiency: disk wake-ups are avoided
  • Reliability: fewer NAND writes to SSD disks

To enable it is a simple:

$ echo "tmpfs /tmp tmpfs rw,nosuid,nodev" | sudo tee -a /etc/fstab
$ sudo reboot

More info at the link above

Collecting Ubuntu Linux System Information

For new computer or Laptop or server, I need to collect the information about its hardware. This is also useful when you need to replace a disk or memory with a vendor. In order to replace hardware you need all information in advance.

Display the system’s host name:

$ hostname

Display the system’s DNS domain name:

$ dnshostname

 

Find the system serial number, manufacturer of the system and model name:

$ sudo dmidecode -s system-serial-number
$ sudo dmidecode -s system-manufacturer
$ sudo dmidecode -s system-product-name
$ sudo dmidecode | more

OR use the lshw command:

# lshw | more
$ sudo lshw -short

Display information about installed hardware

$ sudo lsdev

Find the system CPU info

$ cat /proc/cpuinfo

Find the system main memory (RAM) info. Show statistics about memory usage on the system including total installed and used RAM:

$ less /proc/meminfo

Find the Ubuntu Linux distribution version and related information:

$ lsb_release -a

Find the system kernel architecture (32 bit or 64 bit):

$ uname -m
$ getconf LONG_BIT
$ arch

Show all installed disks and size:

# fdisk -l | grep '^Disk /dev'

Display SCSI devices (or hosts) and their attributes on Linux:

$ lsscsi

Find the system PCI devices information:

$ lspci
$ lspci -vt
$ lspci | grep -i 'something'
$ lspci -vvvn| less

Find the system USB devices information:

$ lsusb
$ lsusb -vt

Find the system Wireless devices information:

$ iwconfig
$ watch -n 1 cat /proc/net/wireless
$ wavemon

Find the system audio devices information:

$ lspci | grep -i audio

Display the system drivers (modules):

$ sudo lsmod
$ sudo modinfo {driver_name}
$ sudo modinfo kvm

Display the list of running services:
### SYS V ###

$ sudo service --status-all

OR
## UPSTART ##

$ sudo initctl list

Find out if service is enabled:
## UPSTART ##

$ sudo initctl status service-name
$ sudo initctl status smbd

OR
## SYS V

$ sudo service serviceName status
$ sudo service nginx status

 

mtr utility

Traditionally the traceroute (print the route packets take to network host) and ping (send ICMP ECHO_REQUEST to network hosts) programs are used as diagnostic tool to solve and isolate networking errors. It may take some time to use both tools to diagnose network issues. However, you can use the mtr program instead of ping and traceroute. It is a network diagnostic tool and it is the combination of traceroute and ping programs (in terms of functionality) and works as a single network diagnostic tool.

Once mtr invoked it starts investigates the network connection between the hosts (workstation) mtr runs on and HOSTNAME by sending packets with purposely low TTLs (time to live). It will continue to send packets with low TTL, noting the response time of the intervening routers. This allows mtr to print the response percentage and response times of the internet route to HOSTNAME.

During this run if you notice a sudden increase in packet-loss or response time is an indication of overloaded link or a bad link.

Examples:

mtr -c 5 -r -w ausinfotech.net

http://www.cyberciti.biz/tips/finding-out-a-bad-or-simply-overloaded-network-link-with-linuxunix-oses.html

Test Internet connection speed with iperf in Linux

How do I test my Internet connection download speed from the console over the ssh session without using HTML5 or Adobe flash/Java applets based websites? How do I test my upload speed from the console?

You can use iperf utility for Linux – see https://iperf.fr/ iPerf3 is a tool for active measurements of the maximum achievable bandwidth on IP networks. It supports tuning of various parameters related to timing, buffers and protocols (TCP, UDP, SCTP with IPv4 and IPv6). For each test it reports the bandwidth, loss, and other parameters. This is a new implementation that shares no code with the original iPerf and also is not backwards compatible. iPerf was orginally developed by NLANR/DAST. iPerf3 is principally developed by ESnet / Lawrence Berkeley National Laboratory. It is released under a three-clause BSD license.

See tutorial at Nixcraft site http://www.cyberciti.biz/faq/linux-unix-test-internet-connection-download-upload-speed/

 

Creating NGINX Rewrite Rules

How to create NGINX rewrite rules (the same methods work for both NGINX Plus and the open source NGINX software). Rewrite rules change part or all of the URL in a client request, usually for one of two purposes:

  • To inform clients that the resource they’re requesting now resides at a different location. Example use cases are when your website’s domain name has changed, when you want clients to use a canonical URL format (either with or without the www prefix), and when you want to catch and correct common misspellings of your domain name. The return and rewrite directives are suitable for these purposes.
  • To control the flow of processing within NGINX and NGINX Plus, for example to forward requests to an application server when content needs to be generated dynamically. The try_files directive is often used for this purpose.

Example of a redirect to a new domain name:

<code>server { listen 80; listen 443 ssl; server_name www.old-name.com; return 301 $scheme://www.new-name.com$request_uri; }

Read more details and examples from the Nginx blog post: https://www.nginx.com/blog/creating-nginx-rewrite-rules/

 

Email Checker

Email Checker is a simple tool for verifying an email address. It’s free and quite easy to use. Just enter the email address and hit check button. Then it tells you whether the email address is real or not. It extracts the MX records from the email address and connect to mail server (over SMTP and also simulates sending a message) to make sure the mailbox really exist for that user/address. Some mail servers do not co-operate in the process, in such cases, the result of this email verification tool may not be accurate as expected.

http://email-checker.net/

 

Linux Troubleshooting

Bootup Issues won’t Boot

Fix grub:
=> check if grub exists: windows installed or wiped from MBR etc – press either Esc or shift after bios boot to see if grub exits.

=> grub prompt: may be corrupt or something missing in the config lines, check another instance on a different computer and see if typing those commands can manually boot the system.

=> Misconfigured prompt: try an older entry in the grub menu to see if it’s an error or new kernel

=> Check /etc/defualt/grub for correct syntax or errors. Run update-grub after file change to update /boot/grub/grub.cfg

=> Repair grub with rescue disk – Ubuntu install disk has a rescue mode option by default that can reinstall grub etc.

=> Can’t mount file system – Check root and partitions uuid labels or the disk label

Disk full can’t write to disk

=> Track down largest directories with:

 du -ckx | sort -n > /tmp/durpt.log 

Other DU examples:

du -sh
du -ckhx | sort -n
du -ah | grep M |sort -n
du -ckxh /var/ | sort -n |grep M
du -a /var | sort -n -r | head -n 10
du -hsx * | sort -rh | head -10

Some find commands to use for disk space usage:

find / -xdev -type f -size +50M
find / -xdev -type f -size +50M -exec ls -alh {} \; | sort -nk 5
find /usr -type f -printf "%s %p\n" | sort -rn | head -n 10

Then tail the log file and check the largest sizes to investigate.

=> Check /var/log for oversized log files and rotate or truncate if possible.

=> Check /tmp for large files .swp files caused by another process or user leaving a log file opened with vim. Use lsof to find the pid and kill that process which in turn will free the disk space. This is often when df -h and du -h provide different results.

=> Out of INODES – As per above if du and df show different results you could be out of inodes. df -i to check free inodes or inodes usage. Check for large amount if small files and see if they can be moved. tar them etc

=> Error that file system is read-only – first see if a remount with rw will work with:

 sudo mount -o remount,rw /home 

Could be due to error on boot and the system is protecting itself, check /var/log/dmesg for mount errors.

=> Check file system with fsck – unmount the system first and run fsck -y -C /dev/sda3 as an example. Also can superblocks check with mke2fs commands.

Website is down

=> Check ports are open e.g. port 80 443 or special app ports 8000 for splunk Check the port is available with either telnet ot nmap and if a firewall/network issue is preventing the access. Then check the netstat command to see if the service is running correctly on the web server itself.

telnet webserver 80
nmap -v -p 80 webserver
netstat -lnp | grep :80

=> Test the webserver via curl

 curl http://webserver.com 

Check for http status code with curl:

curl -w "%{http_code}\n" http://www.example.net

HTTP STATUS CODES

List of most important HTTP Status Codes, before you get into the list, you must know the 4 important categories:

  • Success Codes (2xx)
  • Redirection Codes (3xx)
  • Client Side Error Codes (4xx)
  • Server Side Error Codes (5xx)

1xx: Information:
100     Continue
101     Switching Protocol

Successful responses:
200     OK
201     Created
202     Accepted
203     Non-Authoritative Information
204     No Content
205     Reset Content
206     Partial Content

Redirection messages:
300     Multiple Choice
301     Moved Permanently
302     Found
303     See Other
304     Not Modified
305     Use Proxy
306     unused
307     Temporary Redirect
308     Permanent Redirect

Client error responses:
400     Bad Request
401     Unauthorized
402     Payment Required
403     Forbidden
404     Not Found
405     Method Not Allowed
406     Not Acceptable
407     Proxy Authentication Required
408     Request Timeout
409     Conflict
410     Gone
411     Length Required
412     Precondition Failed
413     Request Entity Too Large
414     Request-URI Too Long
415     Unsupported Media Type
416     Requested Range Not Satisfiable
417     Expectation Failed

Server error responses:
500     Internal Server Error
501     Not Implemented
502     Bad Gateway
503     Service Unavailable
504     Gateway Timeout
505     HTTP Version Not Supported

WebServer Configuration Tests:

Apache2 - apache2ctl configtest
Nginx    - nginx -t

GENERAL

sudo dd if=/dev/mem | cat | strings

cat /proc/meminfo

cat /proc/cpuinfo

lspci

lsusb

fdisk -l

Kill a process ps -A | grep ProgramName kill 7207

List all files that are currently open on the system with “lsof”. This command will allow you to see all the files that are currently open on your system. Limiting the directory or coupling this command with grep is often useful for finding files that are still open restricting the ability to unmount a device. Lsof will also ouput the process id or PID. You can then kill the process using the kill command above.

 lsof 

Keep an eye on something for awhile – watch

The watch command will repeat a command at a set interval (default 2 seconds) and output the response. This is useful for watching directories that change, watching hard drives fill up when a lot of data is being transfered, or using it with lsusb to watch for USB device being plugged in.

watch ls
watch df -h

Find where a binary is stored and its libraries Often times when running a cron command you want to include the absolute path to the command. Sometimes I run scheduled PHP tasks. This can be acomplished by using the ëwhereisë command.

whereis php5

See if you have kernel boot issues dmesg | less

For more logs just cd into the /var/log directory and start using, cat, less, tail, grep, find or any other tool to view and search.