Linux du command alternatives

For years I’ve used ncdu a NCurses Disk Usage utility for Linux. Recently someone alerted me to some other options as well as ncdu:

Dust:
du + rust = dust. Like du but more intuitive, Dust is meant to give you an instant overview of which directories are using disk space without requiring sort or head. Dust will print a maximum of 1 ‘Did not have permissions message’. Dust will list the 20 biggest sub directories or files and will smartly recurse down the tree to find the larger ones. There is no need for a ‘-d’ flag or a ‘-h’ flag. The largest sub directory will have its size shown in red.

https://github.com/bootandy/dust

The Tin Summer:
sn is a replacement for du. It has nicer output, saner commands and defaults, and it even runs faster on big directories thanks to multithreading.

https://github.com/vmchale/tin-summer

NCDU:
Ncdu is a disk usage analyzer with an ncurses interface. It is designed to find space hogs on a remote server where you don’t have an entire graphical setup available, but it is a useful tool even on regular desktop systems. Ncdu aims to be fast, simple and easy to use, and should be able to run in any minimal POSIX-like environment with ncurses installed.
Install from repos: sudo apt install ncdu

man page: https://dev.yorhel.nl/ncdu/man

 

tcpdump101.com – generate tcpdump commands

tcpdump101.com is a great site that you can use to generate tcpdump commands, you enter the parameters it’s asks for and it will generate the command for you. It’s handy if you are not running tcpdump commands very often and then have to either look up the help/man pages or Google for the command switches you want. It also has output for Cisco and Checkpoint firewalls.

From there site they say… tcpdump101.com has been designed to help people capture packets on different devices to assist with network troubleshooting, service troubleshooting and even passive red team activities. There is an assumption that the user has a basic understanding of what they want to capture – As much as this is a tool to help people, the user has to use their own logic since every situation is different. That being said, I strongly suggest that if you’re just starting out with packet captures to grab a copy of Virtual Box and play around with Linux and tcpdump. Although tcpdump may not be what you ultimately use, it will give you an excellent understanding of what you’ll see, even with other products and vendors.

As a safety measure (if at all possible) make sure to set a capture limit on your PCaps. If you make a mistake building your filters, you may end up captuing a lot of traffic. Although the odds are slim, there is a chance that your PCap could fill the NIC buffer and start dropping packets. The worst-case scenario is that it runs out of memory while you’re logged in remotely. With today’s hardware, it most likely won’t happen however you should always expect the best and plan for the worst.

 

 

 

 

Reference: tcpdump101.com

lnav – Log File Navigator

The Log File Navigator

Watch and analyze your log files from a terminal with lnav http://lnav.org/ for Linux and Mac. Just like CCZE https://ausinfotech.net/blog/colorize-log-files-with-ccze-tool/ lnav can produce easy readable logs in colour and also highlight important parts of the logs.

Some Features:

Single Log View
All log file contents are merged into a single view based on message timestamps. You no longer need to manually correlate timestamps across multiple windows or figure out the order in which to view rotated log files. The color bars on the left-hand side help to show which file a message belongs to.

Automatic Log Format Detection
The following formats are built in by default:

  • Common Web Access Log format
  • CUPS page_log
  • Syslog
  • Glog
  • VMware ESXi/vCenter Logs
  • dpkg.log
  • uwsgi
  • “Generic” – Any message that starts with a timestamp
  • Strace
  • sudo

Installation:

See http://lnav.org/downloads for details and/or in Linux Debian/Ubuntu run:

 sudo apt install lnav 

Example:

 

 

netplan – Ubuntu 18.04

Netplan
The network configuration abstraction renderer

Netplan is a utility for easily configuring networking on a linux system. You simply create a YAML description of the required network interfaces and what each should be configured to do. From this description Netplan will generate all the necessary configuration for your chosen renderer tool.

The way you configure a network interface in Ubuntu 18.04 LTS is completely different than the previous Ubuntu 16.04 LTS, 18.04 uses a new methodology with a new tool called Netplan. In fact 17.10 already had this netplan tool, however I didn’t notice this until setting up an 18.04 server for the first time in a DMZ area with no DHCP. This new tool replaces the static interfaces (/etc/network/interfaces) now you must use /etc/netplan/*.yaml to configure Ubuntu interfaces – yes yaml files!

How does it work?
Netplan reads network configuration from /etc/netplan/*.yaml which are written by administrators, installers, cloud image instantiations, or other OS deployments. During early boot, Netplan generates backend specific configuration files in /run to hand off control of devices to a particular networking daemon.

How to configure it?
To configure netplan, save configuration files under /etc/netplan/ with a .yaml extension (e.g. /etc/netplan/config.yaml), then run sudo netplan apply. This command parses and applies the configuration to the system. Configuration written to disk under /etc/netplan/ will persist between reboots.

DHCP and static addressing
To let the interface named ‘enp3s0’ get an address via DHCP, create a YAML file with the following:

DHCP:

network:
version: 2
renderer: networkd
ethernets:
enp3s0:
dhcp4: true

Now run this command to apply it:

sudo netplan apply

Set a static IP address:

network:
version: 2
renderer: networkd
ethernets:
enp3s0:
addresses:
- 10.10.10.2/24
gateway4: 10.10.10.1
nameservers:
search: [mydomain, otherdomain]
addresses: [10.10.10.1, 1.1.1.1]

Now run this command to apply it:

sudo netplan apply

References:
https://netplan.io/

 

INCXI – command line sys info tool

inxi is a full featured CLI system information tool. It is available in most Linux distribution repositories, and also runs somewhat on BSDs.

Get the latest version from Github see below, or install from distro package e.g.

sudo apt install inxi

then simply run inxi

inxi

$ inxi
CPU~Single core Intel Xeon E5-2670 v2 (-MCP-) speed~2494 MHz (max) Kernel~4.4.0-116-generic x86_64 Up~22 days Mem~336.5/990.4MB HDD~12.9GB(36.9% used) Procs~146 Client~Shell inxi~2.2.35

Reference:

http://smxi.org/docs/inxi.htm
https://github.com/smxi/inxi

 

 

apt vs apt-get

I use apt all the time now, even on 14.04 Ubuntu servers (except for apt autoremove) and from 16.04 up I never touch apt-get. What’s the main difference, just Google it and you will found out specific details, for a quick run down read this below.

From the man page:

DIFFERENCES TO APT-GET(8)
The apt command is meant to be pleasant for end users and does not need
to be backward compatible like apt-get(8). Therefore some options are
different:

· The option DPkg::Progress-Fancy is enabled.
· The option APT::Color is enabled.
· A new list command is available similar to dpkg –list.
· The option upgrade has –with-new-pkgs enabled by default.

 

Here is a table outline:

Function apt-get apt
Install package apt-get install <package> apt install <package>
Remove package apt-get remove <package> apt remove <package>
Remove package including configuration apt-get purge <package> apt purge <package>
Update sources apt-get update apt update
Update packages (without removing or reinstalling) apt-get upgrade apt upgrade1
Update packages (with removing and reinstalling) apt-get dist-upgrade apt full-upgrade
Remove unnecessary dependencies apt-get autoremove apt autoremove
Search package apt-get search <package> apt search <package>
Display package information apt-cache show <package> apt show <package>
Display active package sources in detail apt-cache policy apt policy
Display available and installed package versions apt-cache policy <package> apt policy <package>
New Commands
Edit packages sources apt edit-sources
List packages by criteria dpkg –get-selections > list.txt apt list
Set/change package status echo <package> hold | dpkg –set-selections apt-mark <package>

Xenserver does not reclaim space after deleting VDI or Snapshot

In most cases removing old snapshots and then initiating a scan of the SR has resolved this for me.

vhd-util scan -f -m “VHD-*” -l “VG_XenStorage-<uuid_of_SR>” –p

References:

https://sysadmino.wordpress.com/2014/03/03/xenserver-does-not-reclaim-space-after-deleting-vdi-or-snapshot/

https://techblog.jeppson.org/2015/02/reclaim-lost-space-xenserver-6-5/

 

NetSPI – SQL Injection Wiki

https://sqlwiki.netspi.com

This wiki’s mission is to be a one stop resource for fully identifying, exploiting, and escalating SQL injection vulnerabilities across various Database Management Systems (DBMS).

Below is an outline of the wiki’s structure, laid out in the order of a normal escalation path. Certain queries may be version specific.

Step 1: Injection Detection
Step 2: DBMS Identification
Step 3: Injection Types
Step 4: Injection Techniques
Step 5: Attack Queries

 

PDFsam – OpenSource PDF Utility

PDFsam https://pdfsam.org/ is a free and open source desktop application to split, merge, extract pages, rotate and mix PDF files.

There are 3 products, PDFsam Basic, PDFsam Enhanced and PDFsam Visual:

PDFsam Basic

  • Merge
    Merge PDF files, select the pages, merge bookmarks and interactive forms
  • Split
    Split a PDF file at given page numbers, at given bookmarks level or in files of a given size
  • Extract from PDF
    Extract pages from PDF files
  • Rotate
    Rotate PDF files, every page or just the selected pages
  • Mix
    Merge PDF files together taking pages alternatively from one and the other. The perfect tool if you have a single-sided scanner

PDFsam Enhanced

  • Edit
    Modify the PDF content without the need to export it or copy to another format
  • Insert
    Insert pages or hyperlinks and update page numbers once you are done
  • Convert
    Convert PDF files to a number of popular formats like doc, docx, xls, xlsx and many others
  • Review
    Collaborate with your peers by adding notes, highlighting text and drawing on your document with the pencil
  • Forms
    Create fillable forms from scratch or pre designed, import and export data, add actions to print the document, send it by email and other
  • Security
    Protect PDF files with 256 bits AES encryption, set permissions on them and add a digital signature
  • OCR
    An Optical Character Recognition module to extract text from images

PDFsam Visual

  • Combine and Reorder
    Visually reorder pages, move them, rotate them or combine PDF files dragging and dropping pages from multiple documents
  • Split by text
    Split PDF files at pages where text in a selected area changes
  • Crop
    Crop PDF files by drawing the rectangular area you want to keep
  • Split
    Split PDF files visually selecting pages to split at, or split at given bookmarks level or in files of a given size
  • Protect and Unprotect
    Add permissions and encrypt PDF files using AES 256bits or 128bits. Decrypt PDF files supplying a password and create an unprotected version
  • Extract
    Visually select the pages you want to extract from a PDF file
  • More
    Mix PDF files, Rotate PDF files, Merge PDF files…

 

Amazon Time Sync Service

The Amazon Time Sync Service provides a highly accurate and reliable time reference that is natively accessible from Amazon EC2 instances.

Built on Amazon’s proven network infrastructure, the service utilises a fleet of redundant satellite-connected and atomic reference clocks in AWS regions to deliver current time readings of the Coordinated Universal Time (UTC) global standard. The service is designed to be highly available with a continuously monitored time infrastructure and provides a low variance reference time source. Leap seconds are known to cause application errors, and can be a concern for developers and system administrators. The Amazon Time Sync Service automatically smooths out (smears) leap seconds that are periodically added to UTC, so that customers do not have to worry about application errors due to their addition.

Setting the Time for Your Linux Instance

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html

The Amazon Time Sync Service is available through NTP at the 169.254.169.123 IP address for any instance running in a VPC. Your instance does not require access to the internet, and you do not have to configure your security group rules or your network ACL rules to allow access. Use the following procedures to configure the Amazon Time Sync Service on your instance using the chrony client.

To configure your Ubuntu or Debian derivative instance to use the Amazon Time Sync Service

Connect to your instance and use apt to install the chrony package.

ubuntu:~$ sudo apt install chrony 
Note:
If necessary, update your instance first by running sudo apt update.

Open the /etc/chrony/chrony.conf file using a text editor (such as vim or nano). Add the following line after any other server or pool statements that are already present in the file, and save your changes:

 server 169.254.169.123 prefer iburst 

Restart the chrony service.

 ubuntu:~$ sudo /etc/init.d/chrony restart
[ ok ] Restarting chrony (via systemctl): chrony.service.

Verify that chrony is using the 169.254.169.123 IP address to synchronize the time.

 ubuntu:~$ chronyc sources -v
210 Number of sources = 7

.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 169.254.169.123 3 6 17 12 +15us[ +57us] +/- 320us
^- tbag.heanet.ie 1 6 17 13 -3488us[-3446us] +/- 1779us
^- ec2-12-34-231-12.eu-west- 2 6 17 13 +893us[ +935us] +/- 7710us
^? 2a05:d018:c43:e312:ce77:6 0 6 0 10y +0ns[ +0ns] +/- 0ns
^? 2a05:d018:d34:9000:d8c6:5 0 6 0 10y +0ns[ +0ns] +/- 0ns
^? tshirt.heanet.ie 0 6 0 10y +0ns[ +0ns] +/- 0ns
^? bray.walcz.net 0 6 0 10y +0ns[ +0ns] +/- 0ns 

In the output that’s returned, ^* indicates the preferred time source. Verify the time synchronization metrics that are reported by chrony.

 ubuntu:~$ chronyc tracking
Reference ID : 169.254.169.123 (169.254.169.123)
Stratum : 4
Ref time (UTC) : Wed Nov 29 07:41:57 2017
System time : 0.000000011 seconds slow of NTP time
Last offset : +0.000041659 seconds
RMS offset : 0.000041659 seconds
Frequency : 10.141 ppm slow
Residual freq : +7.557 ppm
Skew : 2.329 ppm
Root delay : 0.000544 seconds
Root dispersion : 0.000631 seconds
Update interval : 2.0 seconds
Leap status : Normal 

Setting the Time for a Windows Instance

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/windows-set-time.html