lnav – Log File Navigator

The Log File Navigator

Watch and analyze your log files from a terminal with lnav http://lnav.org/ for Linux and Mac. Just like CCZE https://ausinfotech.net/blog/colorize-log-files-with-ccze-tool/ lnav can produce easy readable logs in colour and also highlight important parts of the logs.

Some Features:

Single Log View
All log file contents are merged into a single view based on message timestamps. You no longer need to manually correlate timestamps across multiple windows or figure out the order in which to view rotated log files. The color bars on the left-hand side help to show which file a message belongs to.

Automatic Log Format Detection
The following formats are built in by default:

  • Common Web Access Log format
  • CUPS page_log
  • Syslog
  • Glog
  • VMware ESXi/vCenter Logs
  • dpkg.log
  • uwsgi
  • “Generic” – Any message that starts with a timestamp
  • Strace
  • sudo

Installation:

See http://lnav.org/downloads for details and/or in Linux Debian/Ubuntu run:

 sudo apt install lnav 

Example:

 

 

netplan – Ubuntu 18.04

Netplan
The network configuration abstraction renderer

Netplan is a utility for easily configuring networking on a linux system. You simply create a YAML description of the required network interfaces and what each should be configured to do. From this description Netplan will generate all the necessary configuration for your chosen renderer tool.

The way you configure a network interface in Ubuntu 18.04 LTS is completely different than the previous Ubuntu 16.04 LTS, 18.04 uses a new methodology with a new tool called Netplan. In fact 17.10 already had this netplan tool, however I didn’t notice this until setting up an 18.04 server for the first time in a DMZ area with no DHCP. This new tool replaces the static interfaces (/etc/network/interfaces) now you must use /etc/netplan/*.yaml to configure Ubuntu interfaces – yes yaml files!

How does it work?
Netplan reads network configuration from /etc/netplan/*.yaml which are written by administrators, installers, cloud image instantiations, or other OS deployments. During early boot, Netplan generates backend specific configuration files in /run to hand off control of devices to a particular networking daemon.

How to configure it?
To configure netplan, save configuration files under /etc/netplan/ with a .yaml extension (e.g. /etc/netplan/config.yaml), then run sudo netplan apply. This command parses and applies the configuration to the system. Configuration written to disk under /etc/netplan/ will persist between reboots.

DHCP and static addressing
To let the interface named ‘enp3s0’ get an address via DHCP, create a YAML file with the following:

DHCP:

network:
version: 2
renderer: networkd
ethernets:
enp3s0:
dhcp4: true

Now run this command to apply it:

sudo netplan apply

Set a static IP address:

network:
version: 2
renderer: networkd
ethernets:
enp3s0:
addresses:
- 10.10.10.2/24
gateway4: 10.10.10.1
nameservers:
search: [mydomain, otherdomain]
addresses: [10.10.10.1, 1.1.1.1]

Now run this command to apply it:

sudo netplan apply

References:
https://netplan.io/

 

INCXI – command line sys info tool

inxi is a full featured CLI system information tool. It is available in most Linux distribution repositories, and also runs somewhat on BSDs.

Get the latest version from Github see below, or install from distro package e.g.

sudo apt install inxi

then simply run inxi

inxi

$ inxi
CPU~Single core Intel Xeon E5-2670 v2 (-MCP-) speed~2494 MHz (max) Kernel~4.4.0-116-generic x86_64 Up~22 days Mem~336.5/990.4MB HDD~12.9GB(36.9% used) Procs~146 Client~Shell inxi~2.2.35

Reference:

http://smxi.org/docs/inxi.htm
https://github.com/smxi/inxi

 

 

apt vs apt-get

I use apt all the time now, even on 14.04 Ubuntu servers (except for apt autoremove) and from 16.04 up I never touch apt-get. What’s the main difference, just Google it and you will found out specific details, for a quick run down read this below.

From the man page:

DIFFERENCES TO APT-GET(8)
The apt command is meant to be pleasant for end users and does not need
to be backward compatible like apt-get(8). Therefore some options are
different:

· The option DPkg::Progress-Fancy is enabled.
· The option APT::Color is enabled.
· A new list command is available similar to dpkg –list.
· The option upgrade has –with-new-pkgs enabled by default.

 

Here is a table outline:

Function apt-get apt
Install package apt-get install <package> apt install <package>
Remove package apt-get remove <package> apt remove <package>
Remove package including configuration apt-get purge <package> apt purge <package>
Update sources apt-get update apt update
Update packages (without removing or reinstalling) apt-get upgrade apt upgrade1
Update packages (with removing and reinstalling) apt-get dist-upgrade apt full-upgrade
Remove unnecessary dependencies apt-get autoremove apt autoremove
Search package apt-get search <package> apt search <package>
Display package information apt-cache show <package> apt show <package>
Display active package sources in detail apt-cache policy apt policy
Display available and installed package versions apt-cache policy <package> apt policy <package>
New Commands
Edit packages sources apt edit-sources
List packages by criteria dpkg –get-selections > list.txt apt list
Set/change package status echo <package> hold | dpkg –set-selections apt-mark <package>

Xenserver does not reclaim space after deleting VDI or Snapshot

In most cases removing old snapshots and then initiating a scan of the SR has resolved this for me.

vhd-util scan -f -m “VHD-*” -l “VG_XenStorage-<uuid_of_SR>” –p

References:

https://sysadmino.wordpress.com/2014/03/03/xenserver-does-not-reclaim-space-after-deleting-vdi-or-snapshot/

https://techblog.jeppson.org/2015/02/reclaim-lost-space-xenserver-6-5/

 

ipcalc – bash IP Calculator

ipcalc takes an IP address and netmask and calculates the resulting broadcast, network, Cisco wildcard mask, and host range. By giving a second netmask, you can design subnets and supernets. It is also intended to be a teaching tool and presents the subnetting results as easy-to-understand binary values.

Enter your netmask(s) in CIDR notation (/25) or dotted decimals (255.255.255.0). Inverse netmasks are recognized. If you omit the netmask ipcalc uses the default netmask for the class of your network. Look at the space between the bits of the addresses: The bits before it are the network part of the address, the bits after it are the host part. You can see two simple facts: In a network address all host bits are zero, in a broadcast address they are all set.

 

 

 

ipcalc is available from most Linux distro’s repo’s and from brew on mac’s.

Reference:
man ipcalc
http://jodies.de/ipcalc

httpstat

httpstat visualizes curl statistics in a way of beauty and clarity.

You can use to measure the speed of your webserver and see how long it takes to load up static or dynamic html pages. It’s a Python script that can be installed onto any Linux system with Python installed and also available with brew on Macs.

 

Installation

There are three ways to get httpstat:

Usage

python httpstat.py ausinfotech.net

python httpstat.py <a href="https://ausinfotech.net/blog/anchore-container-security-analysis/">https://ausinfotech.net/blog/anchore-container-security-analysis/</a> 

Reference:
https://github.com/reorx/httpstat

 

Anchore – Container Security Analysis

Anchore provides you with insight and control over the contents of your containers from the start of development all the way to production. Anchore delivers container security solutions for developers, operations, and security teams to deliver insight and control over the contents, security and compliance of containers from the start of development all the way to production. By allowing the creation of policies for security and compliance that are evaluated by Anchore at each stage of the build pipeline, Anchore ensures that only images containers that adhere to an organisation’s policies are deployed.

What’s Inside Your Container Images?
With Docker and containers it’s never been easier to deploy and run any application. Developers now have access to thousands of applications ready to run right “off the shelf” and the ability to quickly build and publish their own images.

In addition to the application, the container image may contain hundreds of packages and thousands of files including binaries, shared libraries, configuration files, and 3rd party modules. Any one of these components may contain a security vulnerability, an outdated software module, a misconfigured configuration file or simply fail to comply with your operational or security best practices.

For more details and to download/install:
https://anchore.com

Solve sudo sending useless emails “problem with defaults entries”

Whenever a user (whether sssd-ad authenticated user, or local user, or root) uses sudo, it works. But it also sends the administrator a useless email. This problem is caused by sudo looking for directives in a place it cannot find them: sss.
Check the /etc/nsswitch.conf file and modify the sudoers entry.

 sudoers: files sss 

The sss should not be there. The sssd-ad package adds itself there, but very few environments store sudoers directives in sss. It’s far more likely your directives are local, so you should have a /etc/nsswitch file entry like the following:

 sudoers: files 

Thanks to this post for pointing the above out:
https://bgstack15.wordpress.com/2016/06/06/solve-sudo-sending-useless-emails-problem-with-defaults-entries/

snap – snapcraft.io

Package any app for every Linux desktop, server, cloud or device, and deliver updates directly. Snap packages are a great way of running apps in an isolated state without using VMs or Containerisation. The website http://snapcraft.io has more details and examples such as this to get going:

A snap is a fancy zip file containing an application together with its dependencies, and a description of how it should safely be run on your system, especially the different ways it should talk to other software. Most importantly snaps are designed to be secure, sandboxed, containerised applications isolated from the underlying system and from other applications. Snaps allow the safe installation of apps from any vendor on mission critical devices and desktops.

Try this (you may need to install snapd)

 $ sudo snap install hello-world 

Now you have installed a snap. You can take a look inside the snap very easily, it shows up as a new directory on your system:

$ cd /snap/hello-world/current/

$ tree
.
├── bin ← this directory structure is just for convenience
│ ├── echo there is no hardcoded structure requirement other
│ ├── env than meta/snap.yaml
│ ├── evil
│ ├── sh
│ ├── showdev
│ └── usehw
└── meta ← your snap must have this directory
├── icon.png ← no prizes for guessing what this is
└── snap.yaml ← this is the required metadata