Centralised Syslog Systems

Here is my list of some great Opensource Syslog Systems that you can use. Some may not scale very well and others would need to be purchased for larger networks or additional features, such as Splunk. Take a look do you research and test them out first!

 

 

MOSH Mobile Shell

Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes.

Mosh is a replacement for SSH. It’s more robust and responsive, especially over Wi-Fi, cellular, and long-distance links.

Mosh is free software, available for GNU/Linux, FreeBSD, Solaris, Mac OS X, and Android.

Reference:
http://mosh.mit.edu/

Network Packet Generators

From Wikipedia:
A packet generator or packet builder is a type of software that generates random packets or allows the user to construct detailed custom packets.

This can be handy for testing networks and security systems.

Full table list wikipedia:
http://en.wikipedia.org/wiki/Packet_generator

Ostinato:
https://code.google.com/p/ostinato

Scapy:
http://www.secdev.org/projects/scapy

 

TCP Wrappers – What and Why

TCP Wrappers are host based access control mechanism for services in the user space of Linux i.e. TCP Wrappers work at the application layer. Often used with xinetd based services such as SSH FTP and so forth. A TCP Wrapper is not a firewall or should not really be implemented instead of iptables, but rather used in conjuction with iptables as a backup or safeguard. Some other applications and utilities will use a TCP Wrapper such as fail2ban, deny hosts, OSSEC and so forth – although they may also use iptables.

In addition connections can be logged which provides an additional layer of visibility and security for your specific services. TCP Wrappers are run-time live, meaning that configuration changes don’t require a service restart. However this can also be a dangerous thing as you can easily lock yourself out, especially with SSHD services.

Most distros have hosts.allow and hosts.deny located in the /etc directory and that’s where you do the configuration. Example:

hosts.allow:

 sshd: 192.168.1.0/24 

hosts.deny:

 sshd: ALL 

The above example is a very simple allow SSH connections from devices on the LAN of 192.168.1.0/24 but deny everything else. You can of course lock it down to just several IP addresses or a single management station if you really need to. You can see how this is a fail safe to an iptables rule that you might have.

Reference:
https://en.wikipedia.org/wiki/TCP_Wrapper
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Security_Guide/index.html#sect-Security_Guide-Server_Security-Securing_Services_With_TCP_Wrappers_and_xinetd

 

 

ngrep (network grep) utility

ngrep is one of my favourite CLI tools that I use with Linux, it’s very easy to use and just works really well. It’s like using tcpdump with a piped out grep command and some other sorting switches. The utility has been around for sometime and from the author’s page not much has changed in sometime, but why would it, the tool just works great!

Here is the description from the author’s site:
ngrep strives to provide most of GNU grep’s common features, applying them to the network layer. ngrep is a pcap-aware tool that will allow you to specify extended regular or hexadecimal expressions to match against data payloads of packets. It currently recognizes IPv4/6, TCP, UDP, ICMPv4/6, IGMP and Raw across Ethernet, PPP, SLIP, FDDI, Token Ring and null interfaces, and understands BPF filter logic in the same fashion as more common packet sniffing tools, such as tcpdump and snoop.

Installation:

I’ve found the utility to be available in most Linux distributions repositories. Debian/Ubuntu:

 sudo apt-get install ngrep 

Fedora/Centos/Redhat:

 sudo yum install ngrep 

Examples:

Note: ngrep requires elevated privileges to gain access to your NIC, so either sudo or run as root depending on your distro and configuration.

If you want to see all the DNS traffic going to and from your system:

 sudo ngrep -d eth0 port 53 

Want to look at traffic going to and from your mail server:

 sudo ngrep -d eth0 port 25 and src host 192.168.1.7 

As you can see from the output, ngrep provides you with a conversation in a very cool and easy to read format. Here I’m sending a test email on an internal network using a internal SMTP server, which you can see clearly here!

interface: eth0 (192.168.1.0/255.255.255.0)
filter: (ip or ip6) and ( port 25 and src host 192.168.1.7 )
##
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
220 stingray Axigen ESMTP ready..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [A]
......
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
250-stingray Axigen ESMTP hello..250-PIPELINING..250-AUTH PLAIN LOGIN CRAM-MD5 DIGEST-MD5 GSSAPI ..250-AUTH=PLAIN LOGIN CRAM-MD5 DIGEST-MD5 GSSAPI ..250-8 BITMIME..250-BINARYMIME..250-CHUNKING..250-SIZE 52428800..250-STARTTLS..250-HELP..250 OK..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
250 Sender accepted..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
250 Recipient accepted..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
354 Ready to receive data; remember <CRLF>.<CRLF>..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [A]
......
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
250 Continue delivery..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AP]
221-stingray Axigen ESMTP is closing connection..221 Good bye..
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [AF]
......
#
T 192.168.1.7:25 -> 192.168.1.3:44708 [A]
......
exit

Obviously the above is only on a local system and not the entire network, if you want to have visibility for all the network, you need to be connected up to a span port or mirror or even better a network tap. Then you will see everything for the network going in and out, here is an example of looking at traffic going from the LAN network segment to another LAN segment for port 443.

 sudo ngrep port 443 and src 192.168.1.0/24 and dst net 10.10.0.0/24 

Use words to find something you want to be aware of, example look for the word password in a syslog stream:

 sudo ngrep -d any 'password' port syslog 

Dump to a pcap file for analyses with Wireshark or dump the pcap file with pre-defined filters. For example filter only for port 21 FTP:

 sudo ngrep -O p21.pcap -d eth0 port 21 

You could also use ngrep to search the pcap file for certain parameters, such PASS for the ftp password!:

 sudo ngrep -w 'PASS' -I p21.pcap 
input: p21.pcap
match: ((^PASS\W)|(\WPASS$)|(\WPASS\W))
####
T 172.16.16.3:46374 -> 10.10.100.54:21 [AP]
PASS ssde..
############exit

This is why FTP is useless for security!

ngrep is a very handy and powerful utility that I will always keep on hand now.

Reference:
man ngrep
http://ngrep.sourceforge.net
http://en.wikipedia.org/wiki/Ngrep

Ubuntu Server – Enable Disable Services on Startup (Upstart jobs)

I recently needed to have the DHCP service on a Ubuntu 12.04 server configured to not start when the server started. The main reason was the network only needed DHCP for imaging purposes via a PXE boot configuration, and all servers on the network were static.

I had problems finding a way to achieve this, until I discovered the Upstart configuration which is from Ubuntu 11.04 onwards. A simple manual field in an override file is all that’s required. Here is what I did:

Edit the /etc/init/isc-dhcp-server.override file and simply add manual to it.

 echo manual >/etc/init/isc-dhcp-server.override 

Now DHCP service doesn’t start on boot-up and I can control the service manually with the service command “service isc-dhcp-server start” and “service isc-dhcp-server stop”.

Reference:
https://help.ubuntu.com/community/UpstartHowto

The following packages have been kept back…

Sometimes with Ubuntu/Debian Linux systems you might see

“The following packages have been kept back:”

Why and how do I solve it?
I found this post on ASk Ubuntu:
http://askubuntu.com/questions/601/the-following-packages-have-been-kept-back-why-and-how-do-i-solve-it

So to resolve it:

sudo apt-get update
sudo apt-get dist-upgrade 

WARNING: Just be careful with this on Servers especially in production, you might want to investigate what it’s holding back before firing off with that command on your servers!!

Phoronix Test Suite

Open-Source Benchmarking
The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. The software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner.

The Phoronix Test Suite is based upon the extensive testing and internal tools developed by Phoronix.com since 2004 along with support from leading tier-one computer hardware and software vendors. This software is open-source and licensed under the GNU GPLv3.

Originally developed for automated Linux testing, support to the Phoronix Test Suite has since been added for OpenSolaris, Apple Mac OS X, Microsoft Windows, and BSD operating systems. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile and related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.

The Phoronix Test Suite interfaces with OpenBenchmarking.org as a collaborative web platform for the centralized storage of test results, sharing of test profiles and results, advanced analytical features, and other functionality. Phoromatic is an enterprise component to orchestrate test execution across multiple systems with remote management capabilities.

Reference and Download Site:
http://www.phoronix-test-suite.com

Network Management & Monitoring Tools

Here is a short list I’ve compiled of the Network Management and Monitoring tools available, and of course there are plenty more, but these are my short list for now.

Nagios – http://www.nagios.org

Nagios has to be the grand of them all and it’s why it’s the first on the list, many others have been based on Nagios.

Nagios is a powerful monitoring system that enables organizations to identify and resolve IT infrastructure problems before they affect critical business processes. Designed with scalability and flexibility in mind, Nagios gives you the peace of mind that comes from knowing your organization’s business processes won’t be affected by unknown outages. Nagios is a powerful tool that provides you with instant awareness of your organization’s mission-critical IT infrastructure. Nagios allows you to detect and repair problems and mitigate future issues before they affect end-users and customers.

Observium – http://www.observium.org

Observium is an autodiscovering SNMP based network monitoring platform written in PHP which includes support for a wide range of network hardware and operating systems including Cisco, Linux, HP, Dell, FreeBSD, Juniper, Brocade, Netscaler, NetApp and many more. Observium has grown out of a lack of network monitoring platforms which are both simple to manage and pleasant to use. It is intended to provide a navigable interface to the health and performance of your network. Its design goals include collecting as much historical data about devices as possible, using as much auto-discovery as possible with little or no manual intervention, and having a very intuitive interface.

Observium is Free Software. This means that you can use and redistribute the software without permission and without paying anything.

Netdisco – http://www.netdisco.org

Netdisco is an Open Source web-based network management tool first released publically in 2003. The target users are large corporate and university networks administrators. Data is collected into a Postgres database using SNMP and presented with a clean web interface using Mason. Configuration information and connection data for network devices are retrieved via SNMP. Data is stored using a SQL database for scalability and speed. Layer-2 topology protocols such as CDP and LLDP provide automatic discovery of the network topology. Here are some of the favorite uses for this tool:

  • Locate a machine on the network by MAC or IP and show the switch port it lives at.
  • Turn Off a switch port while leaving an audit trail. Admins log why a port was shut down.
  • Inventory your network hardware by model, vendor, switch-card, firmware and operating system.
  • Report on IP address and switch port usage: historical and current.
  • Pretty pictures of your network.

Open Audit – http://www.open-audit.org

Open-AudIT is a network auditing application. It is based on the scripting languages of PHP, Bash and VBScript. It can tell you what is on your network, how it is configured and when it changes. Data is retrieved with Bash and/or VBScript, stored in a database and viewed through a web interface. The server only needs a web server (Apache and IIS have both been tested) and a MySQL install. Both of these applications are free to use.

Your network devices can be queried and audited. Results are stored in a database for viewing, reporting and change auditing. All interaction is via standards compliant web pages. To setup Open-AudIT you will need the following – a webserver (IIS or Apache will do), PHP installed on the webserver, a MySQL database (usually residing on the webserver), the gd and imagick extensions for PHP. Once these are installed and working, simply download the application (download the SVN Trunk for the latest version). Extract the files into a directory on your webserver that is visible to the clients. Call the page setup.php in a web browser. Follow the prompts. It really should be that simple.

Zabbix – http://www.zabbix.com

Zabbix is the ultimate open source availability and performance monitoring solution. Zabbix offers advanced monitoring, alerting, and visualization features today which are missing in other monitoring systems, even some of the best commercial ones. Below is a short list of features available in Zabbix:

  • auto-discovery of servers and network devices
  • low-level discovery
  • distributed monitoring with centralized web administration
  • support for both polling and trapping mechanisms
  • server software for Linux, Solaris, HP-UX, AIX, FreeBSD, OpenBSD, OS X
  • native high performance agents (client software for Linux, Solaris, HP-UX, AIX, FreeBSD, OpenBSD, OS X, Tru64/OSF1, Windows NT4.0, Windows 2000, Windows 2003, Windows XP, Windows Vista)
  • agent-less monitoring
  • secure user authentication
  • flexible user permissions
  • web-based interface
  • flexible e-mail notification of predefined events
  • high-level (business) view of monitored resources
  • audit log

Spiceworks – http://www.spiceworks.com

Spiceworks has many more features than just network monitoring and it might be an overkill for some networks. On the other hand it might be exactly what you want.

Delete Linux MBR

If you need to delete a Linux MBR, you can simply use the dd command, this might be more useful with a USB stick that still has an MBR.

WARNING!! Don’t so any of this if you don’t understand it or be careful not to mix your disk designations up i.e. /dev/sdb or /dev/sdc could be your real disk!

The following will delete the MBR on /dev/sdc:

 dd if=/dev/zero of=/dev/sdc bs=446 count=1 

Reference:
http://www.cyberciti.biz/faq/linux-clearing-out-master-boot-record-dd-command