More load testing with SysBench

Here is another load testing tool for web servers (mainly LAMP) although you could load test any nix based system. The tool is aimed at systems that run databases, so you would also want to run this against a DB server as well.

From the Sourceforge website:
The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.

Current features allow to test the following system parameters:

  • file I/O performance
  • scheduler performance
  • memory allocation and transfer speed
  • POSIX threads implementation performance
  • database server performance (OLTP benchmark)

Installation:
Found in most repos, you can install with a apt-get install or yum install – in this example I will install it onto a CentOS server.

 yum install sysbench 

Running the tool:
There are many options and tests to run such as CPU, Threads, Mutex, Memory, FileIO and MySQL.

CPU:

 sysbench --test=cpu --cpu-max-prime=20000 run 

Threads:

 sysbench --num-threads=64 --test=threads --thread-yields=100 --thread-locks=2 run 

FileIO:

sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=rndrw prep
sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=rndrw run
sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=rndrw cleanup

MySQL:
To measure MySQL performance, we first create a test table in the database test with 1,000,000 rows of data.

sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password=yourrootsqlpassword prepare

Now you can run the MySQL benchmark as follows.

sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password=yourrootsqlpassword --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run

Clean up the system – remove the test table, run:

sysbench --test=oltp --mysql-db=test --mysql-user=root --mysql-password=yourrootsqlpassword cleanup

Reference, docs and more examples:
http://sysbench.sourceforge.net/docs
http://www.howtoforge.com/how-to-benchmark-your-system-cpu-file-io-mysql-with-sysbench

 

Load test webservers comparison and Siege

What are good web server benchmarking tools for Linux?

Siege is an http load testing and benchmarking utility. It was designed to let web developers measure their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP and HTTPS protocols. It lets its user hit a web server with a configurable number of simulated web browsers. I find this utility very useful and it’s also good at checking your infrastructure and security.

In this example I will install Siege onto my Ubuntu Desktop and test my web server which is running CentOS with Apache. Siege should be available in most Linux distributions repositories, it’s certainly in Ubuntu, Debian and CentOS (epel) repos.

Install Siege onto client computer running Ubuntu:

 sudo apt-get install siege 

Test web server which is named tigershark.local in this case:

 sudo siege -c 120 http://tigershark.local 

The above command will simulate 120 concurrent users hitting the site, the -c means concurrent users plus the value. You can also parse other switches such as -t which allows you to specify the time to run the test for. Here is another example using the time option and verbose output:

 sudo siege -v -c 120 -t30M http://tigershark.local 

See the MAN page or help switch for more options and more details including the manual can be found at the creators webpage: http://www.joedog.org/siege-manual

 

Ubuntu 12.04 Post Install Upgrade Tweaks (Desktop)

 

Here are my Ubuntu 12.04 Post Install or Upgrade Tweaks:

1. Install MyUnity from Ubuntu software centre.

2. Install VLC player from Ubuntu software centre.

3. Install Google Chrome from https://www.google.com/chrome

4. Install Ubuntu Tweak:

sudo add-apt-repository ppa:tualatrix/ppa
sudo apt-get update
sudo apt-get install ubuntu-tweak 

5. Install VIM editor

 sudo apt-get install vim 

6. Disable Guest login:  change the allow-guest option to: allow-guest=false on the /etc/lightdm/lightdm.conf file.

7. Install pre-load:

 sudo apt-get install preload 

9. Install archey, a script that displays system info in a terminal:

sudo apt-get install lsb-release scrot
wget http://github.com/downloads/djmelik/archey/archey-0.2.8.deb
sudo dpkg -i archey-0.2.8.deb 

10. For laptop or netbook installations, install hardware temperature indicators:

sudo add-apt-repository ppa:alexmurray/indicator-sensors
sudo apt-get update
sudo apt-get install indicator-sensors 

11. Disable IPV6 (Optional) – edit /etc/sysctl.conf and add these commands:

 # IPv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1 

12. Disable or remove Zeitgeist**
Remove with caution and research first, it could break things!!
The safest way I’ve found is with this sequence of commands:

sudo apt-get --purge remove zeitgeist
sudo apt-get autoremove

Performance Tweaks:

Show Hidden Startup Apps:

 sudo sed -i "s/NoDisplay=true/NoDisplay=false/g" /etc/xdg/autostart/*.desktop 

Install Preload from Software Center:
Open software centre and search plus install the Preload package.

Change Swappiness Value:
Enter following line to see current swappiness value:

 cat /proc/sys/vm/swappiness 

(default value in Ubuntu for swappiness is 60)

Open /etc/sysctl.conf in a text editor:

 sudo vim /etc/sysctl.conf 

Enter your desired swappiness value at the end of the file:
# swappiness value
vm.swappiness=10
Save the file and restart the computer.

Reference sites and articles:
http://www.techdrivein.com/2012/06/25-things-i-did-after-installing-ubuntu.html
http://www.noobslab.com/2012/04/important-things-to-do-after-install_26.html
http://www.unixmen.com/201204-top-things-to-do-after-installing-ubuntu-2
http://www.omgubuntu.co.uk/2012/04/10-things-to-do-after-installing-ubuntu-12-04
http://www.tuxgarage.com/2012/04/things-to-do-after-installing-precise.html
http://techhamlet.com/2012/04/things-to-do-after-installing-ubuntu-12-04-the-start 
http://ubuntuforums.org/showthread.php?t=1773332

Upgrade Ubuntu to 12.04 LTS – Desktop & Server

 

At the time of writing Ubuntu 12.04 LTS (Long Term Support) is the latest version of the Ubuntu Linux distribution – here is a quick run down on how to upgrade previous versions of Ubuntu to 12.04 on both Desktop and Server versions of the OS.

Desktop:

First ensure that the current version of your Ubuntu installation has all the latest updates and fixes applied. To do this start the Update Manager or Software Updater from the menu system and click the Check button, followed by the Install Updates button. If required restart the system and re-launch the Update Manager – you should see a section labelled “New Ubuntu release “12.04 LTS” is available” with an upgrade button next to it, just simply go ahead and select upgrade and follow the instructions. Depending on your computer’s and Internet connection speed it may take sometime to complete.

Note: It’s important to ensure you run the normal updates prior to the upgrade, as I’ve had a system fail the upgrade due to older package versions required for the upgrade to be successful.

 

Server:

The server will require everything performed from the command shell or via SSH.

Change to root user to make the process a bit quicker and easier, or use sudo for these commands if you prefer.

 sudo su 

Update the system with all the latest updates first as with the Desktop procedure above this is required.

 apt-get update; apt-get upgrade 

Now install the package manager core:

 apt-get install update-manager-core 

Edit the release upgrades file and ensure it has Prompt=normal set e.g.:

 vim /etc/update-manager/release-upgrades 

Now start the upgrade process with:

 do-release-upgrade 

Follow the on-screen instructions for the upgrade process, and once the system has been restarted you should see “Ubuntu 12.04 LTS” shown at the login prompt or when you SSH in.

phpSysInfo page on Ubuntu and Debian Servers

phpSysInfo is a customizable PHP Script that parses /proc, and formats information nicely. It will display information about system facts like Uptime, CPU, Memory, PCI devices, SCSI devices, IDE devices, Network adapters, Disk usage, and more. Here is a quick how to on installing and configuring it with Ubuntu or Debian based servers. Note: you will need to have Apache with PHP up and running for phpSysInfo to operate correctly.

Installation:

 sudo apt-get install  phpsysinfo 

Edit the Configuration File:

 sudo vim /etc/apache2/conf.d/phpsysinfo.conf 

Customise the phpsysinfo.conf file to suit your network and server – example:

Alias /phpsysinfo /usr/share/phpsysinfo
<Location /phpsysinfo>
Options None
Order deny,allow
Deny from all
Allow from localhost
Allow from 192.168.1.0/24
# Allow from all
</Location>

Restart Apache:

 sudo /etc/init.d/apache2 restart 

Test from web browser with http://yourserver/phpsysinfo

Reference:
http://sourceforge.net/projects/phpsysinfo

 

PortSentry – Linux port scan detector

PortSentry is a handy Linux port scan detector and is part of the Sentry Tools pack. The Sentry tools provide host-level security services.  PortSentry, Logcheck/LogSentry, and HostSentry protect against portscans, automate log file auditing, and detect suspicious login activity on a continuous basis.

The utility has been around for sometime now and should be available from your repos. In this example I will install it onto a Ubuntu 12.04 Server:

 sudo apt-get install portsentry

Edit the the portsentry.conf file located in /etc/portsentry/ e.g. vim /etc/portsentry/portsentry.conf and configure the settings to suit your server environment. By default PortSentry does not block, just logs – to turn on blocking set the BLOCK_TCP=”0″ to BLOCK_TCP=”1″ and BLOCK_UDP=”0″ to BLOCK_UDP=”1″. You can also fine tune other things such as ports and create a white list for local hosts or Intranets.

Restart the service:

 /etc/init.d/portsentry restart 

To test it out, run a nmap scan from another computer on your network and watch the syslog on the server e.g.

On Server – watch the syslog:

 sudo tail -f /var/log/syslog 

On the client computer run a nmap scan (using nmap v6.1 here):

 sudo nmap -v -A -T4 ubuntuserver 

You should soon see the syslog reporting the attack alert and the nmap scan should come to a screaming stop!

Reference:
http://sourceforge.net/projects/sentrytools

 

Linux find command – some examples

The Linux find command can be very powerful and very handy, here are some examples that I try to remember or at least have handy so that I don’t have to go into the MAN pages or Google!

Find all JPG image files on the system:

 find / -name *.jpg 

 or you can grep results as well

 find / | grep .jpg 

Find all the directories named “conf” on the system:

 find / -type d -name conf 

Find all the files owned by user Cathy:

 find / -user cathy 

To find files owned by a group named engineering

 find / -group engineering 

Find all the files that were changed in the last 10 minutes:

 find / -mmin -10 -type f 

 or to find files changed in the last 10 days

find / -mtime -10 

Find files with certain permissions applied. Hopefully you don’t find any 777 files in your web server area!:

 find /var/www -perm 777 

Find large files on the system with a few different options. This command shows the 5 largest files in the sharedata folder and lists them in order from largest to smallest.:

  find /shareddata/ -type f -exec du -h {} ; | sort -rh | head -5 

Reference: MAN find or find –help

How to Install VirtualBox Guest Additions in CentOS 6.x

I found installing VirtualBox guest additions to be more of a challenge than with Ubuntu/Debian, mainly due to some extra packages required and some Kernel variables – here is how I do it:

 

  1. Make sure everything is updated
     yum update 
  2. Update the kernel
     yum update kernel* 
  3. Install the some packages required for the guest additions
     yum install gcc kernel-devel kernel-headers 
  4. Configure the KERN_DIR variable. * you may need to reboot, or it might be a good idea if you do just to be sure.
     KERN_DIR=/usr/src/kernels/`uname -r`-`uname -m`
    export KERN_DIR 
  5. Install guest additions from  Devices Menu on Virtual Box: Devices > Install Guest Additions
    __________________________________________________________________
  6.  Click OK to open the Autorun Prompt and click Run then the root password and the guest additions should install successfully. Reboot the system and verify everything is running.
    __________________________________________________________________

References:
http://wiki.centos.org/HowTos/Virtualization/VirtualBox/CentOSguest
http://www.virtualbox.org/manual/ch04.html

 

GoAccess – Real-Time Terminal Analyser

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.

GoAccess parses the specified web log file and outputs the data to the X terminal. Reports include:

  • General Statistics, bandwidth etc.
  • Top Visitors
  • Requested files
  • Requested static files, images, swf, js, etc.
  • Referrers URLs
  • 404 or Not Found
  • Operating Systems
  • Browsers and Spiders
  • Hosts, Reverse DNS, IP Location
  • HTTP Status Codes
  • Referring Sites
  • Keyphrases
  • Support for IPv6
  • Different Color Schemes
  • Unlimited log file size
  • Custom log format

More details and to download see the GoAccess website: http://goaccess.prosoftcorp.com

 

Download Manager for Ubuntu – Steadyflow

Steadyflow is a simple and easy to use download manager, however it’s a bit more than just a downloader, it can also perform things on completion such as run another command or open the download file. It’s my first choice for downloading large files such as ISOs!  All you need to do is just add the URL and it will take care of the rest and it can also do parallel downloads.

Installation with Ubuntu is just an apt-get away with:

 sudo apt-get install steadyflow