How do I stop and start EC2 instances at regular intervals using AWS Lambda?

Issue:
I want to reduce my Amazon EC2 usage by stopping and starting instances at predefined times or utilization thresholds. Can I configure AWS Lambda and AWS CloudWatch to help me do that automatically?

Short Description:
You can use a CloudWatch Event to trigger a Lambda function to start and stop your EC2 instances at scheduled intervals.

Resolution:
See AWS Web Page for more: https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/ 

 

LXD Update Script

Simple shell script wrapper to update all lxd container hypervisor images running Debian or Ubuntu Linux. Run the below script on the host of your lxd server to update all the lxd containers running Ubuntu or Debian. Could be modified to suit CentOS/RedHat OS and/or you container OS of choice.

Script:

#!/bin/bash
# A simple shell script to update all lxd container hypervisor
# URL: https://bash.cyberciti.biz/virtualization/shell-script-to-update-all-lxd-container-hypervisor/
# Tested on : Ubuntu 16.04 LTS lxd server
# Tested on : Ubuntu/Debian lxd container hypervisor only
# ----------------------------------------------------------------------------
# Author: nixCraft
# Copyright: 2016 nixCraft under GNU GPL v2.0+
# ----------------------------------------------------------------------------
# Last updated 14 Aug 2016
# ----------------------------------------------------------------------------
# Set full path to bins
_apt="/usr/bin/apt-get"
_lxc="/usr/bin/lxc"
_awk="/usr/bin/awk"
# Get containers list
clist="$(${_lxc} list -c ns | ${_awk} '!/NAME/{ if ( $4 == "RUNNING" ) print $2}')"
# Use bash for loop and update all container hypervisor powered by Debian or Ubuntu
# NOTE: for CentOS use yum command instead of apt-get
for c in $clist
do
echo "Updating Debian/Ubuntu container hypervisor \"$c\"..."
${_lxc} exec $c ${_apt} -- -qq update
${_lxc} exec $c ${_apt} -- -qq -y upgrade
${_lxc} exec $c ${_apt} -- -qq -y clean
${_lxc} exec $c ${_apt} -- -qq -y autoclean
done

Reference:
https://bash.cyberciti.biz/virtualization/shell-script-to-update-all-lxd-container-hypervisor/

 

The detox utility

The detox utility renames files to make them easier to work with. It removes spaces and other such annoyances. It’ll also translate or cleanup Latin-1 (ISO 8859-1) characters encoded in 8-bit ASCII, Unicode characters encoded in UTF-8, and CGI escaped characters.

Sequences:
detox is driven by a configurable series of filters, called a sequence. Sequences are covered in more detail in detoxrc(5) and are discoverable with the -L option. Some examples of default sequences are iso8859_1 and utf_8.

Options:

The main options:

-f configfile
Use configfile instead of the default configuration files for loading translation sequences. No other config file will be parsed.
-h –help
Display helpful information.

-L’ List the currently available sequences. When paired with -v this option shows what filters are used in each sequence and any properties applied to the filters.

-n –dry-run
Doesn’t actually change anything. This implies the -v option.

-r’ Recurse into subdirectories.

-s sequence
Use sequence instead of default.

–special
Works on special files (including links). Normally detox ignores these files.

-v’ Be verbose about which files are being renamed.

-V’ Show the current version of detox.

Reference:
http://detox.sourceforge.net/

 

LSOF Utility and Commands

The lsof command is very handy and all Linux/Unix sysadmins should know abouyt it and make more use of it, here are some further details and examples.

(From Wikipedia) – lsof is a command meaning “list open files”, which is used in many Unix-like systems to report a list of all open files and the processes that opened them. This open source utility was developed and supported by Victor A. Abell, the retired Associate Director of the Purdue University Computing Center. It works in and supports several Unix flavors.

Useful Commands:

List all network connections:

 sudo lsof -i 

SSH:

sudo lsof -i:22
sudo lsof -c ssh

List all network files in use by a specific process:

 sudo lsof -i -a -p 1278 

List processes which are listening on a particular port:

 sudo lsof -i :25 

List all TCP or UDP connections:

 sudo lsof -i tcp; lsof -i udp; 

List processes which opened a specific file:

 lsof /var/log/syslog 

List opened files under a directory:

 lsof +D /var/log/ 

List opened files based on process names starting with:

 lsof -c ssh -c init 

List processes using a mount point:

 sudo lsof /home 

List files opened by a specific user:

 lsof -u keith 

What commands is user Keith using:

 sudo lsof -i -u keith 

List all open files by a specific process:

 lsof -p 1278 

Kill all process that belongs to a particular user:

 kill -9 `lsof -t -u keith` 

 

SSL Server Test

Qualys SSL Server Test is free online service that performs a deep analysis of the configuration of any SSL web server on the public Internet.

https://www.ssllabs.com/ssltest/

To configure Apache and Nginx with acceptable protocols TLS settings follow these guides, they will provide a better score on the above Qualys rating and make your SSL site more secure.

 

Snapcraft

Package any app for any Linux desktop, server, cloud or device.

A ‘snap’ is a universal Linux package

Snaps work on any distribution or device. Snaps are faster to install, easier to create, safer to run, and they update automatically and transactionally so your app is always fresh and never broken.

The public collection of snaps includes the latest and best apps from GitHub and beyond, so you have the whole world of Linux apps at your fingertips. Take the tour below to experience ‘hello world’ as a snap, or jump to the developer guide to create your own snaps.

Reference: http://snapcraft.io/

netdata real-time performance monitoring for Linux

Netdata is a real-time performance monitoring solution.

Unlike other solutions that are only capable of presenting statistics of past performance, netdata is designed to be perfect for real-time performance troubleshooting.

Netdata is a linux daemon you run, which collects data in realtime (per second) and presents a web site to view and analyze them. The presentation is also real-time and full of interactive charts that precisely render all collected values.

Netdata has been designed to be installed on every system, without disrupting the applications running on it:

  • It will just use some spare CPU cycles (check Performance).
  • It will use the memory you want it have (check Memory Requirements).
  • Once started and while running, it does not use any disk I/O, apart its logging (check Log Files). Of course it saves its DB to disk when it exits and loads it back when it starts.
  • You can use it to monitor all your systems and applications. It will run on Linux PCs, servers or embedded devices.

Out of the box, it comes with plugins that collect key system metrics and metrics of popular applications.

Available here: https://github.com/firehol/netdata

400+ Free Resources for DevOps & Sysadmins

In 2014 Google indexed 200 Terabytes of data (1 T of data is equal to 1024 GB, to give you some perspective). And, it’s estimated that Google’s 200 TB is just .004% of the entire internet. Basically the internet is a big place with unlimited information.

So in an effort to decrease searching and increase developing, Morpheus Data published this massive list of free resources for DevOps engineers and System Admins, or really anyone wanting to build something useful out of the internet.

All these resources are free, or offer some kind of free/trial tier. You can use any/all of these tools personally, as a company, or even suggest improvements (in the comments). It’s up to you.

If you find this list useful, please share it with your DevOps/SysAdmin friends on your favorite social network, or visit Morpheus Data to learn how you can 4x your application deployment.

http://www.nextbigwhat.com/devops-sysadmin-tools-resources-297/?_utm_source=1-2-2

AWS CLI and AWS S3 CLI

Backup Files From Ubuntu Or Debian Server’s To Amazon s3:
http://www.unixmen.com/backup-files-ubuntu-debian-servers-amazon-s3/

Using High-Level s3 Commands with the AWS Command Line Interface:
http://docs.aws.amazon.com/cli/latest/userguide/using-s3-commands.html

An Introduction to the AWS Command Line Tool:
https://www.linux.com/learn/tutorials/761430-an-introduction-to-the-aws-command-line-tool

AWS Command Line Interface:
https://aws.amazon.com/cli/