AWS Cloudformation – Resources, Tips and Tricks

Recently been creating CF templates for a job and just learning more in this area of AWS. Listed below are resources, tips and tricks that were helpful.

Resources

AWS User Guide:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html

AWS Github Sample Templates:
https://github.com/awslabs/aws-cloudformation-templates

AWS Sample Templates:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-sample-templates.html

Bogotobogo excellent post on Cloudformation, Templates, Change Sets and CLI:
https://www.bogotobogo.com/DevOps/AWS/aws-CloudFormation-Templates.php

Stelligent Cloudformation Templates
https://github.com/stelligent/cloudformation_templates

AWS::CloudFormation::Init
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html

Tips and Tricks

yamllint – A linter for YAML files.

yamllint does not only check for syntax validity, but for weirdnesses like key repetition and cosmetic problems such as lines length, trailing spaces, indentation, etc.
https://github.com/adrienverge/yamllint

CloudFormation Linter

Validate CloudFormation yaml/json templates against the CloudFormation spec and additional checks. Includes checking valid values for resource properties and best practices.
https://github.com/aws-cloudformation/cfn-python-lint

validate-template

Validates a specified template. AWS CloudFormation first checks if the template is valid JSON. If it isn’t, AWS CloudFormation checks if the template is valid YAML. If both these checks fail, AWS CloudFormation returns a template validation error.
https://docs.aws.amazon.com/cli/latest/reference/cloudformation/validate-template.html

CreationPolicy Attribute

Associate the CreationPolicy attribute with a resource to prevent its status from reaching create complete until AWS CloudFormation receives a specified number of success signals or the timeout period is exceeded. To signal a resource, you can use the cfn-signal helper script or SignalResource API. AWS CloudFormation publishes valid signals to the stack events so that you track the number of signals sent.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-creationpolicy.html

UpdatePolicy Attribute

Use the UpdatePolicy attribute to specify how AWS CloudFormation handles updates to the AWS::AutoScaling::AutoScalingGroup, AWS::Lambda::Alias, or AWS::ElastiCache::ReplicationGroup resources.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-updatepolicy.html

DeletionPolicy Attribute

With the DeletionPolicy attribute you can preserve or (in some cases) backup a resource when its stack is deleted. You specify a DeletionPolicy attribute for each resource that you want to control. If a resource has no DeletionPolicy attribute, AWS CloudFormation deletes the resource by default.

Note that this capability also applies to stack update operations that lead to resources being deleted from stacks. For example, if you remove the resource from the stack template, and then update the stack with the template. This capability does not apply to resources whose physical instance is replaced during stack update operations. For example, if you edit a resource’s properties such that AWS CloudFormation replaces that resource during a stack update.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html


Using Nextcloud’s command line

Here is a great reference for using Nextcloud’s command options. Some of these command line switches are very handy as you can’t always do these tasks from the GUI.

Cleanup Nextclouds filecache:

sudo -u www-data php occ files:cleanup

Re-Scan your Nextcloud data:

sudo -u www-data php occ files:scan --all -v

More info and the full listing at:
https://www.c-rieger.de/using-nextclouds-command-line

AWS cli Builder

AWS cli builder is a very handy site for doing just that…. building AWS cli commands. Choose from 148 services and input your required options and it will build out the command to use.

For example to start an EC2 instance, choose Computer > Elastic Computer Cloud and type start. It should provide an option for starting instances. Now enter your details such as region, instance id and output type.

More info and to access the site go here: https://awsclibuilder.com/home

SSH Audit

Ideally you should whitelist your SSH Servers to your IPs or don’t even expose the service to the Internet and use a VPN to login. In some cases this is not an option and/or you may need to create a Bastion Host / Jump Box server with SSH exposed.

If this is the case you should do or at least consider these options:

  • Change the SSH port (May reduce the noise, but not the security)
  • Only use Key Based Authentication i.e. disable password and use keys only
  • Prevent root user from logging in via SSH
  • Limit user access via SSH

Once the above has been completed, use SSH Audit to check that everything from an SSH configuration is setup correctly and passes the security checks. SSH Audit is a free online tool allows you to audit the configuration of an SSH server.

Reference:
https://www.sshaudit.com

Zstandard compression

Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. It’s backed by a very fast entropy stage, provided by Huff0 and FSE library.

The project is provided as an open-source dual BSD and GPLv2 licensed C library, and a command line utility producing and decoding .zst.gz.xz and .lz4 files. Should your project require another programming language, a list of known ports and bindings is provided on Zstandard homepage.

Installation is from repos:
sudo apt install zstd

Reference: https://facebook.github.io/zstd/

bootiso

Create a USB bootable device from an ISO image easily and securely.

Don’t want to messup the system with dd command? Create a bootable USB from an ISO in one line [see it in action].

Works seamlessly with hybrid and non-hybrid ISOs (SYSLINUX or UEFI compliant) such as any linux ISO, Windows ISO or rescue live-cds like UltimateBootCD. You don’t have to tweak anything: bootiso inspects the ISO file and chooses the best method to make your USB bootable.

Synopsis

bootiso [<options>...] <file.iso>
bootiso <action> [<options>...] <file.iso>
bootiso <action> [<options>...]

The default action [install-auto] as per first synopsis is to install an ISO file to a USB device in automatic mode. In such mode, bootiso will analyze the ISO file and select the best course of actions to maximize the odds your USB stick be proven bootable (see automatic mode behavior).
Other <options> and <actions> are listed in this bellow section.

Examples

To have a quick feedback, [probe] around to check bootiso capabilities with given ISO file and list USB drives candidates [watch video]:

bootiso -p myfile.iso

Quick install

curl -L https://git.io/bootiso -O
chmod +x bootiso

Reference: https://github.com/jsamr/bootiso

grepcidr

grepcidr is a 

grepcidr can be used to filter a list of IP addresses against one or more Classless Inter-Domain Routing (CIDR) specifications. As with grep, there are options to invert matching and load patterns from a file. grepcidr is capable of efficiently processing large numbers of IPs and networks.

grepcidr has endless uses in network software, including: mail filtering and processing, network security, log analysis, and many custom applications.

For detailed instructions and examples, please see the README file or man page. A couple examples of usage:

	grepcidr 2001:db8::/32 logfile
	grepcidr 66.249.64.0/19 access.log

Reference: http://www.pc-tools.net/unix/grepcidr/

Install grepcidr with your package manager:

 sudo apt install grepcidr 

Linux du command alternatives

For years I’ve used ncdu a NCurses Disk Usage utility for Linux. Recently someone alerted me to some other options as well as ncdu:

Dust:
du + rust = dust. Like du but more intuitive, Dust is meant to give you an instant overview of which directories are using disk space without requiring sort or head. Dust will print a maximum of 1 ‘Did not have permissions message’. Dust will list the 20 biggest sub directories or files and will smartly recurse down the tree to find the larger ones. There is no need for a ‘-d’ flag or a ‘-h’ flag. The largest sub directory will have its size shown in red.

https://github.com/bootandy/dust

The Tin Summer:
sn is a replacement for du. It has nicer output, saner commands and defaults, and it even runs faster on big directories thanks to multithreading.

https://github.com/vmchale/tin-summer

NCDU:
Ncdu is a disk usage analyzer with an ncurses interface. It is designed to find space hogs on a remote server where you don’t have an entire graphical setup available, but it is a useful tool even on regular desktop systems. Ncdu aims to be fast, simple and easy to use, and should be able to run in any minimal POSIX-like environment with ncurses installed.
Install from repos: sudo apt install ncdu

man page: https://dev.yorhel.nl/ncdu/man

 

tcpdump101.com – generate tcpdump commands

tcpdump101.com is a great site that you can use to generate tcpdump commands, you enter the parameters it’s asks for and it will generate the command for you. It’s handy if you are not running tcpdump commands very often and then have to either look up the help/man pages or Google for the command switches you want. It also has output for Cisco and Checkpoint firewalls.

From there site they say… tcpdump101.com has been designed to help people capture packets on different devices to assist with network troubleshooting, service troubleshooting and even passive red team activities. There is an assumption that the user has a basic understanding of what they want to capture – As much as this is a tool to help people, the user has to use their own logic since every situation is different. That being said, I strongly suggest that if you’re just starting out with packet captures to grab a copy of Virtual Box and play around with Linux and tcpdump. Although tcpdump may not be what you ultimately use, it will give you an excellent understanding of what you’ll see, even with other products and vendors.

As a safety measure (if at all possible) make sure to set a capture limit on your PCaps. If you make a mistake building your filters, you may end up captuing a lot of traffic. Although the odds are slim, there is a chance that your PCap could fill the NIC buffer and start dropping packets. The worst-case scenario is that it runs out of memory while you’re logged in remotely. With today’s hardware, it most likely won’t happen however you should always expect the best and plan for the worst.

 

 

 

 

Reference: tcpdump101.com