Hosting and deployment

Hosting and Deployment

Hosting

The production and test environments are both hosted on AWS. They both use Elastic Beanstalk for platform management. Elastic Beanstalk (EB) provides a 'Heroku' like experience for deployment. It is not without it's foibles however.

For example : DO NOT TRY AND UPGRADE A RUNNING ENVIRONMENT TO THE LATEST PLATFORM! Sadly you have to create a new platform first and migrate everything across, see below.

CDN

A CDN is used on test and production to handle all static assets. This is provided by AWS CloudFront

Command line

To manage these environments from the command line, install the awscli and aswebcli tools.

AWS CLI AWS EB CLI

If you use HomeBrew, then you can simply

brew install awscli
brew install awsebcli

Deployment to test

Assuming that all new changes have been merged in to the aws-eb-test branch and have been pulled to your local repository:

• make sure you have the aws-eb-test branch checked out
• run the deployment command

eb deploy

Deployment to production

Assuming that all new changes have been merged in to the master branch and have been pulled to your local repository:

• checkout the aws-eb-master branch
• merge master
• push aws-eb-master
• run the deployment command

eb deploy

Configuration

There are two directories which contain EB configuration data in the repository

.elasticbeanstalk

This directory https://github.com/BathHacked/energy-sparks/tree/master/.elasticbeanstalk contains a YAML file holding the configuration around the default branches and environments.

The aws-eb-master branch is linked to the production environment, whilst the aws-eb-test branch is linked to test.

.ebextensions

This directory (directory)https://github.com/BathHacked/energy-sparks/tree/master/.ebextensions contains customisations to the Elastic Beanstalk environment, including

• Setting up Swap space
• Installing Yarn for JavaScript package management
• Install lftp for doing SFTP to the B&NES ftp server
• Building caches for packages
• Setting up the cron jobs

Running tasks on the servers

To run tasks on the server as the appropriate user with the right environment, all tasks must be run with the bin/run_as_webapp wrapper. e.g.:

cd current
sudo bin/run_as_webapp bundle exec rake ...

Creating a new environment on Elastic Beanstalk

• Using the actions drop down, click Create Environment
• Choose Web server environment and click Continue
• Choose an environment name, normally we are using the AWS linux platform version, along with test or prod, i.e. prod-2-10-1 or test-2-8-7
• Choose preconfigured platform of Ruby
• Leave the application code setting as the Sample Application
• Click 'Configure More Options'
• From configuration presets, select 'Custom configuration'

Software

Leave this for now, we will add the environment variables once the environment has been created with the sample app

Instances:

• Set to use General Purpose SSD

• Size 30Gb

• EC2 security groups, just add the default security group for now

• Set instance to t2.small - make sure there is at least 20Gb disk space available.

Capacity:

• Change environment to load balanced
• Set instances Min 1 and Max 1
• Set instance type to t3.small

Load Balancer:

• Choose a Classic load balancer and leave settings as they are for now

Security:

• Add previously created key in security IAM DevOps (i.e. EnergySparksProductionEC2Machine or EnergySparksTestEC2)

Create environment

Click the Create environment button

Set up temporary certificate

• Go to the certificate manager
• Use a name like test-2-11-1.energysparks.uk
• Choose DNS validation
• Setp 5 validation will allow you to automatically set up the certificate in Route 53 - make sure you click the little triangle next to the domain name to open the information panel, this includes a button 'Create record in Route 53' - click this button!
• Click continue - it will take a few minutes to validate

Once environment has been created

• Set environment variables

get existing ones from an existing environment by running eb printenv and then set them on the new environment (easiest to have a branch which is configured as a default to use new environment using .elasticbeanstalk/config.yml) using eb setenv THIS_VAR=x THAT_VAR=y etc

• You will need to: * split the output from printenv in two and * remove the spaces which surround the = signs * take out AWS_ACCESS_KEY_ID as it will not process correctly * take out the mailchimp URL as it will not process either

• Check environment variables through web console and add AWS_ACCESS_KEY_ID and the Mailchimp URL back in again

• Click apply

• Check SSH works to new environment with eb ssh

• Then deploy actual branch

• Set up DNS in Route 53 if creating a brand new environment

• Set up cert and get it to create DNS record if you didn't do this already or are not using an existing one

• Add Listener to Load Balancer - Listener Port 443 Listener Protocol HTTPS Instance Port 80 Instance Protocol HTTP

  and add certificate once it has been validated

• If you are running this environment in parallel with an existing one, make sure you:

  • set the APPLICATION_HOST to what you want it to be
  • stop the CRON jobs running if you are sharing a database on one of the environments
  • Note that the CDN will complain because of CORS unless you create a distribution for that.

Sort out Database config

For existing database:

Get the RDS launch wizard group and add access INBOUND for the AWSEBSecurityGroup created by EB for the new instance

For new database:

1. Set up appropriate database in RDS - make sure the password doesn't have any (or too many) special characters, best to keep to digits and letters if possible!
2. Use pg_dump to get dump of current production database
3. Use psql to get data into new database