Always Get Better

Archive for the ‘AWS’ Category

Fun With Amazon X-Ray

Sunday, November 19th, 2017

While riding on the train the other day, I was watching The Man in the High Castle on Prime and thought I recognized the actor on screen. “Too bad they only show the main actors. It would be nice if we could get a listing of who is in each scene,” I thought to myself. Imagine my surprise when I found out the app does exactly that.

So Cool

The info screen doesn’t just show the main actors for the show like I thought, it shows who is in the current scene!

It even changes as the scene changes. If you leave the info screen on, the actor list updates in real time as actors enter and leave the screen.

Trivia

The info screen doesn’t just show which actors are in the scene, it shows trivia about things going on in the scene; either actions of the characters or story background based on what’s being said.

As the action unfolds, the info changes to provide even more context.

How It Works

As best I can tell (maybe someone has better information?) the x-ray data is filled in large part algorithmically.

The main actor data comes from IMDB. I assume Amazon is using facial recognition technology to match profile photos with actors in each frame. Background actors aren’t included so there’s some intelligence there to either filter out people by screen time or dialogue.

There’s no way everything is done automatically, so there has to be a human component in there as well. Very likely they are using something like mechanical turk to have humans verify the raw data and add contextual tagging.

Your World Augmented

When I think about “augmented reality” this is the kind of thing I’m excited to see. Computer systems that provide more context about the world around us, not try to replace it entirely. Humans augmented by machines to make our whole experience better.

Next time you ask “who is that actor!?” try checking the app you’re watching – it may be able to tell you more authoritatively than a Google search or the person you’re sitting with.

Reverse Proxy with IIS and Elastic Beanstalk

Monday, September 4th, 2017

Suppose your main website is a .NET/IIS stack running on AWS Elastic Beanstalk, and you decided to add a WordPress blog to the mix. Instead of having http://blog.yourdomain.com and http://www.yourdomain.com, you want to host the blog from a subdirectory at http://www.yourdomain.com/blog. There are a few ways you can go about this; this post will go through how to set up a reverse proxy to a dedicated blog server.

You have two options:

1. Host WordPress in IIS – Not fun. You need to configure IIS to run PHP, host a MySQL database, and manage your WordPress file directory and updates in an environment where user uploads and core files can get trashed at any minute when EB rolls out a new server. It’s possible to run an amazing HA WordPress install on Elastic Beanstalk (topic for another post) but in a subdirectory directly hosted in Windows/IIS? Not for the feint of heart)

2. Host WordPress on a Linux server somewhere else and reverse proxy – Let’s do this instead.

Basically you set up WordPress and get it to look how you want, configure it to respond to your domain, then configure IIS to fetch your blog URLs and return them whenever a request comes through to the blog subdirectory on your main site. Every other directory is served from your .NET app as normal.

Reverse Proxy a WordPress blog with IIS

Important: The final step to this is the most important bit if you’re running on Elastic Beanstalk. Make sure you follow it in order to actually enable IIS’ proxy module, which does not come pre-installed on Elastic Beanstalk’s AMI.

Configure WordPress to Respond to Your Domain

Add this section to the bottom of wp-config.php, just before the require_once for wp-settings.php. This tells WordPress to ignore its default server settings and use your website as its base path.

$hostname = 'www.yourdomain.com';
$_SERVER['HTTP_HOST'] = $hostname;
define('WP_HOME','https://www.yourdomain.com/blog');
define('WP_SITEURL','https://www.yourdomain.com/blog');
$current_url = 'https://' . $hostname . $_SERVER['REQUEST_URI'];

Configure IIS Paths

The reverse proxy makes use of the rewrite in IIS. To turn this on for your blog, add the following to the directive in your web.config file:

 <rewrite>
   <rules>
     <rule name="Reverse Proxy to blog" stopProcessing="true">
       <match url="^blog/(.*)" />
       <action type="Rewrite" url="https://blog.yourdomain.com/{R:1}" />
     </rule>
   </rules>
 </rewrite>

This tells IIS to redirect all requests beginning with “blog/” to your WordPress blog. IIS will reach out to your blog server as if it is the requestor, fetch the page, and return it as if the blog were hosted from within IIS itself. The {R:1} variable carries forward any path information to the blog — so your theme files, user uploads and static assets will all pass through.

If you deploy your site and try to access your blog page now, it won’t work. You’ll see a ‘404’ response code even though the rules are definitely set up properly. The final step to this is enabling the Application Request Routing module on IIS – this is not enabled by default in Elastic Beanstalk’s version of Windows.

Enabling the Reverse Proxy Module on Elastic Beanstalk

You could Remote Desktop into your web server machine and manually enable the ARR module, but it would stop working the next time your environment flips, the server gets reloaded for any reason (which can happen even if you are doing all at once deployments and not adding/removing machines), or nodes get added to your environment.

We need to make sure the module gets installed and checked every time you deploy your files, so it’s always present and available to use even when new machines come online.

To do that, we’ll use the .ebextensions scripting hooks to download, install and configure ARR every time a deploy runs.

1. Download the ARR Installer

Download the 65-bit ARR installer (from here) to S3 so it is available to your VM when it boots. We want to install to S3 instead of pulling directly off Microsoft’s servers because we can’t rely on outside links being available when we need to deploy, and if our VM happens to be inside a VM without a NAT then we can use Amazon’s S3 internal endpoints without needing to configure any more advanced network.

2. Add an ebextension hook

In your .ebextensions folder (at the root of your unzipped deploy package), add a new config file (install-arr.config) to instruct Elastic Beanstalk how to install the extension:

packages:
  msi:
    ApplicationRequestRouting: "pathtoyourinstallerons3"

commands:
  01_enablearr:
    command: "C:\\Windows\\system32\\inetsrv\\appcmd.exe set config  -section:system.webServer/proxy /enabled:True  /commit:apphost"

The packages/msi lines tell Elastic Beanstalk to download and run the installer. Since you won’t be physically present when that happens, the script will automatically accept all the license agreements and silently run.

The appcmd command instructs IIS to enable the reverse proxy module, which turns your rewrite instructions into actual reverse proxy commands. Now if you visit www.yourdomain.com/blog/, you will see your WordPress blog.

Bonus: Trailing Slashes

If you visit www.yourdomain.com/blog without a trailing slash, you won’t see the blog. You don’t want to start the rewrite rule at this level because your reverse proxy will try to access blog.yourdomain.com// (with a double slash) for all dependent resources, which can cause problems with WordPress’ routing.

For this case, I like to redirect to a trailing slash at the IIS level. Every time someone comes to yourdomain.com/blog, they will redirect with a clean 302 status command to yourdomain.com/blog/. Just add this rule in your section of web.config

<rule name="Add trailing slash to blog reverse proxy" stopProcessing="true">
  <matchurl="^blog$"/>
  <action type="Redirect" redirectType="Permanent" url="{R:1}/"/>
</rule>

The regular expression in the match url tells the web server this should only apply to the “blog” path, that is, nothing before or after the word “blog”. This lets you have “blog” in other parts of your URL without accidentally redirecting valid pages.

Serverless Contact Forms with AWS Lambda

Thursday, October 27th, 2016

LAMB-da - Serverless Contact FormsServers are a pain to run. They break, get hacked, need updating, and generally need your constant attention or that site you posted two years ago won’t work when you need to make a change. Static sites are a beautiful dream, but what do you do when you need user input? You don’t want to use a third-party service just to get rare contact forms from your visitors. It’s stupid to run a web server to handle this; that completely eliminates the whole purpose of creating a static site. What you need are serverless contact forms.

Architecture

I use Wufoo forms for most of my static sites but today I’m switching to Lambdas. To start, I have a static Jekyll site uploaded to S3. I have a CloudFront distribution set up for edge-caching and SSL. Now I’m going to add a contact form that reaches through AWS’ API Gateway to execute a node.js script hosted on Lambda.

Here is the general data flow:

Serverless Contact Forms Architecture

Serverless Contact Forms Architecture

Email Script

I first write a simple Node.js script that validates the contact form parameters then pumps the messages into the Sendgrid API. You can swap in your preferred client. AWS SES is a popular one for sure and a nice way to keep everything under one umbrella.

Setting Up the Lambda

First create a blank template.

Serverless Contact Forms start with a blank lambda

Serverless Contact Forms start with a blank lambda

Next create an API Gateway trigger for the Lambda. I set my security to ‘Open’ because I am allowing anonymous traffic to send contact forms.

Serverless Contact Forms API Gateway Settings

Serverless Contact Forms API Gateway Settings

Finally, upload the contact form script. Since I’ve used libraries other than Amazon’s own I need to zip everything up and send it as an archive.

Serverless Contact Forms Lambda Setup

Serverless Contact Forms Lambda Setup

Configuring the API Gateway for Serverless Contact Forms

Now that the Lambda function has been created it’s time to open it to the outside world. Go to the ‘API Gateway’ service, find your endpoint, and choose Actions -> Create Method. Add a POST method pointing to your Lambda function’s name.

Serverless Contact Forms API Gateway Setup

Serverless Contact Forms API Gateway Setup

You need to enable CORS because you will be using AJAX to submit the form. This is a cross-domain request while we’re testing it out.

lambda-cors

Then deploy the API. The default environment is prod, which is good enough for our purposes.

lambda-5

Finally you see the the endpoint URL you will use for your form.

lambda-6

Serverless Contact Forms

That’s the backend all set up now you can build out the contact form your visitors will use to contact you. Here’s mine. It’s super ugly but it gets the point across.

When you hit submit the send function executes, posting the email address and message to the endpoint and alerting the results. Nothing fancy, and obviously you would need to code in all the proper validation and UI stuff to make this pretty.

CloudFront Configuration

I can get into the static site setup later, but for now I’m working from an existing distribution.

Since CORS is all set up you could use the endpoint as-is and just launch your contact form, but that’s not as elegant as posting to the same domain as the form itself. You can achieve this illusion because CloudFront is able to forward POST requests now.

Add the origin for the API to the distribution.

lambda-7

Tell CloudFront to forward (and not cache!) your contact form by adding a behaviour for the path where you want to host the form. The path name should match your API resource name:

lambda-8

For this to work make sure you choose the Allowed HTTP Methods option with POST. Setting the Forward Headers option to ALL causes CloudFront to use the origin cache settings (no cache) which will let more than one person use your contact form.

Profit

Overall this feels like it is a longer process than it should be. The first time I did this it took almost 5 hours, but now that I know the process I’m sure next time will be a lot faster. Figuring out the right folders and permission settings (for CORS) was the most finicky part of the process, but the API Gateway has an informative test tool that helped a lot.

This is only going to get better, and Lambda is a fantastic cost-effective tool for replacing on-demand tasks where you would once need to spin up and support a whole server. Serverless contact forms are definitely the way to go if your site is otherwise static.