Category Archives: Javascript

Process Twilio SMS Text Message in NodeJS in AWS Lambda Function via API Gateway

Settings up a Lambda Function on Amazon Web Services to properly accept and validate an incoming Twilio text message (SMS) is much harder than it should be. Here is a very basic configuration to pass through the submitted form data from Twilio as well as the Twilio validation header to your NodeJS script for processing. The first body mapping template converts all parameters (post vars for POST method, querystring for GET) into keys in your event object passed into your NodeJS Lambda function. Additionally I’m passing Twilio’s X-Twilio-Signature HTTP header through as “event.Signature” so you can properly validate the request as being from Twilio for full security. After processing we response with the XML response and pass it through the API gateway as-is back to Twilio through the use of an Integration Response and its Body Mapping Template. Overly complicated, I know, but this is the simplest I’ve figured out to do it and it seems to match even official Twilio tutorials for similar actions.

API Gateway Integration Request Body Mapping Template

Content type: application/x-www-form-urlencoded

Mapping Template Code:

## convert HTML POST data or HTTP GET query string to JSON

## get the raw post data from the AWS built-in variable and give it a nicer name
#if ($context.httpMethod == "POST")
#set($rawAPIData = $input.path("$"))
#elseif ($context.httpMethod == "GET")
#set($rawAPIData = $input.params().querystring)
#set($rawAPIData = $rawAPIData.toString())
#set($rawAPIDataLength = $rawAPIData.length() - 1)
#set($rawAPIData = $rawAPIData.substring(1, $rawAPIDataLength))
#set($rawAPIData = $rawAPIData.replace(", ", "&"))
#else
#set($rawAPIData = "")
#end

## first we get the number of "&" in the string, this tells us if there is more than one key value pair
#set($countAmpersands = $rawAPIData.length() - $rawAPIData.replace("&", "").length())

## if there are no "&" at all then we have only one key value pair.
## we append an ampersand to the string so that we can tokenise it the same way as multiple kv pairs.
## the "empty" kv pair to the right of the ampersand will be ignored anyway.
#if ($countAmpersands == 0)
#set($rawPostData = $rawAPIData + "&")
#end

## now we tokenise using the ampersand(s)
#set($tokenisedAmpersand = $rawAPIData.split("&"))

## we set up a variable to hold the valid key value pairs
#set($tokenisedEquals = [])

## now we set up a loop to find the valid key value pairs, which must contain only one "="
#foreach( $kvPair in $tokenisedAmpersand )
#set($countEquals = $kvPair.length() - $kvPair.replace("=", "").length())
#if ($countEquals == 1)
#set($kvTokenised = $kvPair.split("="))
#if ( ( $kvTokenised[0].length() > 0 ) && ( $kvTokenised[1].length() > 0 ) )
## we found a valid key value pair. add it to the list.
#set($devNull = $tokenisedEquals.add($kvPair))
#end
#end
#end

## next we set up our loop inside the output structure "{" and "}"
{
## Pass Twilio signature header through to event.Signature.
"Signature": "$input.params('X-Twilio-Signature')",
## Pass all valid params through into event var.
#foreach( $kvPair in $tokenisedEquals )
## finally we output the JSON for this pair and append a comma if this isn't the last pair
#set($kvTokenised = $kvPair.split("="))
"$util.urlDecode($kvTokenised[0])" : #if($kvTokenised.size() > 1 && $kvTokenised[1].length() > 0)"$util.urlDecode($kvTokenised[1])"#{else}""#end#if( $foreach.hasNext ),#end
#end
}

TIP: After setting up your mapping template don’t forget to DEPLOY your new changes!

NodeJS Code to Accept & Validate Incoming SMS from Twilio


var twilio = require('twilio');

twilio_auth_token = 'YOUR_AUTH_TOKEN_HERE';
this_api_url = 'LAMBDA_API_URL_HERE'; // Should match the full URL called by Twilio. Best to include trailing slash in both.

exports.handler = function(event, context, callback) {
console.log( 'Parameters:' );
console.log( event );

var twilio_params = JSON.parse( JSON.stringify( event ) ); // Clone event into new var so we can remove Signature param. Can't just copy var since it copies by reference.
delete twilio_params.Signature; // Held the X-Twilio-Signature header contents.
if ( true !== twilio.validateRequest( twilio_auth_token, event.Signature, this_api_url, twilio_params ) ) {
console.log( 'Twilio auth failure.' );
return callback( 'Twilio auth failure.' );
} else {
console.log( 'Twilio auth success.' );
}

twiml = new twilio.TwimlResponse();
// twiml.message( 'Uncomment this line to send this as a text message response back to the sender.' );
return callback( null, twiml.toString() ); // Success.
};

Method Response

Set the Method Response 200 HTTP Status Response Model to a Content type of application/xml instead of the default application/json.

Integration Response

You need to set the 200 response Body Mapping Template to directly output your XML directly. Set a Body Mapping Template of Content-type application/xml and for the template content use the following. Since we are using the twimlResponse() method to build the XML we don’t need to modify the response in any way before passing it out.

#set($inputRoot = $input.path('$'))
$inputRoot

External resources

https://forums.aws.amazon.com/thread.jspa?messageID=725034
https://www.twilio.com/docs/api/security
http://twilio.github.io/twilio-node/#validateRequest
http://stackoverflow.com/questions/728360/most-elegant-way-to-clone-a-javascript-object
https://www.twilio.com/blog/2015/09/build-your-own-ivr-with-aws-lambda-amazon-api-gateway-and-twilio.html

Websockets not working on Elastic Beanstalk with NodeJS when using nginx as a proxy

Amazon Web Service’s default nginx configuration does not have websocket support enabled by default. I am running SailsJS with Socket.io on NodeJS with the default nginx proxy and discovered that websockets were failing to connect. Creating a directory named .ebextensions in the root of my application and a file named 01_files.config within it with the configuration below solved the problem as it instructs nginx to pass websockets through. Just drop this in, deploy your app, and you should be good to go with your websockets functioning.

.ebextensions/01_files.config contents:


files:
    "/etc/nginx/conf.d/websocketupgrade.conf" :
        mode: "000755"
        owner: root
        group: root
        content: |
             proxy_set_header        Upgrade         $http_upgrade;
             proxy_set_header        Connection      "upgrade";

I searched high and low for a solution to this problem and after finding several similar but functioning solutions I combined them to get the above.

Deploying first SailsJS node.js app to DigitalOcean using Dokku on Mac

Here’s a (very) quick and dirty overview of the steps. Follow everything below in order and you should have your node.js app (optionally running SailsJS) deployed on a dokku server on DigitalOcean within a few minutes.

Useful Links:
* Dokku – https://github.com/progrium/dokku
* MariaDB plugin – https://github.com/Kloadut/dokku-md-plugin


Set your SSH key into DigitalOcean if you haven’t already
* cat ~/.ssh/id_rsa.pub | pbcopy
* Paste this key into digitalocean control panel.


Create Droplet

* Select 1GB
* San Fran
* Dokku
* Select your existing SSH key.
* Virtio + backups (maybe virt network if need to connect)


Set up Dokku

* Visit in your browser: http://YOUR-SERVER-IP/
* Submit, optionally setting the app URL to use subdomains. Most settings should default to correct at this point.

Initialize the app access dokku

* cat ~/.ssh/id_rsa.pub | ssh [email protected] "sudo sshcommand acl-add dokku YOUR-APP-NAME"
** Note that if instead of YOUR-APP-NAME you put a full domain it will use that as the URL instead of setting up a subdomain. Eg api.dustinbolton.com instead of dustinbolton-api


Configuring local app

Initialize the local repo if you have not already
* git init && git add -A && git commit -m "Initial commit"

Assign production (or staging) destination:
* git remote add production [email protected]:YOUR-APP-NAME

Create file to tell server what to run on deployment:
* touch Procfile && open Procfile
* Add the following into this new file & save:
* web: sails lift
** For non-sails framework, instead of “sails lift” this would be “node app.js”.


Deploy & launch the app

* git push -u production master
<3>Done!


If you need to manually re-run the app or see the output of the run attempt (such as to troubleshoot):
* ssh [email protected]
cd /home/dokku/YOUR-APP-NAME
dokku run YOUR-APP-NAME sails lift
(or instead of “sails lift”, “node app.js”)


Adapted from the guide at:
http://matthewpalmer.net/blog/2014/02/19/how-to-deploy-node-js-apps-on-digitalocean-with-dokku/


MariaDB (mysql drop-in alternative) plugin:

cd /var/lib/dokku/plugins
git clone https://github.com/Kloadut/dokku-md-plugin mariadb
#git clone https://github.com/musicglue/dokku-user-env-compile.git user-env-compile
dokku plugins-install


Create Database Instance

dokku mariadb:create YOUR-APP-NAME
* This creates the mariadb instance and links it to your app automatically since the name matches. The database name is “db” by default.

You can use the credentials displayed or access them via environmental variables such by: process.env.DB_USER


Setting Custom Environment Variables

Database environment variables should automatically exist but I have seen them drop off. I have not found the cause yet and have decided to manually set the one(s) I need for the time being.
dokku config:set YOUR-APP-NAME DATABASE_URL=whateverhere


If you ever need to restart your app. Only change “YOUR-APP-NAME”:

docker restart `cat /home/dokku/YOUR-APP-NAME/CONTAINER`

Programmatically submitting an HTML form via jQuery submit() or trigger(‘submit’) doesn’t work? Solution!

Say you are trying to submit an HTML form via jQuery. If you’re trying to submit it using .submit() or .trigger(‘submit’) you may have ran into a very strange problem where nothing happens for no reason you can figure out. Here is a very basic example below. Why on Earth is this not working?

[code lang=”js” highlight=”12″]

<script type="text/javascript">

jQuery(‘#my-form-submit’).click( function(e) {
e.preventDefault();
// Do some stuff here…
jQuery( ‘#my-form’).submit();
});

</script>

<form action="?ajax=4" method="post" id="my-form">
<input type="submit" name="submit" value="Submit this form!" id="my-form-submit">
</form>

[/code]

 

Browsers reserve some key words for input names, such as “submit”. As you see in line 12 we have name=”submit”. This simple thing is the problem. Just change name=”submit” to name=”my-submit”:

Does not play nicely with jQuery triggering of submit:
[code lang=”js” highlight=”12″ firstline=”12″]

<input type="submit" name="submit" value="Submit this form!" id="my-form-submit">

[/code]

Works:

[code lang=”js” highlight=”12″ firstline=”12″]

<input type="submit" name="my-submit" value="Submit this form!" id="my-form-submit">

[/code]

Unfortunately browsers offer no error or exlanation to the developer as to why triggering the submit is not doing anything. This can be quite frustrating until you figure this tiny aspect out.