Category Archives: NodeJS

Process Twilio SMS Text Message in NodeJS in AWS Lambda Function via API Gateway

Settings up a Lambda Function on Amazon Web Services to properly accept and validate an incoming Twilio text message (SMS) is much harder than it should be. Here is a very basic configuration to pass through the submitted form data from Twilio as well as the Twilio validation header to your NodeJS script for processing. The first body mapping template converts all parameters (post vars for POST method, querystring for GET) into keys in your event object passed into your NodeJS Lambda function. Additionally I’m passing Twilio’s X-Twilio-Signature HTTP header through as “event.Signature” so you can properly validate the request as being from Twilio for full security. After processing we response with the XML response and pass it through the API gateway as-is back to Twilio through the use of an Integration Response and its Body Mapping Template. Overly complicated, I know, but this is the simplest I’ve figured out to do it and it seems to match even official Twilio tutorials for similar actions.

API Gateway Integration Request Body Mapping Template

Content type: application/x-www-form-urlencoded

Mapping Template Code:

## convert HTML POST data or HTTP GET query string to JSON

## get the raw post data from the AWS built-in variable and give it a nicer name
#if ($context.httpMethod == "POST")
#set($rawAPIData = $input.path("$"))
#elseif ($context.httpMethod == "GET")
#set($rawAPIData = $input.params().querystring)
#set($rawAPIData = $rawAPIData.toString())
#set($rawAPIDataLength = $rawAPIData.length() - 1)
#set($rawAPIData = $rawAPIData.substring(1, $rawAPIDataLength))
#set($rawAPIData = $rawAPIData.replace(", ", "&"))
#else
#set($rawAPIData = "")
#end

## first we get the number of "&" in the string, this tells us if there is more than one key value pair
#set($countAmpersands = $rawAPIData.length() - $rawAPIData.replace("&", "").length())

## if there are no "&" at all then we have only one key value pair.
## we append an ampersand to the string so that we can tokenise it the same way as multiple kv pairs.
## the "empty" kv pair to the right of the ampersand will be ignored anyway.
#if ($countAmpersands == 0)
#set($rawPostData = $rawAPIData + "&")
#end

## now we tokenise using the ampersand(s)
#set($tokenisedAmpersand = $rawAPIData.split("&"))

## we set up a variable to hold the valid key value pairs
#set($tokenisedEquals = [])

## now we set up a loop to find the valid key value pairs, which must contain only one "="
#foreach( $kvPair in $tokenisedAmpersand )
#set($countEquals = $kvPair.length() - $kvPair.replace("=", "").length())
#if ($countEquals == 1)
#set($kvTokenised = $kvPair.split("="))
#if ( ( $kvTokenised[0].length() > 0 ) && ( $kvTokenised[1].length() > 0 ) )
## we found a valid key value pair. add it to the list.
#set($devNull = $tokenisedEquals.add($kvPair))
#end
#end
#end

## next we set up our loop inside the output structure "{" and "}"
{
## Pass Twilio signature header through to event.Signature.
"Signature": "$input.params('X-Twilio-Signature')",
## Pass all valid params through into event var.
#foreach( $kvPair in $tokenisedEquals )
## finally we output the JSON for this pair and append a comma if this isn't the last pair
#set($kvTokenised = $kvPair.split("="))
"$util.urlDecode($kvTokenised[0])" : #if($kvTokenised.size() > 1 && $kvTokenised[1].length() > 0)"$util.urlDecode($kvTokenised[1])"#{else}""#end#if( $foreach.hasNext ),#end
#end
}

TIP: After setting up your mapping template don’t forget to DEPLOY your new changes!

NodeJS Code to Accept & Validate Incoming SMS from Twilio


var twilio = require('twilio');

twilio_auth_token = 'YOUR_AUTH_TOKEN_HERE';
this_api_url = 'LAMBDA_API_URL_HERE'; // Should match the full URL called by Twilio. Best to include trailing slash in both.

exports.handler = function(event, context, callback) {
console.log( 'Parameters:' );
console.log( event );

var twilio_params = JSON.parse( JSON.stringify( event ) ); // Clone event into new var so we can remove Signature param. Can't just copy var since it copies by reference.
delete twilio_params.Signature; // Held the X-Twilio-Signature header contents.
if ( true !== twilio.validateRequest( twilio_auth_token, event.Signature, this_api_url, twilio_params ) ) {
console.log( 'Twilio auth failure.' );
return callback( 'Twilio auth failure.' );
} else {
console.log( 'Twilio auth success.' );
}

twiml = new twilio.TwimlResponse();
// twiml.message( 'Uncomment this line to send this as a text message response back to the sender.' );
return callback( null, twiml.toString() ); // Success.
};

Method Response

Set the Method Response 200 HTTP Status Response Model to a Content type of application/xml instead of the default application/json.

Integration Response

You need to set the 200 response Body Mapping Template to directly output your XML directly. Set a Body Mapping Template of Content-type application/xml and for the template content use the following. Since we are using the twimlResponse() method to build the XML we don’t need to modify the response in any way before passing it out.

#set($inputRoot = $input.path('$'))
$inputRoot

External resources

https://forums.aws.amazon.com/thread.jspa?messageID=725034
https://www.twilio.com/docs/api/security
http://twilio.github.io/twilio-node/#validateRequest
http://stackoverflow.com/questions/728360/most-elegant-way-to-clone-a-javascript-object
https://www.twilio.com/blog/2015/09/build-your-own-ivr-with-aws-lambda-amazon-api-gateway-and-twilio.html

“invalid ELF header” running a NodeJS script on Raspberry Pi

Problem:

You see something like the following when trying to run your NodeJS script:

Error: [...]/node_modules/epoll/build/Release/epoll.node: invalid ELF header

Possible cause #1:

You’re using a Raspberry Pi (in my case 2 B) with Raspbian with the built-in Node v0.10.29 which is missing a UTF8 patch. See my other post HERE for the solution if you have the following error (you’ll just need to upgrade to v0.12.x or newer):

../node_modules/nan/nan.h:328:47: error: 'REPLACE_INVALID_UTF8' is not a member of 'v8::String'
static const unsigned kReplaceInvalidUtf8 = v8::String::REPLACE_INVALID_UTF8;

Possible cause #2:

You copied your node_modules directory over from your computer to the Raspberry Pi. npm needs to compile some node modules specially for the Raspberry Pi, so simply copying over the modules won’t always work.

Solution for cause #2:

Install your node modules on the Pi itself via npm, not just copy the node_modules directory over. This is because things need to be compiled slightly differently when running on the Pi. In my case epoll is a submodule of the repo “onoff”, so in my case I’d do the following on the Raspberry Pi where I want it installed:

npm install onoff

Note: If you don’t have npm installed on your Raspberry Pi yet, do the following:

sudo apt-get install nodejs npm

‘REPLACE_INVALID_UTF8’ is not a member of ‘v8::String’ installing NodeJS packages on Raspbian (Debian) on Raspberry Pi 2 B

Problem:

The current distribution of node (v0.10.29) packaged with the Debian distro that comes with the Raspberry Pi 2 B is missing a patch related to UTF8. Because of this you may encounter packages which are unable to compile. You’ll see the following before various errors and eventual failure of installing a package via npm. In my case this failed when trying to install the “onoff” package which relied on “epoll”.

The Error:

../node_modules/nan/nan.h:328:47: error: 'REPLACE_INVALID_UTF8' is not a member of 'v8::String'
static const unsigned kReplaceInvalidUtf8 = v8::String::REPLACE_INVALID_UTF8;

Solution:

Install a newer version of Node, such as v0.12.x:

curl -sL https://deb.nodesource.com/setup_0.12 | sudo -E bash -
sudo apt-get install -y nodejs

Verify this worked with:

node --version

That’s it! You should now be ready to resume installing your packages. Enjoy.

Websockets not working on Elastic Beanstalk with NodeJS when using nginx as a proxy

Amazon Web Service’s default nginx configuration does not have websocket support enabled by default. I am running SailsJS with Socket.io on NodeJS with the default nginx proxy and discovered that websockets were failing to connect. Creating a directory named .ebextensions in the root of my application and a file named 01_files.config within it with the configuration below solved the problem as it instructs nginx to pass websockets through. Just drop this in, deploy your app, and you should be good to go with your websockets functioning.

.ebextensions/01_files.config contents:


files:
    "/etc/nginx/conf.d/websocketupgrade.conf" :
        mode: "000755"
        owner: root
        group: root
        content: |
             proxy_set_header        Upgrade         $http_upgrade;
             proxy_set_header        Connection      "upgrade";

I searched high and low for a solution to this problem and after finding several similar but functioning solutions I combined them to get the above.