Category Archives: AWS

S3 PHP SDK v3: [Delete] is missing and is a required parameter

If you see this error after migrating from the v2 to v3 sdk change:

$s3client->deleteObjects( array(
'Bucket' => $settings['bucket'],
'Objects' => $fileKeys
) );

to:


$s3client->deleteObjects( array(
'Bucket' => $settings['bucket'],
'Delete' => array(
'Objects' => $fileKeys
)
) );

Note the new Delete array wrapping Objects. I discovered this while updating BackupBuddy.

S3 PHP SDK v3 CompleteMultipartUpload error using low level API or coming from v2 SDK

If you’re upgrading from the v2 to v3 SDK or following the low level API documentation for Multipart uploads such as http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFilePHP.html you will run into an impassable error. I ran into this problem while working on adding the new S3 SDK into BackupBuddy.

The Error:

Error executing “CompleteMultipartUpload” on “https://s3.amazonaws.com/[…]”; AWS HTTP error: Client error: `POST https://s3.amazonaws.com/[…]` resulted in a `400 Bad Request` response:
InvalidRequestYou must specify at least one part

The problem is that the CompleteMultipartUpload function parameters have changed but Amazon has not updated their documentation properly and it is missing from their migration guide.

Solution:

The “Parts” parameter now needs to be placed in a new array named “MultipartUpload”.

Incorrect parameters (as per documentation or v2 SDK):
$result = $s3->completeMultipartUpload(array(
'Bucket' => $bucket,
'Key' => $key,
'UploadId' => $uploadId,
'Parts' => $parts,
));

Correct parameters for v3 SDK:
$result = $s3->completeMultipartUpload(array(
'Bucket' => $bucket,
'Key' => $key,
'UploadId' => $uploadId,
'MultipartUpload' => array(
'Parts' => $parts,
),
));

I’ve posted a post in the S3 forums ( https://forums.aws.amazon.com/thread.jspa?messageID=740339&#740339 ) requesting Amazon update their documentation so this may be fixed there at some point. Special thanks to this StackOverflow post: http://stackoverflow.com/questions/30383982/aws-s3-completemultipartupload-error

Process Twilio SMS Text Message in NodeJS in AWS Lambda Function via API Gateway

Settings up a Lambda Function on Amazon Web Services to properly accept and validate an incoming Twilio text message (SMS) is much harder than it should be. Here is a very basic configuration to pass through the submitted form data from Twilio as well as the Twilio validation header to your NodeJS script for processing. The first body mapping template converts all parameters (post vars for POST method, querystring for GET) into keys in your event object passed into your NodeJS Lambda function. Additionally I’m passing Twilio’s X-Twilio-Signature HTTP header through as “event.Signature” so you can properly validate the request as being from Twilio for full security. After processing we response with the XML response and pass it through the API gateway as-is back to Twilio through the use of an Integration Response and its Body Mapping Template. Overly complicated, I know, but this is the simplest I’ve figured out to do it and it seems to match even official Twilio tutorials for similar actions.

API Gateway Integration Request Body Mapping Template

Content type: application/x-www-form-urlencoded

Mapping Template Code:

## convert HTML POST data or HTTP GET query string to JSON

## get the raw post data from the AWS built-in variable and give it a nicer name
#if ($context.httpMethod == "POST")
#set($rawAPIData = $input.path("$"))
#elseif ($context.httpMethod == "GET")
#set($rawAPIData = $input.params().querystring)
#set($rawAPIData = $rawAPIData.toString())
#set($rawAPIDataLength = $rawAPIData.length() - 1)
#set($rawAPIData = $rawAPIData.substring(1, $rawAPIDataLength))
#set($rawAPIData = $rawAPIData.replace(", ", "&"))
#else
#set($rawAPIData = "")
#end

## first we get the number of "&" in the string, this tells us if there is more than one key value pair
#set($countAmpersands = $rawAPIData.length() - $rawAPIData.replace("&", "").length())

## if there are no "&" at all then we have only one key value pair.
## we append an ampersand to the string so that we can tokenise it the same way as multiple kv pairs.
## the "empty" kv pair to the right of the ampersand will be ignored anyway.
#if ($countAmpersands == 0)
#set($rawPostData = $rawAPIData + "&")
#end

## now we tokenise using the ampersand(s)
#set($tokenisedAmpersand = $rawAPIData.split("&"))

## we set up a variable to hold the valid key value pairs
#set($tokenisedEquals = [])

## now we set up a loop to find the valid key value pairs, which must contain only one "="
#foreach( $kvPair in $tokenisedAmpersand )
#set($countEquals = $kvPair.length() - $kvPair.replace("=", "").length())
#if ($countEquals == 1)
#set($kvTokenised = $kvPair.split("="))
#if ( ( $kvTokenised[0].length() > 0 ) && ( $kvTokenised[1].length() > 0 ) )
## we found a valid key value pair. add it to the list.
#set($devNull = $tokenisedEquals.add($kvPair))
#end
#end
#end

## next we set up our loop inside the output structure "{" and "}"
{
## Pass Twilio signature header through to event.Signature.
"Signature": "$input.params('X-Twilio-Signature')",
## Pass all valid params through into event var.
#foreach( $kvPair in $tokenisedEquals )
## finally we output the JSON for this pair and append a comma if this isn't the last pair
#set($kvTokenised = $kvPair.split("="))
"$util.urlDecode($kvTokenised[0])" : #if($kvTokenised.size() > 1 && $kvTokenised[1].length() > 0)"$util.urlDecode($kvTokenised[1])"#{else}""#end#if( $foreach.hasNext ),#end
#end
}

TIP: After setting up your mapping template don’t forget to DEPLOY your new changes!

NodeJS Code to Accept & Validate Incoming SMS from Twilio


var twilio = require('twilio');

twilio_auth_token = 'YOUR_AUTH_TOKEN_HERE';
this_api_url = 'LAMBDA_API_URL_HERE'; // Should match the full URL called by Twilio. Best to include trailing slash in both.

exports.handler = function(event, context, callback) {
console.log( 'Parameters:' );
console.log( event );

var twilio_params = JSON.parse( JSON.stringify( event ) ); // Clone event into new var so we can remove Signature param. Can't just copy var since it copies by reference.
delete twilio_params.Signature; // Held the X-Twilio-Signature header contents.
if ( true !== twilio.validateRequest( twilio_auth_token, event.Signature, this_api_url, twilio_params ) ) {
console.log( 'Twilio auth failure.' );
return callback( 'Twilio auth failure.' );
} else {
console.log( 'Twilio auth success.' );
}

twiml = new twilio.TwimlResponse();
// twiml.message( 'Uncomment this line to send this as a text message response back to the sender.' );
return callback( null, twiml.toString() ); // Success.
};

Method Response

Set the Method Response 200 HTTP Status Response Model to a Content type of application/xml instead of the default application/json.

Integration Response

You need to set the 200 response Body Mapping Template to directly output your XML directly. Set a Body Mapping Template of Content-type application/xml and for the template content use the following. Since we are using the twimlResponse() method to build the XML we don’t need to modify the response in any way before passing it out.

#set($inputRoot = $input.path('$'))
$inputRoot

External resources

https://forums.aws.amazon.com/thread.jspa?messageID=725034
https://www.twilio.com/docs/api/security
http://twilio.github.io/twilio-node/#validateRequest
http://stackoverflow.com/questions/728360/most-elegant-way-to-clone-a-javascript-object
https://www.twilio.com/blog/2015/09/build-your-own-ivr-with-aws-lambda-amazon-api-gateway-and-twilio.html

Connecting to mySQL instance over SSL on Amazon RDS with Sequel Pro

1. Download the root certificate (bundle) here.
2. Download the intermediate certificate for YOUR instance’s region here.

Open both files in a text editor. Copy the contents of the intermediate certificate to the TOP of the root certificate above the existing contents.

Point Sequel Pro to this file hen selecting the “Key file”. Leave other SSL options blank except for the checkbox enabling SSL.

If you do this wrong you will get an error like:

SSL connection error: ASN: bad other signature confirmation

Amazon S3 PHP SDK v2 – getCommand() must be of the type array, string given – Solution!

Catchable fatal error: Argument 2 passed to Guzzle\Service\Client::getCommand() must be of the type array, string given, called in ../_s3lib2/Guzzle/Service/Client.php on line 76 and defined in ../_s3lib2/Guzzle/Service/Client.php on line 79

If you’re getting this error when working with the version 2 PHP SDK for Amazon Web Services’ S3 service, fear not. It’s probably as simple as you passing the wrong parameters into the function. In this example I was calling listObjects() using the old SDK’s parameters rather than the new SDK’s. Simply fix!

Wrong (eg. old SDK v1 way):

$response = self::$_client->listObjects( $settings['bucket'] );

Right (eg. new SDK v2 way):

$response = self::$_client->listObjects( array( 'Bucket' => $settings['bucket'], 'Prefix' => $prefix ) );

All fixed!