All posts by Dustin Bolton

Mae Ploy Thai Yellow Curry Recipe

Buy at Amazon.com

⁃ Measure out 50 grams of curry paste and put in sauce pan
⁃ Put about twice as much (~100grams, doesn’t need to be exact) of the coconut milk in sauce pain
⁃ Gently fry the paste in the coconut oil until fragrant (1 or 2 minutes) to bring out flavors
⁃ Add rest of coconut milk from can and turn up heat to start bringing to boil
⁃ Add in Xylitol or other sweetener to taste
⁃ Add in vegetables or meats in order of how long they take so they all finish cooking around the same time such as
⁃ Carrots
⁃ Onions
⁃ Potatoes
⁃ Chicken
⁃ Shrimp
⁃ Reduce to simmering once it comes to a boil
⁃ It’s ready when veggies and meat are done to your liking.

Notes:
Serve on steamed white (Jasmine tastiest) or brown rice, converted rice (lower glycemix index), cauliflower rice, or try things.
Top with a handful of raw cashews.
If you want it thicker then let paste/coconut milk mixture cook longer before adding veggies and meat.
If flavor is too strong then skip frying the paste at the beginning and just mix paste and coconut milk together at the start.
Ground beef does NOT work well with this. It soaks up the curry and then you can’t taste it.

S3 PHP SDK v3: [Delete] is missing and is a required parameter

If you see this error after migrating from the v2 to v3 sdk change:

$s3client->deleteObjects( array(
'Bucket' => $settings['bucket'],
'Objects' => $fileKeys
) );

to:


$s3client->deleteObjects( array(
'Bucket' => $settings['bucket'],
'Delete' => array(
'Objects' => $fileKeys
)
) );

Note the new Delete array wrapping Objects. I discovered this while updating BackupBuddy.

S3 PHP SDK v3 CompleteMultipartUpload error using low level API or coming from v2 SDK

If you’re upgrading from the v2 to v3 SDK or following the low level API documentation for Multipart uploads such as http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFilePHP.html you will run into an impassable error. I ran into this problem while working on adding the new S3 SDK into BackupBuddy.

The Error:

Error executing “CompleteMultipartUpload” on “https://s3.amazonaws.com/[…]”; AWS HTTP error: Client error: `POST https://s3.amazonaws.com/[…]` resulted in a `400 Bad Request` response:
InvalidRequestYou must specify at least one part

The problem is that the CompleteMultipartUpload function parameters have changed but Amazon has not updated their documentation properly and it is missing from their migration guide.

Solution:

The “Parts” parameter now needs to be placed in a new array named “MultipartUpload”.

Incorrect parameters (as per documentation or v2 SDK):
$result = $s3->completeMultipartUpload(array(
'Bucket' => $bucket,
'Key' => $key,
'UploadId' => $uploadId,
'Parts' => $parts,
));

Correct parameters for v3 SDK:
$result = $s3->completeMultipartUpload(array(
'Bucket' => $bucket,
'Key' => $key,
'UploadId' => $uploadId,
'MultipartUpload' => array(
'Parts' => $parts,
),
));

I’ve posted a post in the S3 forums ( https://forums.aws.amazon.com/thread.jspa?messageID=740339&#740339 ) requesting Amazon update their documentation so this may be fixed there at some point. Special thanks to this StackOverflow post: http://stackoverflow.com/questions/30383982/aws-s3-completemultipartupload-error

Execution failed due to configuration error: API Gateway does not have permission to assume the provided role

When setting up a custom lambda authorizer in Amazon Web Service’s API Gateway you have the option to specify a role. If you run into permissions problem with this it is often due to a misconfigured “Trust Relationship” with the role it is assuming. You will see an error:

Execution failed due to configuration error: API Gateway does not have permission to assume the provided role
Execution failed due to configuration error: Authorizer error
AuthorizerConfigurationException

To fix this go to the role in your IAM and select the “Trust Relationships” tab. From here edit the policy and for the Principal Service add in “apigateway.amazonaws.com” as seen below. This will grant the API Gateway the ability to assume roles to run your function in addition to the existing lambda permission.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": ["apigateway.amazonaws.com","lambda.amazonaws.com"]
},
"Action": "sts:AssumeRole"
}
]
}

Process Twilio SMS Text Message in NodeJS in AWS Lambda Function via API Gateway

Settings up a Lambda Function on Amazon Web Services to properly accept and validate an incoming Twilio text message (SMS) is much harder than it should be. Here is a very basic configuration to pass through the submitted form data from Twilio as well as the Twilio validation header to your NodeJS script for processing. The first body mapping template converts all parameters (post vars for POST method, querystring for GET) into keys in your event object passed into your NodeJS Lambda function. Additionally I’m passing Twilio’s X-Twilio-Signature HTTP header through as “event.Signature” so you can properly validate the request as being from Twilio for full security. After processing we response with the XML response and pass it through the API gateway as-is back to Twilio through the use of an Integration Response and its Body Mapping Template. Overly complicated, I know, but this is the simplest I’ve figured out to do it and it seems to match even official Twilio tutorials for similar actions.

API Gateway Integration Request Body Mapping Template

Content type: application/x-www-form-urlencoded

Mapping Template Code:

## convert HTML POST data or HTTP GET query string to JSON

## get the raw post data from the AWS built-in variable and give it a nicer name
#if ($context.httpMethod == "POST")
#set($rawAPIData = $input.path("$"))
#elseif ($context.httpMethod == "GET")
#set($rawAPIData = $input.params().querystring)
#set($rawAPIData = $rawAPIData.toString())
#set($rawAPIDataLength = $rawAPIData.length() - 1)
#set($rawAPIData = $rawAPIData.substring(1, $rawAPIDataLength))
#set($rawAPIData = $rawAPIData.replace(", ", "&"))
#else
#set($rawAPIData = "")
#end

## first we get the number of "&" in the string, this tells us if there is more than one key value pair
#set($countAmpersands = $rawAPIData.length() - $rawAPIData.replace("&", "").length())

## if there are no "&" at all then we have only one key value pair.
## we append an ampersand to the string so that we can tokenise it the same way as multiple kv pairs.
## the "empty" kv pair to the right of the ampersand will be ignored anyway.
#if ($countAmpersands == 0)
#set($rawPostData = $rawAPIData + "&")
#end

## now we tokenise using the ampersand(s)
#set($tokenisedAmpersand = $rawAPIData.split("&"))

## we set up a variable to hold the valid key value pairs
#set($tokenisedEquals = [])

## now we set up a loop to find the valid key value pairs, which must contain only one "="
#foreach( $kvPair in $tokenisedAmpersand )
#set($countEquals = $kvPair.length() - $kvPair.replace("=", "").length())
#if ($countEquals == 1)
#set($kvTokenised = $kvPair.split("="))
#if ( ( $kvTokenised[0].length() > 0 ) && ( $kvTokenised[1].length() > 0 ) )
## we found a valid key value pair. add it to the list.
#set($devNull = $tokenisedEquals.add($kvPair))
#end
#end
#end

## next we set up our loop inside the output structure "{" and "}"
{
## Pass Twilio signature header through to event.Signature.
"Signature": "$input.params('X-Twilio-Signature')",
## Pass all valid params through into event var.
#foreach( $kvPair in $tokenisedEquals )
## finally we output the JSON for this pair and append a comma if this isn't the last pair
#set($kvTokenised = $kvPair.split("="))
"$util.urlDecode($kvTokenised[0])" : #if($kvTokenised.size() > 1 && $kvTokenised[1].length() > 0)"$util.urlDecode($kvTokenised[1])"#{else}""#end#if( $foreach.hasNext ),#end
#end
}

TIP: After setting up your mapping template don’t forget to DEPLOY your new changes!

NodeJS Code to Accept & Validate Incoming SMS from Twilio


var twilio = require('twilio');

twilio_auth_token = 'YOUR_AUTH_TOKEN_HERE';
this_api_url = 'LAMBDA_API_URL_HERE'; // Should match the full URL called by Twilio. Best to include trailing slash in both.

exports.handler = function(event, context, callback) {
console.log( 'Parameters:' );
console.log( event );

var twilio_params = JSON.parse( JSON.stringify( event ) ); // Clone event into new var so we can remove Signature param. Can't just copy var since it copies by reference.
delete twilio_params.Signature; // Held the X-Twilio-Signature header contents.
if ( true !== twilio.validateRequest( twilio_auth_token, event.Signature, this_api_url, twilio_params ) ) {
console.log( 'Twilio auth failure.' );
return callback( 'Twilio auth failure.' );
} else {
console.log( 'Twilio auth success.' );
}

twiml = new twilio.TwimlResponse();
// twiml.message( 'Uncomment this line to send this as a text message response back to the sender.' );
return callback( null, twiml.toString() ); // Success.
};

Method Response

Set the Method Response 200 HTTP Status Response Model to a Content type of application/xml instead of the default application/json.

Integration Response

You need to set the 200 response Body Mapping Template to directly output your XML directly. Set a Body Mapping Template of Content-type application/xml and for the template content use the following. Since we are using the twimlResponse() method to build the XML we don’t need to modify the response in any way before passing it out.

#set($inputRoot = $input.path('$'))
$inputRoot

External resources

https://forums.aws.amazon.com/thread.jspa?messageID=725034
https://www.twilio.com/docs/api/security
http://twilio.github.io/twilio-node/#validateRequest
http://stackoverflow.com/questions/728360/most-elegant-way-to-clone-a-javascript-object
https://www.twilio.com/blog/2015/09/build-your-own-ivr-with-aws-lambda-amazon-api-gateway-and-twilio.html

Things I hate about the Android experience

In short, Android and Google have major issues with:
1) Backups. They suck and I do not trust my data with Android or Google’s backup systems. I write backup software for a living and know it’s a hard problem but Android’s backup system makes it sound like it backs up everything which is blatantly false.
2) Duplicate files. Both photos and music have tons of duplicating issues making usage a pain.
3) App permissions. On install you’re shown what the app wants and once you agree, the app has 24/7 access to all these.

Details:
* I had to wipe my phone earlier this year. I made sure to set my phone to backup all apps and their data which informed me that it would save ALL settings and auto-restore them on reset. After wiping I discovered that I had lost most of my app data. Apparently this feature is opt-in for developers so most apps will NOT be backed up. I lost tons of data on my apps. I also had to set everything back up on the phone manually after restore.

* I have had my phone set to sync all of my photos to Google since I got it. A few days ago I discovered that since Google separated their photos out of Google+, my phone was no longer backing up photos. I was shocked that Google would stop backing up my photos and not even inform me that I needed to take action to get them backing up again. My photo backups has stopped several months ago. I had to download the new “Google Photos” app since my phone’s built-in photo backup feature no longer functioned. I thought this has solved the problem but…

* I now have duplicates of every photo on my device. When I look in the Photos app I now see a copy of my local photo stored on my SD card as well as the photo backed up to Google Photos. Scrolling through my photos I’m now stuck with two copies of everything and I cannot find a way to fix this.

* Using the official “Google Play Music Manager” app on OS X to keep my iTunes music synced into Google Play Music has resulted in dozens of song duplicates in all my playlists on my phone. I have cleaned these duplicates out once but now I have even more duplicates. This makes playlists useless.

* The permission system on Android is awful. Apps demand all their permissions upfront and once you grant them they have them for the time they are installed. On ios you are prompted before a lot of usage unless you grant permanent access. The permission system on Android is also not very granular so even the simplest apps are requesting wide-open access to the entire phone just to do so simple things. This is a security disaster. I know Google is working on solving this but it should have been solved long ago.

There have been many more hiccups along the way but these are some of the big ones.

“invalid ELF header” running a NodeJS script on Raspberry Pi

Problem:

You see something like the following when trying to run your NodeJS script:

Error: [...]/node_modules/epoll/build/Release/epoll.node: invalid ELF header

Possible cause #1:

You’re using a Raspberry Pi (in my case 2 B) with Raspbian with the built-in Node v0.10.29 which is missing a UTF8 patch. See my other post HERE for the solution if you have the following error (you’ll just need to upgrade to v0.12.x or newer):

../node_modules/nan/nan.h:328:47: error: 'REPLACE_INVALID_UTF8' is not a member of 'v8::String'
static const unsigned kReplaceInvalidUtf8 = v8::String::REPLACE_INVALID_UTF8;

Possible cause #2:

You copied your node_modules directory over from your computer to the Raspberry Pi. npm needs to compile some node modules specially for the Raspberry Pi, so simply copying over the modules won’t always work.

Solution for cause #2:

Install your node modules on the Pi itself via npm, not just copy the node_modules directory over. This is because things need to be compiled slightly differently when running on the Pi. In my case epoll is a submodule of the repo “onoff”, so in my case I’d do the following on the Raspberry Pi where I want it installed:

npm install onoff

Note: If you don’t have npm installed on your Raspberry Pi yet, do the following:

sudo apt-get install nodejs npm

‘REPLACE_INVALID_UTF8’ is not a member of ‘v8::String’ installing NodeJS packages on Raspbian (Debian) on Raspberry Pi 2 B

Problem:

The current distribution of node (v0.10.29) packaged with the Debian distro that comes with the Raspberry Pi 2 B is missing a patch related to UTF8. Because of this you may encounter packages which are unable to compile. You’ll see the following before various errors and eventual failure of installing a package via npm. In my case this failed when trying to install the “onoff” package which relied on “epoll”.

The Error:

../node_modules/nan/nan.h:328:47: error: 'REPLACE_INVALID_UTF8' is not a member of 'v8::String'
static const unsigned kReplaceInvalidUtf8 = v8::String::REPLACE_INVALID_UTF8;

Solution:

Install a newer version of Node, such as v0.12.x:

curl -sL https://deb.nodesource.com/setup_0.12 | sudo -E bash -
sudo apt-get install -y nodejs

Verify this worked with:

node --version

That’s it! You should now be ready to resume installing your packages. Enjoy.