Skip to content

Blog

Cheap serverless containers using API Gateway

Sometimes I need to run a long-lived app. In those cases I reach for AWS ECS Fargate instead of AWS Lambda. You can run a container on Fargate for as little as $9/month, or $2.70/month if you're happy to roll the dice with Fargate Spot (I usually do!)

If you have a web app, you almost certainly use a load balancer in front of your containers. And this is where the cost goes from "fun side project" to "oh, I'm not sure I'm willing to spend that much money on this." The load balancer by itself is at least $16.40/month - you could run six containers for that price!

openrolesanywhere - an IAM Roles Anywhere client

Update: AWS now has an open source implementation of a Roles Anywhere credential_process provider - and it even supports PKCS#11. I'll keep the following project online for historical purposes, but there's not much need for it.

I just published a proof-of-concept CLI tool named openrolesanywhere. It lets you assume a role in AWS using IAM Roles Anywhere and a private key stored in your SSH agent - rather than on-disk as required by the official client. It implements AWS4-X509-RSA-SHA256, AWS4-X509-ECDSA-SHA256 via a forked copy of the SigV4 signer in the AWS SDK for Go.

AWS VPC data exfiltration using CodeBuild

UPDATE: This issue has been fixed. See the end of the article for timeline and details.

In September 2020, I published a guest blog post on Ian Mckay's blog. The tl;dr is that "escaping" a privileged container running in an Amazon-managed AWS account isn't a security concern for Amazon, thanks to defence-in-depth on both an EC2 and IAM level. Here's a few paragraphs I left out of that blog post at the time.

AWS GWLB: Deep Packet Manipulation

AWS introduced Gateway Load Balancers back in November 2020. A reasonably accurate tl;dr would be that they are like having highly available, auto-scaling NAT instances. That intro blog post will explain them better than I can.

The blog post mentions a dozen AWS partners that implement various flavours of firewalls, deep packet inspection, DDoS protection, etc. It's all useful and serves a genuine need, but not very exciting from a developer perspective.

CloudWatch EMF in Honeycomb

The CloudWatch embedded metric format (EMF) is a convenient way to publish metrics to CloudWatch from your apps running in AWS - especially Lambda functions. You just need to emit logs in the right JSON format and CloudWatch Logs will automatically publish them to CloudWatch Metrics.

This is okay for basic use cases where you have zero or few dimensions on your metrics and those dimensions are low cardinality. If they're high cardinality, Amazon advises that you instead do not create dimensions and instead treat the metrics as logs and query them using CloudWatch Log Insights. So you have to make a trade-off between something useful, something affordable and something easy to use.

No need for AWS IAM users

It's long been considered "best practice" to avoid having IAM users in AWS. Where possible IAM roles are preferable as role session credentials are short-lived. As far as I can tell, the only justification for AWS IAM users that I hear nowadays is for usage on non-interactive systems outside of AWS (so AWS SSO won't work), e.g. a Raspberry Pi in your closet.

I created a proof-of-concept project cloudkey to show that even that scenario can avoid IAM users. It uses the little-known iot:AssumeRoleWithCertificate functionality to avoid that.

Two approaches to cross-account EventBridge

Since November 2020, EventBridge has supported two methods of creating cross-account event subscriptions. Both require a level of indirection in the form of an "intermediary" bus in the subscriber's account. Here are examples of how they work. We will refer to the account whose bus has interesting events as account B. We will refer to the subscriber account as account S.

AWS SigV4 caching

Say you find yourself doing silly things with AWS APIs on a lazy Sunday afternoon. And you are getting an inexplicable error when using perfectly valid credentials.