My AWS CLI notebook

Some examples for AWS


Get a list of EC2 instances where we use a filter to query (tag with value PROD), only want the output of instanceId, PublicDnsName and the Name and present it in a table:

aws ec2 describe-instances --filters Name=tag-value,Values=PROD --query "Reservations[*].Instances[*].{Instance:InstanceId,PublicDnsName:PublicDnsName,Name:Tags[?Key=='Name']|[0].Value}" --output table

Delete all unused security groups (github source here):

#!/usr/bin/env bash

# lists all unused AWS security groups.
# a group is considered unused if it's not attached to any network interface.
# requires aws-cli and jq.

# all groups
aws ec2 describe-security-groups \
  | jq --raw-output '.SecurityGroups[] | [.GroupName, .GroupId] | @tsv' \
  | sort > /tmp/sg.all

# groups in use
aws ec2 describe-network-interfaces \
  | jq --raw-output '.NetworkInterfaces[].Groups[] | [.GroupName, .GroupId] | @tsv' \
  | sort \
  | uniq > /tmp/

diff /tmp/sg.all /tmp/ |grep "<" |cut -d ' ' -f2-3

Cert Manager
Get the public certificate:

aws acm get-certificate --certificate-arn arn:aws:acm:eu-central-1:XXXX:certificate/YYYYYYY &gt; output.json


Create a user and skip the force password change flow

aws cognito-idp admin-set-user-password --user-pool-id XXXXX --username YYYYY --password ZZZZZ --permanent
Leave a comment

Posted by on 09-10-2019 in AWS


Tags: ,

How to combine Azure AD SSO with AWS programmatic access ?

Since we both use Azure AD (Office365 / LDAP) for the companies user management and AWS for our hosting we already enabled federation between these cloud providers. This works perfectly by assuming an IAM Role through the Azure AD credentials and have console access through single-sing-on.

However to use the AWS CLI a user would need programmatic access and it’s access and secret keys. Luckily for us there is an excellent open-source project for that called aws-azure-login. And since it’s on npm very easy to run on Mac, Linux and Windows.

What do we need ?

  • install the package as mentioned: npm install -g aws-azure-login (or not global if you fancy that)
  • you will need your Azure Tenant ID which the Azure administrator can supply ( Azure AD -> Properties -> Directory ID )
  • and the Azure Application ID and again the Azure administrator ( Enterprise Applications -> Amazon Web Services -> Properties -> Application ID )
  • you might want to know the AWS IAM Role max session duration to prevent some trial on error which your AWS administrator can supply (default is 1 hour)

First we will create an Azure profile:

jvzoggel$ aws-azure-login --configure --profile rubix
Configuring profile 'rubix'
? Azure Tenant ID: some-guid-you-will-need-part-1
? Azure App ID URI: some-guid-you-will-need-part-2
? Default Username:
? Default Role ARN (if multiple):
? Default Session Duration Hours (up to 12): 2

The max session duration on the AWS IAM Role is the last parameter, if you enter a to high value you will receive an error but only after the next step login. So ask first, or trial on error.

Then we actually login on Azure:

jvzoggel$ aws-azure-login --profile rubix --no-prompt
Logging in with profile 'rubix'...
? Password: [hidden]
Please type in the code displayed on your authenticator app from your device
? Verification Code: 424242
Assuming role arn:aws:iam::42:role/rbx-sso-myrole

Since we configured MFA on Azure AD there is also the Verification Code besides the password to provide. When we succeed the aws-azure-login actually generated a profile in the AWS credentials configuration which holds the access and secret key. So let’s check.

Connect to AWS

jvzoggel$ cat /Users/jvzoggel/.aws/credentials


jvzoggel$ export AWS_PROFILE=rubix
jvzoggel$ aws s3api list-buckets
    "Buckets": [
            "Name": "awsamplify-20181220092431-deployment",


Leave a comment

Posted by on 01-02-2019 in Uncategorized


Tags: , , , ,

Reminder: Example password complexity check in typescript

Since I mostly blog/commit to remember for my own future usage, another handy codesnip which I probably will need again. Requirement from our IAM provider for is user credentials was a certain password complexity of at least 8 characters (easy) and the combination of at least 3 out of 4 character sets (lowercase, uppercase, numbers and weird characters aka non-alphanumeric. The last criteria could be done with a blunt knife but we tried to be gentle.

    // Check password complexity
    const hasUpperCase = /[A-Z]/.test(password);
    const hasLowerCase = /[a-z]/.test(password);
    const hasNumbers = /\d/.test(password);
    const hasNonalphas = /\W/.test(password);
    const totalPassword = [ hasUpperCase, hasLowerCase, hasNumbers, hasNonalphas ].filter(v =&amp;amp;gt; v).length;
    if (password.length < 8 || totalPassword < 3) {
        console.error(`password length:${password.length} charsets:${totalPassword}`);
        // fancy error handling stuff

The regex for frontend if usable:



Leave a comment

Posted by on 31-01-2019 in Uncategorized


Tags: , , , ,

How to use AWS Simple Email Service (SES) from TypeScript on NodeJS example

In our application flows we use AWS Simple Email Service to send emails to our users. Since the documentation and examples of AWS SES are not that clear, it might take some trial and error to figure out which of the parameters are mandatory and, more important, obsolete. The AWS examples here crash if you actually configure the parameters with empty strings or null like mentioned there.

This TypeScript code below initiates a AWS SES connection and uses an earlied AWS CloudFront generated Email template (see example below somewhere). I found it quite surprising that you can actually refer to the template by name and not ARN only.

import * as AWS from 'aws-sdk';
import * as https from 'https';

const ses = new AWS.SES({
    httpOptions: {
        agent: new https.Agent({
            keepAlive: true

 * Send email through AWS SES Templates
export async function sendMail(email: string, name: string): Promise(String) {

    try {
        // Create SES sendTemplatedEmail templateData content
        const templatedata = {
            parameter_name: name
        // console.debug(`sendMail templatedata: ${JSON.stringify(templatedata)}`);

        // Create SES sendTemplatedEmail full message
        const params = {
            Destination: {
                ToAddresses: [ email ]
            Source: '',
            Template: 'myFanceTerra10EmailTemplate',
            TemplateData: JSON.stringify(templatedata)
        // console.debug(`sendMail param: ${JSON.stringify(params)}`);

        const sesResponse = await ses.sendTemplatedEmail(params).promise();
        console.log(`sendMail requestId: ${sesResponse.$response.requestId} and messageId: ${sesResponse.MessageId}`);
        return 'OK';
    } catch (e) {
        console.error(`sendMail unexpected: ${e.message}`);
        return 'something fancy error handling';

Here is an example AWS CloudFormation resource for the AWS SES email template. You can use the always handy sub function to prevent any complex character escaping or unreadable 1-line HTML. I love it for EC2 UserData and for stuff like this:

    Type: AWS::SES::Template
        TemplateName: myFanceTerra10EmailTemplate
        SubjectPart: My Subject
#       TextPart: "Nobody uses this anymore right ???"
          Fn::Sub: |
            <img src="">
            <h1>Sir/Madam {{parameter_name}},</h1>
            <p>Ho ya doin ?</p>
            <strong>the T10 crew</strong>

In case you use the Serverless Framework (you should) for Serverless deployments the following code is necessary in your Serverless.yaml. This allows your Lambda function to use the Email template on runtime. In our case the domain is hosted on AWS Route53 as well which saves you some problems.

- Effect: Allow
  - ses:SendTemplatedEmail
  - "arn:aws:ses:eu-west-1:*:identity/"

Hope it helps!


  • Link to the original article due to RSS feeds @
  • Sending Email Using Amazon SES
1 Comment

Posted by on 26-11-2018 in Uncategorized


Tags: , , , , , ,

Using AWS Key Management (KMS) to encrypt and decrypt in AWS Lambda (NodeJS)

AWS Key Management (KMS) is a fully managed service that makes it easy to create and control encryption keys on AWS which can then be utilised to encrypt and decrypt data in a safe manner. The service leverages Hardware Security Modules (HSM) under the hood which in return guarantees security and integrity of the generated keys.

You can enable AWS KMS to store your data-at-rest on many AWS storage solutions (like DynamoDB and EBS). However in our Lambda functions we would like to encrypt (and decrypt) certain values on runtime. So we needed some code to encrypt and decrypt. Took me some while to figure it out, so here it is.

Here is the function to encrypt any string against the AWS KMS Key. You can create a KMS Key through AWS IAM in the console or (much better) AWS CloudFormation.

const kmsClient = new AWS.KMS({region: 'eu-west-1'});

 * Encrypt
async function encryptString(text: string): Promise<string> {

    const paramsEncrypt = {
        KeyId: 'arn:aws:kms:eu-west-1:........',
        Plaintext: new Buffer(text)

    const encryptResult = await kmsClient.encrypt(paramsEncrypt).promise();
    // The encrypted plaintext. When you use the HTTP API or the AWS CLI, the value is Base64-encoded. Otherwise, it is not encoded.
    if (Buffer.isBuffer(encryptResult.CiphertextBlob)) {
        return Buffer.from(encryptResult.CiphertextBlob).toString('base64');
    } else {
        throw new Error('Mayday Mayday');

And we can use the result (an encrypted string) to store it in DynamoDB, Aurora or send it wherever we want. So in the end we would like to decrypt it as well. So …

 * Decrypt
async function decryptEncodedstring(encoded: string): Promise<string> {

    const paramsDecrypt: AWS.KMS.DecryptRequest = {
        CiphertextBlob: Buffer.from(encoded, 'base64')

    const decryptResult = await kmsClient.decrypt(paramsDecrypt).promise();
    if (Buffer.isBuffer(decryptResult.Plaintext)) {
        return Buffer.from(decryptResult.Plaintext).toString();
    } else {
        throw new Error('We have a problem');

Hope it helps!


  • Example code including test function available in my Github as well


Leave a comment

Posted by on 05-11-2018 in AWS


Tags: , , , , ,

How to setup unit testing for AWS Lambda serverless functions (on NodeJS) ?

We use AWS Lambda serverless functions combined with TypeScript and NodeJS which results in an extreme powerful developer toolset. Due to the fact that functions contain isolated logic they are ideal for automated unit testing in CI/CD pipelines. So eventually looking at our options we decided to use the features of mocha, chai and nock combined. This resulted in a very easy and powerful solution for unit testing.

I’m sharing this after a chat at a meetup where the use of EKS instead of Lambda even for really simple functions was advocated due to the fact that serverless was hard to isolate (run local) and was hard to setup any unit testing. I beg to differ because:

So let’s go …

Our example function

is a simple function for retrieving a single record from a AWS DynamoDB table

// Handler for serverless framework
export async function handler(event: APIGatewayProxyEvent, _context: Context, callback: Callback) {
    try {
        callback(undefined, await getWerkgever(event));
    } catch (err) {

// Main logic
export async function getRecord(event: APIGatewayProxyEvent) { 
  const id = '1'; // pointless, but good enough for this example
  const queryParams = { TableName: process.env.dynamotable, Key: { id } }; 
  const result = await documentClient.get(queryParams).promise(); 
  if (result.Item) { 
    return { statusCode: 200, headers, body: JSON.stringify(result.Item) }; 
  } else {
    return { statusCode: 404, headers, body: undefined };

So what happens here ?

  • We deliberately split the logic between handler (for the serverless framework) and the main logic
  • We need to export the main logic for use in our unit tests (and local development)

Using Mocha & Nock

Since we are running node we can use both mochaJS and nock for our unit testing. Setting up the specification file (.spec) for our simple function we first run the test with.

import { APIGatewayProxyEvent } from 'aws-lambda';
import { expect } from 'chai';
import * as nock from 'nock';
import { getRecord } from './getRecord';

process.env.dynamotable = 'myTable';

describe('getRecord', () => {

    it('UT001 - getRecord with valid response', async() => {
        const event: APIGatewayProxyEvent = {
            body: '',
            headers: {},
            httpMethod: 'GET',
            isBase64Encoded: false,
            path: '',
            pathParameters: {},
            queryStringParameters: undefined,
            stageVariables: {},
            requestContext: {},
            resource: '' };

        const response = await getRecord(event);


So what happened ?

  • We set the environment variables (like the DynamoDB table) which normally is done by AWS Lambda
  • We configure nock.recorder for auditing the upcoming execution
  • We define a dummy APIGatewayProxyEvent which has some mandatory elements which we leave mostly empty or undefined
  • We define the call to our AWS Lambda function and since we isolated the main logic we can call this directly
  • If your AWS profile used on your dev machine has enough IAM grants the code can execute against AWS DynamoDB (we use a special user for this to keep it clean)

By running mocha with the Nock recorder we can see the actual callout to AWS DynamoDB from our developer machine

<-- cut here -->

nock('', {"encodedQueryParams":true})
.get('/', {"TableName":"myTable","Key":{"id":{"S":"1"}}})
.reply(200,{Item: {name: {S: 'myName'}, id: {S: '1'}});
......... (much stuff)

So with nock we actually recorded the https call to DynamoDB. Which now, we can easily use with nock to mock the response during unit testing. So next change the code in our spec file with the info from nock recorder:

// nock.recorder.rec();
    .get('/' )
    .reply(200,{Item: {name: {S: 'myName'}, id: {S: '1'}});

So what happened ?

  • Disabled the recorder, we don’t need it anymore
  • Setup nock to catch the HTTPS GET call to the dynamoDB endpoint
  • Configured nock to reply with a 200 and the specified Item record (you can also use reply with files)

Using Chai

With this basis setup we can execute unit tests in our pipeline with Mocha where nock will handle the mocking of the endpoints. With some little Chai magic we can define expectations in our specification file to make sure all message logic of our function is done properly and the HTTP reply is as expected.

expect(response.headers).to.deep.include({ 'Content-Type': 'application/json' });
expect(JSON.parse(response.body)).to.deep.equal({ name: 'myName', id: '1'});

And there is more

With this it’s easy to catch all outbound HTTPS requests and mock different responses (0 records, multiple records, etc etc) for extensive unit testing. The possibilities are endless, so hope it helps …

Leave a comment

Posted by on 26-10-2018 in Uncategorized


Tags: , , , , , , , ,

How to determine your AWS Lambda@Edge regions and find your CloudWatch logs

You must review AWS CloudWatch log files in the correct region to see the log files created when CloudFront executed your Lambda function. I found this very usefull bash AWS CLI based bash command here which allows to determine a list of Regions where your Lambda@Edge function have received traffic so storing for future (personal) reference.

for region in $(aws --output text  ec2 describe-regions | cut -f 3) 
    for loggroup in $(aws --output text  logs describe-log-groups --log-group-name "/aws/lambda/us-east-1.$FUNCTION_NAME" --region $region --query 'logGroups[].logGroupName')
        echo $region $loggroup

You can just leave FUNCTION_NAME empty to get a list of all functions.


Leave a comment

Posted by on 25-10-2018 in Uncategorized


Tags: , , , , , ,