Categorias
PHP Programação

Lambda extension to cache SSM and Secrets Values for PHP Lambda on CDK

Introduction

Managing secrets securely in AWS Lambda functions is crucial for maintaining the integrity and confidentiality of your applications. AWS provides services like AWS Secrets Manager and AWS Systems Manager Parameter Store to manage secrets. However, frequent retrieval of secrets can introduce latency and additional costs. To optimize this, we can cache secrets using a Lambda Extension.

In this article, we will demonstrate how to use a pre-existing Lambda Extension to cache secrets for a PHP Lambda function using the Bref layer and AWS CDK for deployment.

On a high-level, these are the components involved:

Lambda Execution Components

Using the AWS Parameter and Secrets Lambda extension to cache parameters and secrets

The new AWS Parameters and Secrets Lambda extension provides a managed parameters and secrets cache for Lambda functions. The extension is distributed as a Lambda layer that provides an in-memory cache for parameters and secrets. It allows functions to persist values through the Lambda execution lifecycle, and provides a configurable time-to-live (TTL) setting.

When you request a parameter or secret in your Lambda function code, the extension retrieves the data from the local in-memory cache, if it is available. If the data is not in the cache or it is stale, the extension fetches the requested parameter or secret from the respective service. This helps to reduce external API calls, which can improve application performance and reduce cost.

Prerequisites

  • AWS Account
  • AWS CLI configured
  • AWS CDK installed
  • PHP installed
  • Composer installed

If you have Docker, all requirements are being installed by it.

Repository Overview

The code for this project is available in the following GitHub repository: rafaelbernard/serverless-patterns. The relevant files are located in the lambda-extension-ssm-secrets-cdk-php folder.

Step-by-Step Guide

1. Cloning the Repository

First, clone the repository and navigate to the relevant directory:

git clone --branch rafaelbernard-feature-lambda-extension-ssm-secrets-cdk-php https://github.com/rafaelbernard/serverless-patterns.git
cd serverless-patterns/lambda-extension-ssm-secrets-cdk-php

2. Project Structure

The project structure is as follows:

.
├── assets
│   └── lambda
│       └── lambda.php
├── bin
│   └── cdk.ts
├── cdk
│   └── cdk-stack.ts
├── cdk.json
├── docker-compose.yml
├── Dockerfile
├── example-pattern.json
├── Makefile
├── package.json
├── package-lock.json
├── php
│   ├── composer.json
│   ├── composer.lock
│   └── handlers
│       └── lambda.php
├── README.md
├── run-docker.sh
└── tsconfig.json

3. Setting Up the Lambda Function

The main logic for fetching and caching secrets is in php/handlers/lambda.php:

<?php

use Bref\Context\Context;
use Bref\Event\Http\HttpResponse;
use GuzzleHttp\Client;
use Symfony\Component\HttpFoundation\JsonResponse;

// Responsibilities are simplified into one file for demonstration purposes
// We would have have those methods in a Service class

function getParam(string $parameterPath): string
{
    // Set `withDecryption=true if you also want to retrieve SecureString SSMs
    $url = "http://localhost:2773/systemsmanager/parameters/get?name={$parameterPath}&withDecryption=true";

    try {
        $client = new Client();

        $response = $client->get($url, [
            'headers' => [
                'X-Aws-Parameters-Secrets-Token' => getenv('AWS_SESSION_TOKEN'),
            ]
        ]);

        $data = json_decode($response->getBody());
        return $data->Parameter->Value;
    } catch (\Exception $e) {
        error_log('Error getting parameter => ' . print_r($e, true));
    }
}

function getSecret(string $secretName): stdClass
{
    $url = "http://localhost:2773/secretsmanager/get?secretId={$secretName}";

    try {
        $client = new Client();

        $response = $client->get($url, [
            'headers' => [
                'X-Aws-Parameters-Secrets-Token' => getenv('AWS_SESSION_TOKEN'),
            ]
        ]);

        $data = json_decode($response->getBody());
        return json_decode($data->SecretString);
    } catch (\Exception $e) {
        error_log('Error getting secretsmanager => ' . print_r($e, true));
    }
}

return function ($request, Context $context) {
    $secret = getSecret(getenv('THE_SECRET_NAME'));
    $response = new JsonResponse([
        'status' => 'OK',
        getenv('THE_SSM_PARAM_PATH') => getParam(getenv('THE_SSM_PARAM_PATH')),
        getenv('THE_SECRET_NAME') => [
            'password' => $secret->password,
            'username' => $secret->username,
        ],
    ]);

    return (new HttpResponse($response->getContent(), $response->headers->all()))->toApiGatewayFormatV2();
};

4. Setting Up AWS CDK Stack

The AWS CDK stack is defined in cdk/cdk-stack.ts:

import { CfnOutput, CfnParameter, Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { join } from "path";
import { packagePhpCode, PhpFunction } from "@bref.sh/constructs";
import { FunctionUrlAuthType, LayerVersion, Runtime } from "aws-cdk-lib/aws-lambda";
import { StringParameter } from "aws-cdk-lib/aws-ssm";
import { Policy, PolicyStatement } from 'aws-cdk-lib/aws-iam';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';

export class CdkStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const stackPrefix = id;

    // May be set as parameter new CfnParameter(this, 'parameterStoreExtensionArn', { type: 'String' });
    const parameterStoreExtensionArn = 'arn:aws:lambda:us-east-1:177933569100:layer:AWS-Parameters-and-Secrets-Lambda-Extension:11';
    const parameterStoreExtension = new CfnParameter(this, 'parameterStoreExtensionArn', { type: 'String', default: parameterStoreExtensionArn });

    const paramTheSsmParam = new StringParameter(this, `${stackPrefix}-TheSsmParam`, {
      parameterName: `/${stackPrefix.toLowerCase()}/ssm/param`,
      stringValue: 'the-value-here',
    });

    // CDK cannot create SecureString
    // You would create the SecureString out of CDK and use the param name here
    // const paramAnSsmSecureStringParam = StringParameter.fromSecureStringParameterAttributes(this, `${stackPrefix}-AnSsmSecureStringParam`, {
    //   parameterName: `/${stackPrefix.toLowerCase()}/ssm/secure-string/params`,
    // });

    const templatedSecret = new Secret(this, 'TemplatedSecret', {
      generateSecretString: {
        secretStringTemplate: JSON.stringify({ username: 'postgres' }),
        generateStringKey: 'password',
        excludeCharacters: '/@"',
      },
    });

    // The param path that will be used to retrieve value by the lambda
    const lambdaEnvironment = {
      THE_SSM_PARAM_PATH: paramTheSsmParam.parameterName,
      THE_SECRET_NAME: templatedSecret.secretName,
      // If you create the SecureString
      // THE_SECURE_SSMPARAM_PATH: paramAnSsmSecureStringParam.parameterName,
    };

    const functionName = `${id}-lambda`;
    const theLambda = new PhpFunction(this, `${stackPrefix}${functionName}`, {
      handler: 'lambda.php',
      phpVersion: '8.3',
      runtime: Runtime.PROVIDED_AL2,
      code: packagePhpCode(join(__dirname, `../assets/lambda`)),
      functionName,
      environment: lambdaEnvironment,
    });

    // Add extension layer
    theLambda.addLayers(
      LayerVersion.fromLayerVersionArn(this, 'ParameterStoreExtension', parameterStoreExtension.valueAsString)
    );

    // Set additional permissions for parameter store
    theLambda.role?.attachInlinePolicy(
      new Policy(this, 'additionalPermissionsForParameterStore', {
        statements: [
          new PolicyStatement({
            actions: ['ssm:GetParameter'],
            resources: [
              paramTheSsmParam.parameterArn,
              // If you create the SecureString
              // paramAnSsmSecureStringParam.parameterArn,
            ],
          }),
        ],
      }),
    )

    templatedSecret.grantRead(theLambda);

    const fnUrl = theLambda.addFunctionUrl({ authType: FunctionUrlAuthType.NONE });

    new CfnOutput(this, 'LambdaUrl', { value: fnUrl.url });
  }
}

5. Deploying with AWS CDK

Make sure you have already AWS variables set and run below command to install required dependancies:

# Using docker -- check run-docker.sh
make up

or

# Using local
npm ci
cd php && composer install --no-scripts && cd -

After that, you will have all dependencies installed. Deploy it executing:

# Using docker
make deploy

or

# Using local
npm run deploy

6. Testing the Lambda Function

The CDK output will have the Lambda function URL, which you can use to test and retrieve the values:

Outputs:
LambdaExtensionSsmSecretsCdkPhpStack.LambdaUrl = https://keamdws766oqzr6dbiindaix3a0fdojb.lambda-url.us-east-1.on.aws/

You should see the secret value and parameter value returned by the Lambda function. Subsequent invocations should retrieve the values from the cache, reducing latency and cost.

{
  "status": "OK",
  "/lambdaextensionssmsecretscdkphpstack/ssm/param": "the-value-here",
  "TemplatedSecret3D98B577-4jOWSbUMCHmF": {
    "password": "!o9GpBzpa>dYdo.Gx3J2!<zd(s-Fg;ev",
    "username": "postgres"
  }
}

Performance benefits

A similar example application written in Python performed three tests, reducing API calls ~98%. I am quoting their findings, as the benefits are the same for this PHP Lambda:

To evaluate the performance benefits of the Lambda extension cache, three tests were run using the open source tool Artillery to load test the Lambda function.

config:
 target: "https://lambda.us-east-1.amazonaws.com"
phases:
  -
duration: 60
arrivalRate: 10
rampTo: 40
Test 1: The extension cache is disabled by setting the TTL environment variable to 0. This results in 1650 GetParameter API calls to Parameter Store over 60 seconds.

Test 2: The extension cache is enabled with a TTL of 1 second. This results in 106 GetParameter API calls over 60 seconds.
Test 3: The extension is enabled with a TTL value of 300 seconds. This results in only 18 GetParameter API calls over 60 seconds.

In test 3, the TTL value is longer than the test duration. The 18 GetParameter calls correspond to the number of Lambda execution environments created by Lambda to run requests in parallel. Each execution environment has its own in-memory cache and so each one needs to make the GetParameter API call.

In this test, using the extension has reduced API calls by ~98%. Reduced API calls results in reduced function execution time, and therefore reduced cost.

7. Clean up

To delete the stack, run:

make bash
npm run destroy

Conclusion

In this article, we demonstrated how to use a pre-existing Lambda Extension to cache secrets for a PHP Lambda function using the Bref layer and AWS CDK for deployment. By caching secrets, we can improve the performance and reduce the cost of our serverless applications. The approach detailed here can be adapted to various use cases, enhancing the efficiency of your AWS Lambda functions.

For more information on the Parameter Store, Secrets Manager, and Lambda extensions, refer to:

For more serverless learning resources, visit Serverless Land.

Categorias
Tropeçando

Tropeçando 98

Operating Lambda: Debugging code – Part 1
Operating Lambda: Debugging configurations – Part 2
Operating Lambda: Debugging configurations – Part 3

In the Operating Lambda series, I cover important topics for developers, architects, and systems administrators who are managing AWS Lambda-based applications. This three-part series discusses core debugging concepts for Lambda-based applications.

Those pesky pull request reviews

They’re everywhere. In Slack: “hey, can I get a review on this?” In email: “Your review is requested!” In JIRA: “8 user stories In-Progress” (but code-complete). In your repository: 5 open pull requests. They’re slowing your delivery. They’re interrupting your developers.

How can we get people to review pull requests faster??

Operating Lambda: Using CloudWatch Logs Insights

In the Operating Lambda series, I cover important topics for developers, architects, and systems administrators who are managing AWS Lambda-based applications. This three-part series discusses monitoring and observability for Lambda-based applications and covers:

  • Using Amazon CloudWatch, CloudWatch Logs Insights, and AWS X-Ray to apply monitoring
    across services.
  • How existing monitoring concepts apply to Lambda-based applications.
  • Troubleshooting application issues in an example walkthrough.
    This post explains how to use CloudWatch Logs Insights in your serverless applications.

CDK Lambda Deployment takes about a minute - how about sub second Function Code Deployment?

Creation of Lambda infrastructure with the CDK is really powerful. Updating the Function code is really slow. Here is a fix for that to get to a sub-second Lambda function deployment time.

Best practices for developing cloud applications with AWS CDK

In this post, we discuss strategies for organizing the development of complex cloud applications with large teams, using the AWS Cloud Development Kit (AWS CDK) as a central technology. AWS CDK allows developers and administrators to define their cloud applications using a familiar programming language, such as TypeScript, Python, Java, or C#. Applications are organized into stages, stacks, and constructs, which allows for modular design techniques in both runtime logic (such as AWS Lambda code or containerized services) and infrastructure components such as Amazon Simple Storage Service (Amazon S3) buckets, Amazon Relational Database Service (Amazon RDS) databases, and network infrastructure.