Categorias
PHP Programação

A bref AWS PHP story – Part 3

We are starting Part 3 of the Series "A bref AWS PHP history". You can check Part 1, where I presented the PHP language as a reliable and good alternative for Serverless applications and Part 2 where we see the usage of CDK features in favour of a faithful CI/CD.

Part 3 is to show the upgrade path to Bref 2 and to achieve more coverage of the AWS resources. We will use DynamoDB, a powerful database for serverless architectures.

Some of those topics seem straightforward to some people, but I would like to avoid guessing that this is known to the audience since I have experienced some PHP developers struggling to put all these together for the first time due to the paradigm change. It should be fun.

Table of contents:

  1. What else are we doing?
  2. Describing more AWS services - Adding a DynamoDB table
  3. Bref upgrade
  4. Testing CDK
  5. PHP and AWS Services
  6. Wrap-up

What else are we doing?

In this section, we'll explore additional functionalities and enhancements to our serverless application. Building upon the foundation laid in Part 2, we'll introduce new features and integrations to further extend the capabilities of our AWS PHP application.

The Part 2 uses the result of the Fibonacci of a provided integer or a random integer from 400 to 1000 (to get a good image and not to overflow integer). This integer is the number of pixels of an image from the bucket and an arbitrary request metadata we are creating. If the image does not exist, the lambda will fetch a random image from the web with that number of pixels, save it and generate the metadata.

The computing complexity is irrelevant because it could be very complex logic or very simple, and the topics we are discussing in this part of the series will use the same design.

The lambda will now search the metadata in a DynamoDB table, saving the metadata when it does not exist. DynamoDB is largely used in Lambda code.

Get the part-3 source-code on GitHub and the diff from part-2.

Describing more AWS services - Adding a DynamoDB Table

DynamoDB plays a crucial role in serverless architectures, offering scalable and high-performance NoSQL database capabilities. In this section, we'll delve into the process of integrating DynamoDB into our AWS CDK stack, expanding our application's data storage and retrieval capabilities.

DynamoDB is a fully managed NoSQL database service provided by AWS, offering seamless integration with other AWS services, automatic scaling, and built-in security features. Its scalability, low latency, and flexible data model make it well-suited for serverless architectures and applications with varying throughput requirements.

    const table = new Table(this, TableName, {
      partitionKey: { name: 'PK', type: AttributeType.STRING },
      sortKey: { name: 'SK', type: AttributeType.STRING },
      removalPolicy: RemovalPolicy.DESTROY,
      tableName: TableName,
    });

Following the same principles for creating other AWS resources, we utilize the AWS CDK to define a DynamoDB table within our stack. Let's dive into the key parameters of the Table constructor:

  • partitionKey: This parameter defines the primary key attribute for the DynamoDB table, used to distribute items across partitions for scalability. In our example, { name: 'PK', type: AttributeType.STRING } specifies a partition key named 'PK' with a string type. The naming convention ('PK') is arbitrary and can be tailored to suit your application's needs.
  • sortKey: For tables requiring a composite primary key (partition key and sort key), the sortKey parameter comes into play. Here, { name: 'SK', type: AttributeType.STRING } defines a sort key named 'SK' with a string type. Like the partition key, the name and type of the sort key can be customized based on your data model.
  • removalPolicy: This parameter determines the behaviour of the DynamoDB table when the CloudFormation stack is deleted. By setting RemovalPolicy.DESTROY, we specify that the table should be deleted (destroyed) along with the stack. Alternatively, you can opt for RemovalPolicy.RETAIN to preserve the table post-stack deletion, which may be useful for retaining data.

By decoupling configuration from implementation, we adhere to SOLID principles, ensuring cleaner and more robust code. This approach fosters flexibility, allowing our code to seamlessly adapt to changes, such as modifications to the table name while maintaining its functionality.

The implementation code is aware that the name will come from an environment variable and will work with that (yes, if you think that test will be easy to write, you are right):

    const lambdaEnvironment = {
      TableName,
      TableArn: table.tableArn,
      BucketName: brefBucket.bucketName,
    };

Bref Upgrade

Bref, the PHP runtime for AWS Lambda, continually evolves to provide developers with the latest features and optimizations. In this section, we'll discuss the upgrade to Bref 2.0 and explore how it enhances the deployment process and performance of our serverless PHP applications.

In this section, we're upgrading our usage of Bref, a PHP runtime for AWS Lambda, to version 2.0. Bref simplifies the deployment of PHP applications to AWS Lambda, enabling us to run PHP code serverlessly.

The upgrade involves modifying our AWS CDK code to utilize the new features and improvements introduced in Bref 2.0. One notable improvement is the automatic selection of the latest layer of the PHP version, which simplifies the deployment process and ensures that our Lambda functions run on the most up-to-date PHP environment available.

  const getLambda = new PhpFunction(this, ${stackPrefix}${functionName}, {
    handler: 'get.php',
    phpVersion: '8.3',
    runtime: Runtime.PROVIDED_AL2,
    code: packagePhpCode(join(__dirname, ../assets/get), {
      exclude: ['test', 'tests'],
    }),
    functionName,
    environment: lambdaEnvironment,
  });
  • `PhpFunction` Constructor: We're using the `PhpFunction` constructor provided by Bref to define our Lambda function. This constructor allows us to specify parameters such as the handler file, PHP version, runtime, code location, function name, and environment variables.
  • `handler`: Specifies the entry point file for our Lambda function, where the execution starts.
  • `phpVersion`: Defines the PHP version to be used by the Lambda function. In this case, we're using PHP version 8.3.
  • `runtime`: Indicates the Lambda runtime environment. Here, `Runtime.PROVIDED_AL2` signifies the use of the Amazon Linux 2 operating system.
  • `code`: Specifies the location of the PHP code to be deployed to Lambda.
  • `functionName`: Sets the name of the Lambda function.
  • `environment`: Allows us to define environment variables required by the Lambda function, such as database connection strings or configuration settings.

By upgrading to Bref 2.0 and configuring our Lambda function accordingly, we ensure compatibility with the latest enhancements and optimizations provided by Bref, thereby improving the performance and reliability of our serverless PHP applications on AWS Lambda.

Testing CDK

Ensuring the correctness and reliability of our AWS CDK infrastructure is crucial for maintaining a robust serverless architecture. In this section, we'll delve into testing our CDK resources, focusing on the DynamoDB table we added in the previous section.

As described earlier, we utilized the AWS CDK to provision a DynamoDB table within our serverless stack. Now, let's ensure that the table is configured correctly and behaves as expected by writing tests using the CDK's testing framework.

First, let's revisit how we added the DynamoDB table:

const table = new Table(this, TableName, {
  partitionKey: { name: 'PK', type: AttributeType.STRING },
  sortKey: { name: 'SK', type: AttributeType.STRING },
  removalPolicy: RemovalPolicy.DESTROY,
  tableName: TableName,
});

In this code snippet, we define a DynamoDB table with specified attributes such as partition key, sort key, removal policy, and table name. Now, to ensure that this table is created with the correct configuration, we'll write tests using CDK's testing constructs.

Check the following thest:

test('Should have DynamoDB', () => {
  expectCDK(stack).to(
    haveResource(
      'AWS::DynamoDB::Table',
      {
        "DeletionPolicy": "Delete",
        "Properties": {
          "AttributeDefinitions": [
            {
              "AttributeName": "PK",
              "AttributeType": "S",
            },
            {
              "AttributeName": "SK",
              "AttributeType": "S",
            },
          ],
          "KeySchema": [
            {
              "AttributeName": "PK",
              "KeyType": "HASH",
            },
            {
              "AttributeName": "SK",
              "KeyType": "RANGE",
            },
          ],
          "ProvisionedThroughput": {
            "ReadCapacityUnits": 5,
            "WriteCapacityUnits": 5,
          },
          "TableName": "BrefStory-table",
        },
        "Type": "AWS::DynamoDB::Table",
        "UpdateReplacePolicy": "Delete",
      },
      ResourcePart.CompleteDefinition,
    )
  );
});

This test ensures that the DynamoDB table is created with the correct attribute definitions, key schema, provisioned throughput, table name, and other properties specified during its creation. By writing such tests, we validate that our CDK infrastructure is provisioned accurately and functions as intended.

PHP and AWS Services

Leveraging PHP in a serverless environment opens up new possibilities for interacting with AWS services. In this section, we'll examine how PHP code seamlessly integrates with various AWS services, following best practices for maintaining clean and modular code architecture.

This is the part where we have fewer serverless needs impacting the code, as the PHP code will follow the same logic we might be using to communicate with AWS services on any other platform overall (there are always some specific use cases).

The reuse of the same existing logic is excellent. It leverages the decision to keep using PHP when moving that workload to Serverless, as the bulk of the knowledge and already proven code would remain as-is. We may escape the trap of classifying that PHP code as legacy as if it should be avoided, terminated or halted.

As a side note, a few external layers of our software architecture are touched if a good software architecture was applied before. Therefore, during the implementation of this architectural change, it should be quick to realise how beneficial and time-saving it is to have a well-architectured application with a balanced decision for patterns, principles, and designs to be applied, ultimately giving flexibility to the application and its features.

The handler is simplified now and should accommodate everything to a class in the direction of following SRP, a principle that we are bringing to the code during the code bites:

Applications, domains, infrastructure, etc

Our `PicsumPhotoService` is still orchestrating the business logic. The Single Responsibility Principle and Inversion of Control are applied. We are injecting the specialized services in the constructor:

// readonly class PicsumPhotoService
    public function __construct(
        private HttpClientInterface $httpClient,
        private ImageStorageService $storageService,
        private ImageRepository $repository,
    )
    {
    }

Each specialized service has all its dependencies injected in the constructor as well. We can see the factory instantiation:

    public static function createPicsumPhotoService(): PicsumPhotoService
    {
        return new PicsumPhotoService(
            HttpClient::create(),
            new S3ImageService(
                new S3Client(),
                getenv('BucketName'),
            ),
            new DynamoDbImageRepository(
                new DynamoDbClient(),
                getenv('TableName'),
            ),
        );
    }

The `ImageStorageService` will handle all image operations, connecting to the AWS Service when appropriate and observing business logic details. This is a slim interface:

interface ImageStorageService
{
    public function getImageFromBucket(int $imagePixels): ?array;

    public function saveImage(int $imagePixels, mixed $fetchedImage): void;

    public function createAndPutMetadata(int $imagePixels, array $metadata): PutObjectOutput;
}

Instead of `: PutObjectOutput`, usually we would return a domain object, to not couple the interface with implementation details of using S3 Services, but for simplicity, I did not create a domain object here. It would be preferable though.

The `ImageRepository` will handle all metadata operations. It will save into a repository and observe logic details as well. Following the same principles, this is a slim interface:

interface ImageRepository
{
    public function findImage(int $imagePixels): ImageMetadataItem;

    public function addImageMetadata(ImageMetadataItem $imageMetadataItem): PutItemOutput;
}

The `ImageMetadataItem` is a representation of one of the domain objects we have in our codebase.

readonly class ImageMetadataItem
{
    public function __construct(public int $imagePixels, public array $metadata)
    {
    }

    public function toDynamoDbItem(): array
    {
        return [
            'PK' => new AttributeValue(['S' => 'IMAGE']),
            'SK' => new AttributeValue(['S' => "PIXELS#{$this->imagePixels}"]),
            'pixels' => new AttributeValue(['N' => "{$this->imagePixels}"]),
            'metadata' => new AttributeValue(['S' => json_encode($this->metadata)]),
            ...ConvertToDynamoDb::item($this->metadata),
        ];
    }

    /**
     * @param array $item
     */
    public static function fromDynamoDb(array $item): static
    {
        return new static(
            (int) $item['pixels']->getN(),
            (array) json_decode($item['metadata']->getS()),
        );
    }
}

If you check the implementation details, it operates transparently with all the services, business logic and AWS Services without any high couple with them. There are two utility functions:

  • toDynamoDbItem: to transform the object into a valid DynamoDb Item to be added
  • fromDynamoDb: to perform the opposite operation, transforming a DynamoDb Item into a domain object

The scope of the operation is very clear and does not bring the domain into dependency on those services, as the domain object can be used independently. It does not block any other way of dealing with it, giving the usage with other types of services, such as different databases or APIs. This is very important to the maintainability of the application without sacrificing the ease of readiness as it keeps the context of the utilities in the right place.

If you check all PHP code carefully, Bref is such a great abstraction layer that, removing the code from the handler file, any other line of code can be used as a lambda or a web application interchangeably without changing any line of code. This is very powerful, as you can imagine how you can leverage and migrate some of the existing code to lambda by just creating a handler that will trigger your existing code, if the code is well structured.

Wrap-up

It would be simple like that. Check more details in the source code, install it and try it yourself. This project is ready to:

  • Extend lambda function using Bref
  • Upgrade to use Bref 2.0
  • Create a DynamoDB table
  • Test the stack Cloudformation code
  • Separate the PHP logic
  • Have PHP communicating with AWS Services

Links:

Categorias
Tropeçando

Tropeçando 112

Treezor: a serverless banking platform

This case study dives into how Treezor went serverless for their banking platform. From legacy code running on servers to a serverless monolith, and then event-driven microservices on AWS with Bref.

Treezor is a high available banking application running mostly in PHP.

Wait, is cloud bad?

Forrest Brazeal review 37signals (Basecamp) movement from the Cloud back to DataCenter, their use-case and some reasoning about the mentioned arguments for Data Center.

ECS Blue/Green deployment with CodeDeploy and Terraform

How to make Rector Contribute Your Pull Requests Every Day

Do you enjoy making code-reviews with hundreds of rules in your head and adding extra work to the pull-request author?

We don't, so we let Rector for us in active code review.

Docker for the late majority

This is a guide for people who would like a brief introduction to Docker and are too afraid to ask for one. I get it. Everyone around you already seems to know what they’re talking about. Looking ignorant is no fun.

10 Essential PHP.ini Tweaks for Improved Web Performance

If you're running a website or web application with PHP, you may have encountered issues with slow loading times, high memory usage, or other performance problems. Fortunately, there are several tweaks you can make to your PHP configuration file (php.ini) to optimize your scripts and improve your website's performance. In this article, I'll cover the top 10 most common changes you might need to make to your php.ini file for best performance.

Categorias
Solving Problems

Solving problems 1: ECS, Event Bridge Scheduler, PHP, migrations

I love Mondays and Business as Usual. Solving problems is a delightful day-to-day task. Maybe this is what working with software means in the end. Do not take me wrong, it opens the doors for greenfield projects and experimentation. While mastering the business I can experiment, change and rebuild.

The solving problems series is just a way to share small ideas, experiences and outcomes of solving daily problems as I go. I wonder if some tips or experiences shared can help you build better what you are working on right now.


During the last months, I have been migrating an important PHP service to ECS Fargate along with the runtime upgrade. The service is composed of a lot of parts and we have been architecting the migration so the operation causes no downtime to customers, even when they are over four different continents and many time zones.

One very important part of the service is already running in production for some months with success. We are preparing the next service.

For the migration plan, we deployed infrastructure ahead of starting moving traffic, planned to daily incremental traffic switch, like 5, 10, 25, 50, 75, and close monitoring. Also prepared a second plan to avoid rollback in case some performance issue arises. While monitoring we created backlog tickets with the observability outcomes.

During migration phases prepare yourself beforehand for the initial (1%, or 5%) traffic switch, so you can catch quickly those hidden use cases that only happen in production and act quickly. If you do so, other phases are just a matter of watching how scaling works.

Using containers (of course Kubernetes is a great alternative) is a fantastic opportunity to upgrade PHP runtimes efficiently at the same time where we use a much better platform that helps with delivery and developer experiences. The very first and most important step I recommend is to review how you deal with your secret and environment variables. This is pivotal for the success of a smooth migration.

We can expect that those type of applications has a fair amount of cron jobs associated with them. This is a great opportunity to follow the old saying "use the right tool for the right problem" and my suggestion would be to rewrite it, turning it into Lambda or Step Functions, as applicable to each of what the cron job is doing. This is closer to what and how a job should run.

It happens that not always we can start refactoring right away, and then I can say that my experiences with Event Bridge Scheduler triggering ECS tasks (previously cron jobs) are great. They are interestingly cheap alternatives while waiting for the refactoring project to take over. Don't take this as your permanent solution though, because it is not just right and a waste of resources and couple the cron job too much with parts of the application not really related.

We were reviewing the backlog and observability results of the last service. As we could prioritise and execute some backlog tickets, the dashboard and metrics highlighted that we had some room to review scaling and resource thresholds. We changed them carefully, resulting in a bill ~50% cheaper, CPU and memory resource stable and no performance degradation.

Some notes:

  • Investing in test automation is good for your developer experience, site reliability and revenue; also a great support for technology improvements
  • It is worth taking a look at the ALBRequestCountPerTarget metric if you have CPU-heavy processes as you can better control how ECS will handle scale policies, avoiding peak of CPU where the CPU average metric is not enough for scaling

Links:

Categorias
Tropeçando

Tropeçando 111

Don't do this: creating useless indexes

This is why, when I’m called for a performance problem (or for an audit), my first take is to look at the size of the data compared to the size of the indexes. If you store more indexes than data for a transactional workload, that’s bad. The worst I’ve seen was a database with 12 times more indexes stored on disk than data! Of course, it was a transactional workload… Would you buy a cooking book with 10 pages of recipes and 120 pages of indexes at the end of the book?

The problem with indexes is that each time you write (insert, update, delete), you will have to write to the indexes too! That can become very costly in resources and time.

PXP

PXP is a superset of the PHP programming language. It is heavily inspired by the goals of the TypeScript project and aims to improve and enhance PHP with transpilation.

Functional Classes

A place for everything, and everything in its place.

What is a class? According to the dictionary a class is:

A set, collection, group, or configuration containing members regarded as having certain attributes or traits in common; a kind or category.

The Simple Class

I work in many legacy code bases, and in fact, I’ve made it a big part of my career. I love diving into big monoliths that have grown out of proportion and tidying them up. One of the best parts of that work is rewriting a God class into a collection of small reusable classes. Let’s take a look at what makes a simple class great.

The economics of clean code

Code smarter. Code balanced. That is OK to have some debt. But pay them off quickly.

Categorias
Tropeçando

Tropeçando 110

Enabling the Optimal Serverless Platform Team — CDK and Team Topologies

Serverless, and related technologies, have enabled teams to move faster, reduce total cost of ownership and overall empowered developers to have greater ownership of the systems they build. However, Serverless is not a silver bullet — there is an organisational side that’s key to unlock the full benefits of Cloud.

Restructuring a Laravel Controller using Services, Events, Jobs, Actions, and more

A simple but nice walk-though about code decoupling.

The Serverless Server

I'm Will Jordan, and I work on SRE at Fly.io. We transmogrify Docker containers into lightweight micro-VMs and run them on our own hardware in racks around the world, so your apps can run close to your users. Check it out—your app can be up and running in minutes. This is a post about how services like ours are structured, and, in particular, what the term "serverless" has come to mean to me.

Keep Cognitive Complexity Low with PHPStan

What is cognitive complexity? It's the amount of information we have to hold in our heads simultaneously to understand the code. The more indents, continue, break, nested foreach, and if/else branches, the harder is code to read.

You can use PHPStan rules to decrease the cognitive complexity of your codebase. This brings matuiry to your application and a more maintainable code.

How to release PHP 8.1 and 7.2 package in the Same Repository

Some steps to release a package in more than one version, to allow compatibility for different PHP runtimes.

Categorias
Tropeçando

Tropeçando 109

How to Measure Your Type Coverage

Type coverage check for PHP with PHPStan.

Event Sourcing in Laravel

Granular interfaces

After refactoring to a granular interface, our system became more flexible and composable. Small interfaces communicate intent more clearly, making it easier to understand the flow of a system.

Serverless Laravel applications with AWS Lambda and PlanetScale

The Tighten Test: 12 Steps to a Better Team

Working in a good team turn your life entirely different. Tighten published this post with their heavily opinionated, based on their shared values, and sourced from their experience as web and app developers who regularly work with a variety of different software organizations. This list is based on Joel's 12 steps for better code.

Categorias
Tropeçando

Tropeçando 108

Why I Will Never Use Alpine Linux Ever Again

Alpine image is heavily use as a base image for all sort of applications. Some applications, usually running in Kubernetes, are facing issues due to Alpine implementation of musl. This article describes how those issues can cause a great amount of grief.

3 years of lift-and-shift into AWS Lambda

Let’s set the scene. We’re looking for scaling a PHP application. Googling around take us to find out that AWS Lambda is the most scalable service out there. It doesn’t support PHP natively, but we got https://bref.sh. Not only that, we also have Serverless Visually Explained which walk us through what we need to know to get PHP up and running on AWS Lambda. But we have a 8 year old project that was not designed from the ground up to be serverless. It’s not legacy. Not really. It works well, has some decent test coverage, a handful of engineers working on it and it’s been a success so far. It just has not been designed for horizontal scaling. What now?

Different beliefs about software quality

Good advices on how to deal with an environment where you have conflicts about your beliefs and how the environment work.

Increase code coverage successively

I often come across legacy projects that have a very low code coverage (or none at all). Getting such a project up to a high code coverage can be very frustrating as you will have a poor code coverage for a very long time.

So instead of generating an overall code coverage report with every pull request I tend to create a so called patch coverage report that checks how much of the patch is actually covered by tests.

Conway's Law

Pretty much all the practitioners I favor in Software Architecture are deeply suspicious of any kind of general law in the field. Good software architecture is very context-specific, analyzing trade-offs that resolve differently across a wide range of environments. But if there is one thing they all agree on, it's the importance and power of Conway's Law. Important enough to affect every system I've come across, and powerful enough that you're doomed to defeat if you try to fight it.

Is it a DTO or a Value Object?

A common misunderstanding in my workshops (well, whose fault is it then? ;)), is about the distinction between a DTO and a value object. And so I've been looking for a way to categorize these objects without mistake.

Categorias
PHP Programação

A bref AWS PHP story – Part 2

We are starting Part 2 of the Series "A bref AWS PHP history". You can check Part 1, where I presented the PHP language as a reliable and good alternative for Serverless applications.

Part 2 is to show how CDK will describe more AWS resource dependencies; how policies and roles are involved in this process; how to test if they are applied as expected; and how PHP services will use those resources.

Some of those topics seem straightforward to some people, but I would like to avoid guessing that this is known to the audience since I have experienced some PHP developers struggling to put all these together for the first time due to the paradigm change. It should be fun.

Table of contents:

  1. What else are we doing?
  2. Describing more AWS services - Adding an S3 bucket
  3. Services permissions
  4. Testing CDK
  5. PHP and AWS Services
    1. Handlers
    2. Application, Domain, Infrastructure, etc
  6. Wrap-up
  7. P.S.: Stats

What else are we doing?

The Part 1 function was returning a Fibonacci result from an int. Very simple. We will keep it simple for now to focus on putting the PHP code into a lambda and allowing PHP code to interact with AWS Services.

The computing complexity is irrelevant because it could be very complex logic or very simple, and the topics we are discussing in this part of the series will use the same design.

The lambda will now use the result of the Fibonacci of a provided integer or a random integer from 400 to 1000 (to get a good image and not to overflow integer). This integer is the number of pixels of an image from the bucket and an arbitrary request metadata we are creating. If the image does not exist, the lambda will fetch a random image from the web with that number of pixels, save it and generate the metadata.

Get the part-2 source-code on GitHub and the diff from part-1.

Describing more AWS services - Adding an S3 bucket

S3 buckets are simple yet compelling services for multipurpose workloads. It will be added to the series as a basic storage mechanism. The lambda function, now called GetFibonacciImage function, will need some permissions to manage the bucket.

Starting from the bucket definition, CDK give fantastic constructs, and it goes like this:

cdk-stack.ts

    const brefBucket = new Bucket(this, `${stackPrefix}Bucket`, {
      autoDeleteObjects: true,
      removalPolicy: RemovalPolicy.DESTROY,
    });

By default, buckets will not be deleted during a CDK destroy because they need to be empty. So you will have a hanging bucket in your account. I don't want to keep those contents if the lambda no longer exists. Then autoDeleteObjects and removalPolicy options are selected to enable the destruction of the buckets and their contents if I execute a stack destroy.

We want to decouple the configuration from the implementation to have a more SOLID code. That way, we avoid hard-coded configuration, making our code cleaner and more robust. Then, the code is ready to work, no matter the bucket name.

The implementation code is aware that the name will come from an environment variable and will work with that (yes, if you think that test will be easy to write, you are right):

cdk-stack.ts

and

      environment: {
        BUCKET_NAME: brefBucket.bucketName,
      }

Services permissions

There is a Lambda Function and an S3 Bucket. The described use case determines that the lambda needs read and write permissions to the bucket. And nothing more. It is a good practice to give the minimum necessary permission to a resource:

cdk-stack.ts

    brefBucket.grantReadWrite(getLambda);

The result is a list of actions added to the policy recommended by AWS for operations requiring only read and write.

          Action: [
            "s3:GetObject*",
            "s3:GetBucket*",
            "s3:List*",
            "s3:DeleteObject*",
            "s3:PutObject",
            "s3:PutObjectLegalHold",
            "s3:PutObjectRetention",
            "s3:PutObjectTagging",
            "s3:PutObjectVersionTagging",
            "s3:Abort*",
          ],

Testing CDK

Testing is a great feature of CDK, and we can see how tests can verify our changes with npm t:

That there is a function

  const functionName = 'GetFibonacciImage';
  /* ... */
  it('Should have a lambda function to get fibonacci', () => {
    template.hasResourceProperties('AWS::Lambda::Function', {
      Layers: [Cdk.CdkStack.brefLayerFunctionArn],
      FunctionName: functionName,
    });
  });

And if only the permissions the lambda needs were granted:

  it('Should have a policy for S3', () => {
    template.hasResourceProperties('AWS::IAM::Policy', {
      PolicyName: Match.stringLikeRegexp(`^${stackPrefix}${functionName}ServiceRoleDefaultPolicy`),
      PolicyDocument: {
        Statement: [{
          Action: [
            "s3:GetObject*",
            "s3:GetBucket*",
            "s3:List*",
            "s3:DeleteObject*",
            "s3:PutObject",
            "s3:PutObjectLegalHold",
            "s3:PutObjectRetention",
            "s3:PutObjectTagging",
            "s3:PutObjectVersionTagging",
            "s3:Abort*",
          ],
        }],
      },
    });
  });

You may want to check cdk-stack.test.ts to see more details.

PHP and AWS Services

This is the part where we have fewer serverless needs impacting the code, as the PHP code will follow the same logic we might be using to communicate with AWS services on any other platform overall (there are always some specific use cases).

The reuse of the same existing logic is excellent. It leverages the decision to keep using PHP when moving that workload to Serverless, as the bulk of the knowledge and already proven code would remain as-is. We may escape the trap of classifying that PHP code as legacy as if it should be avoided, terminated or hated.

As a side note, a few external layers of our software architecture are touched if a good software architecture was applied before. Therefore, during the implementation of this architectural change, it should be quick to realise how beneficial and time-saving it is to have a well-architectured application with a balanced decision for patterns, principles, and designs to be applied, ultimately giving flexibility to the application and its features.

The handler is simplified now and should accommodate everything to a class in the direction of following SRP, a principle that we are bringing to the code during the code bites:

Handlers

php/handler/get.php

return function ($request, $context) {
    return \BrefStory\Application\ServiceFactory::createGetFibonacciImageHandler()
        ->handle($request, $context)
        ->toApiGatewayFormatV2();
};

To handle the request details, the Fibonacci code now lives in a proper event handler (implements Bref\Event\Handler).

php/src/Event/Handler/GetFibonacciImageHandler.php

    public function handle($event, Context $context): HttpResponse
    {
        $int = (int) (
            $event['queryStringParameters']['int'] ?? random_int(
                self::MIN_PIXELS_FOR_REASONABLE_IMAGE_AND_NOT_BIG_FIBONACCI,
                self::MAX_PIXELS_FOR_REASONABLE_IMAGE_AND_NOT_BIG_FIBONACCI
            )
        );

        $metadata = $this->photoService->getJpegImageFor($int);

        $responseBody = [
            'context' => $context,
            'now' => $this->dateTimeImmutable()->format('Y-m-d H:i:s'),
            'int' => $int,
            'fibonacci' => $this->fibonacci($int),
            'metadata' => $metadata,
        ];

        $response = new JsonResponse($responseBody);

        return new HttpResponse($response->getContent(), $response->headers->all());
    }

We would also like to start testing the PHP code. As the Event Handler might be a new layer (although very similar to widely used controllers), php/tests/unit/Event/Handler/GetFibonacciImageHandlerTest.php test class was created for that. The part-2 will only focus on this test class to avoid overloading with too many changes, but we would usually have test coverage for all the code in the repository.

Applications, domains, infrastructure, etc

Finally, we are inside the layers where we are most used to. To fit our purposes, the Event Handler will depend on and call an Application layer service that will orchestrate all the steps to fetch the image metadata.

php/src/Application/PicsumPhotoService.php#L34-L42

    public function getJpegImageFor(int $imagePixels): array
    {
        try {
            return $this->getImageFromBucket($imagePixels);
        } catch (NoSuchKeyException) {
            // do nothing
        }

        return $this->fetchAndSaveImageToBucket($imagePixels);
    }

The interesting thing to mention about using AWS Services is how simple S3Client is instantiated. There is a factory to create service:

php/src/Application/ServiceFactory.php#L22-L29

    public static function createPicsumPhotoService(): PicsumPhotoService
    {
        return new PicsumPhotoService(
            HttpClient::create(),
            new S3Client(),
            getenv('BUCKET_NAME'),
        );
    }
  • new S3Client is all we need because the environment will use AWS credentials, provided to lambda at execution time, as an assumed role that will carry the policies we defined in the CDK constructs stack, i.e., with read and write permissions to the bucket
  • getenv('BUCKET_NAME'), which is gracefully provided by CDK when creating our bucket with any dynamic name it pleases to

I asked ChatGPT about this class:

The PicsumPhotoService class seems to be following the Single Responsibility Principle (SRP) as it has only one responsibility, which is to provide methods for fetching and saving JPEG images from the Picsum website.

The class has methods to fetch the image from an S3 bucket, and if it's not available, fetches it from the Picsum website, saves it to the S3 bucket, and creates and puts metadata for the image in the S3 bucket.

The class has a clear separation of concerns, where the S3Client and HttpClientInterface are injected through the constructor, and the different functionalities are implemented in separate private methods. Additionally, each method is doing a single task, which makes the code easy to read, test, and maintain.

Therefore, it can be concluded that the PicsumPhotoService class follows SRP.

Wrap-up

It would be simple like that. Check more details in the source code, install it and try it yourself. This project is ready to:

  • Create a lambda function using Bref
  • Create an S3 Bucket with read and write permissions to the lambda
  • Test the stack Cloudformation code
  • Separate the PHP logic
  • Have PHP communicating with AWS Services
  • Start PHP testing

P.S.: Stats

I did not plan to talk widely about stats now, but I think I can share the most two significant measures I had with this simple code so far.

[Update 22/03/23] Using https://k6.io/

1 - With a brand new stack and a cold lambda:

scenarios: (100.00%) 1 scenario, 200 max VUs, 2m30s max duration (incl. graceful stop):
           * default: 200 looping VUs for 2m0s (gracefulStop: 30s)

     data_received..................: 49 MB  409 kB/s
     data_sent......................: 7.8 MB 65 kB/s
     http_req_blocked...............: avg=2.36ms   min=671ns    med=2.27µs   max=581.87ms p(90)=4.18µs   p(95)=7µs
     http_req_connecting............: avg=712.63µs min=0s       med=0s       max=193.34ms p(90)=0s       p(95)=0s
     http_req_duration..............: avg=531.51ms min=204.46ms med=485.24ms max=3.81s    p(90)=517.98ms p(95)=534.3ms
       { expected_response:true }...: avg=513.6ms  min=204.46ms med=485.07ms max=3.67s    p(90)=516.62ms p(95)=531.5ms
     http_req_failed................: 0.60%  ✓ 272        ✗ 44761
     http_req_receiving.............: avg=123.76µs min=13.77µs  med=44.04µs  max=16.78ms  p(90)=71.27µs  p(95)=85.71µs
     http_req_sending...............: avg=14.79µs  min=4.27µs   med=12.43µs  max=402.74µs p(90)=23.97µs  p(95)=31.4µs
     http_req_tls_handshaking.......: avg=1.37ms   min=0s       med=0s       max=330.58ms p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=531.37ms min=204.36ms med=485.11ms max=3.81s    p(90)=517.77ms p(95)=534.13ms
     http_reqs......................: 45033  373.683517/s
     iteration_duration.............: avg=533.96ms min=204.55ms med=485.34ms max=4.37s    p(90)=518.07ms p(95)=534.4ms
     iterations.....................: 45033  373.683517/s
     vus............................: 200    min=200      max=200
     vus_max........................: 200    min=200      max=200

running (2m00.5s), 000/200 VUs, 45033 complete and 0 interrupted iterations

2 - After the first initial execution, cold lambda and all available images already saved to the bucket, where we got ~3K more requests being served for the same time

scenarios: (100.00%) 1 scenario, 200 max VUs, 2m30s max duration (incl. graceful stop):
           * default: 200 looping VUs for 2m0s (gracefulStop: 30s)

     data_received..................: 53 MB  442 kB/s
     data_sent......................: 8.4 MB 70 kB/s
     http_req_blocked...............: avg=2.26ms   min=631ns    med=2.24µs   max=612.22ms p(90)=4.04µs   p(95)=6.47µs
     http_req_connecting............: avg=663.23µs min=0s       med=0s       max=215.19ms p(90)=0s       p(95)=0s
     http_req_duration..............: avg=490.8ms  min=199.95ms med=484.02ms max=3.17s    p(90)=514.86ms p(95)=527ms
       { expected_response:true }...: avg=490.53ms min=199.95ms med=484.02ms max=2.4s     p(90)=514.85ms p(95)=526.99ms
     http_req_failed................: 0.01%  ✓ 5         ✗ 48754
     http_req_receiving.............: avg=108.86µs min=12.44µs  med=42.68µs  max=17.62ms  p(90)=69.23µs  p(95)=81.87µs
     http_req_sending...............: avg=14.42µs  min=3.9µs    med=12.14µs  max=786.01µs p(90)=23.03µs  p(95)=30.35µs
     http_req_tls_handshaking.......: avg=1.27ms   min=0s       med=0s       max=332.34ms p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=490.68ms min=199.9ms  med=483.91ms max=3.17s    p(90)=514.75ms p(95)=526.89ms
     http_reqs......................: 48759  404.56812/s
     iteration_duration.............: avg=493.16ms min=200.05ms med=484.11ms max=3.17s    p(90)=514.96ms p(95)=527.1ms
     iterations.....................: 48759  404.56812/s
     vus............................: 200    min=200     max=200
     vus_max........................: 200    min=200     max=200

running (2m00.5s), 000/200 VUs, 48759 complete and 0 interrupted iterations
Categorias
Tropeçando

Tropeçando 107

Why Writing Is Important for Engineers

Learning by teaching has been an important pedagogical approach for a long time. As engineers, most of us are probably familiar with the story of the rubber duck, made popular in The Pragmatic Programmer, where explaining your problem to an inanimate object helps you understand it better yourself.

A Comprehensive Guide to Undoing Changes In Git

Git is a powerful version control system for tracking source code changes, for small and large projects alike. Sometimes you’ll encounter situations that require you to undo changes you’ve made to a Git repository.

Git changes cannot be undone with a simple back button. This is intended to protect the integrity of the codebase. Instead, you’ll need to learn the proper Git commands and the appropriate situations for using each command.

How the PHP Middleware Pattern works and can easily be applied

In this post we'll be looking at Middleware in PHP. This pattern is most common in the handling of requests and responses. But the Middleware Pattern can also be applied in various other places. We'll look into what middleware is, how middleware works, when middleware can be useful and what an alternative to middleware might be.

Tuning PostgreSQL Auto-vacuum

In PostgreSQL, rows that are deleted or modified are not completely removed. Rather they are marked as dead tuples. In other words, PostgreSQL does not physically remove these rows but places a marker on them to prevent future queries from returning or accessing dead tuples.

PostgreSQL provides the VACUUM command and VACUUM ANALYZE commands to get rid of dead rows or tuples. However, both commands differ in how they operate.

Learn Vim Progressively

tl;dr: You want to teach yourself vim (the best text editor known to human kind) in the fastest way possible. This is my way of doing it. You start by learning the minimal to survive, then you integrate all the tricks slowly.

Vim the Six Billion Dollar editor

Better, Stronger, Faster.

Categorias
PHP Programação

A bref AWS PHP story – Part 1

The PHP language is a true and good alternative for Serverless applications. PHP is a fast and flexible programming language, and there are many business treasures inside PHP applications, business logic running well for years inside company codebases worldwide.

We don't need to look at PHP as a language that could not run inside a modernized stack. We can move some of this code without total refactoring to Serverless applications, benefiting from an already proven successful code. And we know we all have flows suitable to run as a lambda function.

And not only legacy code. New features are also perfect candidates to be run in PHP and lambdas due to the team's experience, consistency of the technology stack, speed, etc. PHP has served the world well and will remain operating well. PHP is alive.

Table of contents:

  1. The series
  2. Functions
  3. Code
    1. Requirements
    2. The lambda
  4. Wrap-up

The Serie

I am starting a series as a walkthrough for PHP into Serverless, specifically to run as lambdas functions.

We will use Bref, a composer package, to deploy PHP applications to AWS.

Bref (which means "brief" in french) comes as an open source Composer package and helps you deploy PHP applications to AWS and run them on AWS Lambda.
https://bref.sh/docs/

Bref relies on the Serverless framework and AWS access keys to deploy applications.
https://bref.sh/docs/installation.html

The Serverless framework is excellent, but I am more of a fan of AWS CDK. Mainly because it is designed to use an imperative programming framework that speeds up the required infrastructure with excellent constructs on different levels (reasonable defaults), and its output can be run against a test framework (predictability).

There are already some CDK constructs for PHP, but, as far as I see, they are intended to be used by Web Apps lambdas (i.e. using frameworks such as Laravel and Symfony). However, the purpose of this series is to run Event-Driven functions, so I will start using pure CDK constructs.

Functions

As a walkthrough, we will digest the series in affordable bites, starting from simple functions that we will improve as the series continues and we use more AWS resources.

Code

Let's start our PHP lambda function. First, it will begin as an HTTP-based lambda, expecting a request and returning a response. Then, it will execute a trivial piece of computing code: it will return fibonacci.

Get the part 1 source-code in GitHub.

Requirements

(optional) There is a Dockerfile and a docker-compose.yml file for your convenience if you prefer to use docker. It will require you to set AWS environment variables for use by the container.

The lambda

You can check the complete source code for part 1, and we will highlight essential parts from the CDK code, as the PHP code has a few different things from what we are used to code.

The stack creator

In our case, it will create the serverless Stack and related infrastructure, i.e., IAM, lambda function, and URL. If anything else we need, it would be defined and requested by this class.

bin/cdk-stack.ts

export class CdkStack extends Stack {

  // Get Bref layer ARN from https://runtimes.bref.sh/
  public static brefLayerFunctionArn = 'arn:aws:lambda:us-east-1:209497400698:layer:php-82:16';

  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const layer = LayerVersion.fromLayerVersionArn(this, 'php-layer', CdkStack.brefLayerFunctionArn);

    const getLambda = new LambdaFunction(this, 'get', {
      layers: [layer],
      handler: 'get.php',
      runtime: Runtime.PROVIDED_AL2,
      code: Code.fromAsset(join(__dirname, `../assets/get`)),
      functionName: 'part1-get',
    });

    const fnUrl = getLambda.addFunctionUrl({authType: FunctionUrlAuthType.NONE});

    new CfnOutput(this, 'TheUrl', {
      // The .url attributes will return the unique Function URL
      value: fnUrl.url,
    });
  }
}

Highlights

The bref php layer:

public static brefLayerFunctionArn = 'arn:aws:lambda:us-east-1:209497400698:layer:php-82:16';

Where you point your entry point and source code:

      handler: 'get.php',
      runtime: Runtime.PROVIDED_AL2,
      code: Code.fromAsset(join(__dirname, `../assets/get`)), // get.php file inside the zip file located at this path

Using AWS Lambda built-int function URL (we will change to API Gateway later if needed):

    const fnUrl = getLambda.addFunctionUrl({authType: FunctionUrlAuthType.NONE});

Output

You would see CDK outputting the lambda function URL you will use to run your application. Something like:

Outputs:
CdkStack.TheUrl = https://6eoftivwkq4ht65d2h2fwlmsga0vnpfs.lambda-url.us-east-1.on.aws/

The handler

The PHP entry point has usually named a handler to the code. Its responsibility would be to forward the request to a controller or service that will perform the business rules and prepare the response to be returned. This is an HTTP-based lambda; the response should be an HTTP-valid response.

Obs.: You can note by the words above that any existing code that fits in the lambda computing model can be the controller or service to be called by the handler. Theoretically, you only need to create the handler compatible with the lambda environment, instantiate your controller or service, pass whatever it requires as an argument, and then return the expected response.

php/handlers/get.php

<?php
return function ($request) {
    $int = (int) ($request['queryStringParameters']['int'] ?? random_int(1, 300));

    $responseBody = [
        'response' => 'OK. Time: ' . time(),
        'now' => date('Y-m-d H:i:s'),
        'int' => $int,
        'result' => fibonacci($int),
    ];

    $response = new \Symfony\Component\HttpFoundation\JsonResponse($responseBody);

    return (new \Bref\Event\Http\HttpResponse($response->getContent(), $response->headers->all()))->toApiGatewayFormatV2();
};

Highlights

All handlers receive a request object. This is how to access /?int=myValue query string param.

    $int = (int) ($request['queryStringParameters']['int'] ?? random_int(1, 300));

The call to the function fibonacci() is how we would call any other controller or service.

'result' => fibonacci($int),

Using the Symfony Response to validate and prepare a valid HTTP response:

$response = new \Symfony\Component\HttpFoundation\JsonResponse($responseBody);

AWS API Gateway requires a certain Response shape. To be sure to have a valid API Gateway response:

    return (new \Bref\Event\Http\HttpResponse($response->getContent(), $response->headers->all()))->toApiGatewayFormatV2();

And that is it. You can now use your lambda function URL as in the output of the CDK stack above and call it with or without the query string param ?/int=.

➜   curl https://6eoftivwkq4ht65d2h2fwlmsga0vnpfs.lambda-url.us-east-1.on.aws/
{"response":"OK. Time: 1674612343","now":"2023-01-25 02:05:43","int":273,"result":5.05988662735923e+56}%

➜   curl https://6eoftivwkq4ht65d2h2fwlmsga0vnpfs.lambda-url.us-east-1.on.aws/\?int\=500
{"response":"OK. Time: 1674612353","now":"2023-01-25 02:05:53","int":500,"result":1.394232245616977e+104}%

➜   curl https://6eoftivwkq4ht65d2h2fwlmsga0vnpfs.lambda-url.us-east-1.on.aws/\?int\=500
{"response":"OK. Time: 1674612356","now":"2023-01-25 02:05:56","int":500,"result":1.394232245616977e+104}%

The test

We can predict the resources we create via CDK and check if those resources are as expected. The output of the CDK is a CloudFormation template, which we can put under test. That is solid, as unexpected behaviour or changes will fail in our CI pipeline test step.

test/cdk.test.ts

test('Lambda created', () => {
  const app = new cdk.App();
    // WHEN
  const Stack = new Cdk.CdkStack(app, 'MyTestStack');
    // THEN
  const template = Template.fromStack(stack);

  template.hasResourceProperties('AWS::Lambda::Function', {
    Layers: [Cdk.CdkStack.brefLayerFunctionArn]
  });
});

Highlights

We are checking if there is a lambda function and if that function is using the expected specific bref layer:

  template.hasResourceProperties('AWS::Lambda::Function', {
    Layers: [Cdk.CdkStack.brefLayerFunctionArn]
  });

Wrap-up

We have created our Stack and our first simple HTTP-based PHP lambda function using CDK (with tests). Next, we will improve our lambda to use more AWS resources and communication with more complex application services.