Serverless isn't great at queue management

This is a continuation of my previous post where I talked about the challenges of using serverless/Function as a Service (FaaS) compute systems for ETL (Extract, Transform, Load) jobs. It sat in my drafts folder for a long time, so I just decided to publish it as is.

I used to work at AWS, and predictably we used a lot of AWS cloud services. In many cases, when an engineer looks for a service platform, they’ll often go directly to AWS Lambda because “it’s Serverless” with the justification that it’s simple and alternatives are too complex and not worth considering.

Continue reading...

Creating Kyverno policies using LLMs in Open WebUI

Kyverno is a handy policy engine for Kubernetes. For example, I use them to improve security by enabling user namespaces, or fix compatibility issues. I frequently use to validate or mutate resources that are created.

However, the YAML format can be a bit tricky for some of my more complicated policies. LLMs do alright creating many policies, but can struggle with my more complicated policies because they have to one-shot the policy and create it in one pass with no feedback or validation.

In this post, I’m going to show how I configure OpenWebUI to help me draft Kyverno policies.

Continue reading...

Forgejo deploying workflows to Kubernetes using OIDC

As I switch from GitHub to Forgejo for my various project repositories, I find myself looking for a way to securely deploy new services into my Kubernetes cluster without having to keep updating Kubernetes auth tokens and storing them as a workflow secret.

GitHub Actions and, as of Forgejo v15, Forgejo Actions also supports providing temporary JWT tokens to workflow steps that securely state what repository and other commits details a given build is for. When a workflow is running, it can pass it to Kubernetes and other supported services and authenticate as a secure client with all the identity information of that repository and workflow. This is called OpenID Connect or OIDC.

This is super powerful because can federate identity and minimize the number of secrets required–great the next time a widely used workflow step gets compromised and steps all your secrets. Having fewer secrets helps.

In this post, I’m going to show I configured Nix, Kubernetes, and Forgejo to securely update a deployment.

Continue reading...