We’re developing a software product whose production environments run entirely on AWS. Keeping local development as close as possible to those environments is important to us: It reduces surprises when deploying, shortens the feedback loop when debugging infrastructure-related issues, and means developers can work confidently without needing access to a shared cloud account. To achieve…
Read MoreThere’s a particular kind of irony in watching your observability stack become the thing you most need to observe. That’s more or less our day job, and it’s why we tend to be unusually deliberate about tooling decisions: The small ones have a habit of compounding over time. So when it came to choosing a…
Read MoreFor a long time, Sealed Secrets was our go-to solution for managing confidential data like database credentials, tokens and private keys within Kubernetes. However, as our operations grew, the operational overhead of this approach led us to migrate to External Secrets. This transition has significantly streamlined our workflow, enhanced our security posture during credential rotations,…
Read MoreHi everyone 😃 Today I’d like to walk you through some experiments we ran about load-balancing requests to LLMs in Kubernetes. Let’s dive deep into it! Why Deploy LLMs in Kubernetes? Well, Kubernetes has now established itself as a leading technology when it comes to workloads orchestration. And with time the increasing support for accelerators…
Read MoreOne of your servers is reporting moderate CPU usage, enough available memory, and storage that’s busy but not saturated. The dashboard looks healthy, the application does not. Requests to it are slowing down, background jobs are taking longer, and latency is rising even though no resource appears fully exhausted. The problem here is that most…
Read MoreIn a previous post, I wrote about the challenges of building a public GitHub organization correctly: The need for structure, consistency, and enforcement rather than relying on memory. What I didn’t cover was how we actually solved it. The short answer: We treat the organization as infrastructure. Repositories, teams, branch protection rules, labels, permissions, all…
Read MoreHow we adapted our monitoring approach when the default choice didn’t fitand what we learned along the way. You’ve probably been there. You walk into a new observability project, playbook in hand, only to realize that no two projects are ever really the same. Every team has its own tools, its own platform quirks, and…
Read MoreA Simple Reverse Proxy Might Turn out to Be Dangerous Sometimes we inadvertently make assumptions that undermine our infrastructure security. In today’s article I want to share with you one of the most common mistakes that are made when setting up a reverse proxy. As always, real world use cases are the best ones to…
Read MoreCreating a GitHub organization is easy. Creating a public one that is actually well-structured, secure, and maintainable over time… not so much. At the beginning, it feels like a simple task: create the org, push some repositories, maybe define a couple of teams, and you’re done. But as soon as things become public, the whole…
Read MoreThis is the first article in the RTO series The Problem Red team and penetration testing activities are full of repetition: the network scans, reconnaissance, OSINT collection, and routine validation tasks are all necessary, but they’re also time-consuming and error-prone when executed manually. Over time, most teams end up with a zoo of scripts, half-maintained…
Read MoreThe year has just come to an end, and after a long stretch of work I could really use some rest. But before closing everything out completely, there’s one last thing I wan to share, and it needs a bit of context about the project. We’re currently building a distributed application consisting of several components…
Read MoreTests are often treated as a safety net: something that catches bugs before they reach users. While that’s true, it’s only part of the story. A test suite is also a mirror. If you look closely, it tells you a lot about the structure, health, and long-term maintainability of your codebase. If you know how…
Read MoreKubernetes Operators are one of those ideas that feel magical when they work: you declare an intent/goal in YAML, and software continuously makes the cluster match it – handling upgrades, failures, drift, and lifecycle cleanup: like a purpose-built SRE on autopilot. Although in theory it looks like sci-fi fiction, in practice Operators are just code…
Read MoreWhen I got back from my last trip to Japan, I quickly bumped into a small but annoying problem: I had finally reached my free storage limit on Google Photos. Instead of just buying more space for a few euros a year, I saw it as the perfect excuse to try something I’ve been considering…
Read MoreA modern web app isn’t one single big monolith: it’s made of quite a lot of pieces! For instance, we relied on a setup such as this one for a recent one we are developing: That’s a lot of moving parts. You could glue them together with scripts, sticky notes, and caffeine… but then most…
Read More