Matt and I are out in Los Angeles this week for KubeCon 2021 this week. At the GitOpsCon event Tuesday we were excited to attend this Kubernetes session: GitOps in the Real World: Opportunities for Developer Experience Improvement
GitOps is a pattern for managing the state of Kubernetes clusters using git as the source of truth. The entire state of the cluster is declared in manifests stored in git repositories and any changes to the manifests follow well-known git processes. Once the manifests are version controlled in git, then there’s a number of state reconcilers (Flux, ArgoCD and the like) that can automatically apply changes from the repository. However, this leaves a significant gap in the process: How do we build and get the manifests *into the git repositories* in the first place? This talk will walk through Chick-fil-A’s experiences with GitOps to manage the state of our production clusters at scale and offer what we see as opportunities to improve the frontend of the process.
Christopher Lane & Alex Crane, Chick-fil-A
Architecture of the Sandwich
The Chick-fil-A team shared part of their high level architecture of the application. They shared that 20% of orders come through a digital channel and these include both web and mobile. Their backend system is running in AWS, they mentioned being able to swap between both us-east and us-west for redundancy. At a high level this looks like a collection of microservices with supporting cloud services for network and storage.
Developer Experience: GitOps
One of the interesting perspectives from this presentation is about how GitOps can make life easier for developers. With their pre-configured Application Platform, the developers do not need to know as much about the infrastructure in order to be able to continuously deploy. Chick-fil-A covers some of the top technologies they use as part of their platform:
- ArgoCD – this is their primary GitOps system and is how they push changes into the environment
- Prometheus / Grafana / Thanos – this is their primary Kubernetes monitoring solution
- Speedscale – we were excited to get a shout-out about our technology being core for traffic capture and replay between environments
- AWS – they rely heavily on cloud services for identity, compute, and networking
Dealing with Scale
Their system has to deal with spiky traffic around meal time, peaking at over 300K requests per minute. In addition to closely monitoring these services in production, they also need to validate that new code can handle this kind of scale.
As Chick-fil-a continues to develop for resiliency and great dining experience, managing high traffic and optimizing their applications will be a key part of the journey. Being able to observe realistic traffic and test the scalability of improvements is a critical capability accelerating their rollout.
Speedscale at KubeCon
If you are interested in some of the sessions we are check out: Looking forward to KubeCon.
And if this user story made you hungry for more information about Speedscale, request a trial today.