GSoC 2020, Prometheus (CNCF)

This post summarizes the work I did in Prometheus (Cloud Native Computing Foundation) as part of Google Summer of Code 2020.

I'm thankful to Hrishikesh Barman (mentor) for being extremely helpful and supportive throughout the period.

During GSoC I worked on supporting Amazon EKS for Prombench and Funcbench — the benchmarking stack Prometheus uses to stress-test scrapers, readers, and the data path at scale. Prombench wasn't glamorous (thousands of pods, scrapers, readers), but it was honest: you learn quickly that observability is a database problem wearing a monitoring costume.

prometheus/test-infra

Added support for EKS

This was one of my main tasks for the GSoC period. I worked with the EKS SDK to add CLI-based commands for cluster operations: create cluster, create node groups, apply resources, and related workflows.

Make test infra more generic

Initially test infra supported only GKE, so commands and docs were GKE-specific. As we added EKS, I refactored commands and documentation so new providers could plug in without copying GKE assumptions everywhere.

Programmatic subnets for EKS

Unlike GKE, EKS requires a VPC before you create a cluster, and that VPC is tightly coupled to the cluster name. For Prombench — where we mostly spin up nodes for a given test — that model works. For Funcbench, where we create a new cluster for every test run, it breaks down. I added support for programmatic subnets so Funcbench could keep creating fresh clusters without VPC collisions.

Grafana dashboard

I planned to update the Grafana dashboard during GSoC but didn't finish. I did update the Prometheus meta dashboard version; the full dashboard work was left for after the EKS and infra changes landed.

prometheus/prometheus

GitHub workflow updates

Changes in test-infra (making it provider-generic) required updates to Prombench and Funcbench workflow files in the main Prometheus repo so CI could use the new commands and paths.

Future work

Several items from my proposal didn't fit in the GSoC window. I planned to continue once the EKS and generic infra work was merged — including the Grafana dashboard work above.

Originally published on Medium (Aug 29, 2020).