https://zhanyl-tech.github.io/tags/prometheus/ Recent content in Prometheus on Zhanyl Abdybaeva Hugo en-us Sun, 26 Jul 2026 00:00:00 +0000 https://zhanyl-tech.github.io/projects/ib-slurm-exporter/ Sun, 26 Jul 2026 00:00:00 +0000 https://zhanyl-tech.github.io/projects/ib-slurm-exporter/ Correlate InfiniBand and RoCE counters with the Slurm job that owns them, so a slow collective can be traced to the fabric. https://zhanyl-tech.github.io/projects/gpu-reaper/ Sun, 26 Jul 2026 00:00:00 +0000 https://zhanyl-tech.github.io/projects/gpu-reaper/ Find wasted GPU allocations on a Slurm cluster and escalate them through alert, drain, and cancel. https://zhanyl-tech.github.io/experiments/2026-07-26-slurm-25-11-upgrade-notes/ Sun, 26 Jul 2026 00:00:00 +0000 https://zhanyl-tech.github.io/experiments/2026-07-26-slurm-25-11-upgrade-notes/ Three versions, two config renames that break on restart, and one cgroup change that quietly breaks every job-attribution tool you own.