Omar
El Jamal

A self-operated 4-node Kubernetes cluster (Talos VMs on Proxmox, k8s v1.33) that runs real personal infrastructure — media stack, databases, monitoring — entirely through GitOps with FluxCD. The repo enforces a hard rule: no manual kubectl apply, every change goes through Git review and automated reconciliation. It covers the full SRE/platform stack: Ceph RBD storage, HA PostgreSQL via CloudNativePG, SOPS+age field-level secret encryption, MetalLB/Traefik load balancing behind a keepalived HA pair, and an internal ACME CA via cert-manager + step-ca. What sets it apart is the operational paper trail — a written disaster-recovery runbook and real incident postmortems committed alongside the manifests, including recovering a Ceph OSD_FULL/WAL-corruption event via a Barman archive restore. This is infrastructure that has broken in production and been brought back, on the record.

Stood up and operate the internal tooling stack for a startup — Matrix/Synapse with Element Web and Element Call (real bidirectional audio/video verified end to end), self-hosted Vikunja, Trilium, Grafana and Prometheus, and a self-hosted Plane whose eight containers share Postgres and Valkey with the rest of the stack while keeping their own RabbitMQ and MinIO. Twenty services on a single Hetzner CX43, fronted by Caddy with DNS-01 wildcard TLS and reachable only over Tailscale. What sets it apart is a real disaster-recovery program rather than a checkbox: automated daily backups sync off-site to a Terraform-managed S3 bucket whose IAM user deliberately holds no s3:DeleteObject — retention is enforced by bucket lifecycle rules on a grandfather-father-son schedule (3d / 21d / 180d / 1095d), so a compromised host cannot erase its own backups. Then it was actually tested: a full restore drill on a throwaway VM verified three PostgreSQL databases, all 11 named volumes and a byte-identical auth-service config, and produced measured RPO (~24 h) and RTO (~30 min) instead of assumed ones. The write-up keeps the trap the drill exposed — replaying the logical dumps and the raw postgres_data snapshot onto a running container silently corrupts it — and states plainly what the drill did not cover.

Platform build-out for a university scientific-computing group, under contract: a Talos Linux Kubernetes cluster on the sc1 host, running across seven libvirt VMs provisioned with boxman — the same tool I deliver features to upstream, which closes a genuinely useful loop between the two engagements. Machine lifecycle runs through a self-hosted Omni instance; everything above it is FluxCD GitOps over Kustomize base/overlays, with cert-manager, Traefik on hostNetwork, Rook-Ceph across two storage tiers, and SOPS/age-encrypted secrets. On top of that, Matrix Synapse, Element Web and PostgreSQL are deployed Kubernetes-native rather than lifted from compose. The commit log is deliberately honest about what the environment actually enforced — a run of nine consecutive fixes in a single day pins down each constraint as it was found: the PodSecurity baseline profile rejecting hostNetwork, the chart validating hostPort against port, and a rolling update needing maxSurge: 0 alongside a non-zero maxUnavailable once ports are host-bound.

A faculty-supervised capstone (with a two-person team) building and evaluating adversarial-robustness defenses for image classifiers — ResNet18 and WideResNet34-10 against FGSM, PGD, Carlini-Wagner, and adversarial-patch attacks — plus an original "adversarial routing" detection layer whose scoring heuristics were iteratively retuned across dozens of commits ("max-confidence routing" → "suspicious floor" → "patch-routing collapse" fixes). It pairs roughly 17,500 lines of PyTorch/Optuna research code, run as batch sweeps on the university HPC cluster via SLURM, with a separate full-stack FastAPI + React/Vite dashboard (~2,000 lines) for live inference monitoring, model configuration, evaluation, and experiment launching. Genuinely rare to see research depth and a usable product UI shipped from the same hands.

A complete distributed system, solo end-to-end: a Kotlin / Jetpack Compose Android app collects cellular signal telemetry (signal power, SNR, network type, operator, band) via background WorkManager jobs and posts it to a FastAPI + PostgreSQL backend. The backend runs JWT/OAuth2 auth with bcrypt hashing, per-IP rate limiting via SlowAPI, transactional upsert helpers, and dual SQLite/PostgreSQL compatibility for easy local dev. Built in a tight five-week sprint, it ships with a 359-line pytest suite against an in-memory SQLite engine, a GitHub Actions pipeline that produces signed Android release APKs and pushes Docker images, and a Helm chart for Kubernetes rollout. Genuinely deployable, not a tutorial app.

Six independent Node.js / Express microservices — auth, catalog, cart, checkout, admin, invoice — built with deliberate polyglot persistence that mirrors real AWS service semantics: PostgreSQL for relational data, DynamoDB for ephemeral cart state with TTL, and an async S3 + SES + SQS pipeline for invoice PDF generation handled by a dedicated worker. Alongside it, 3,508 lines of Terraform stand up the matching ECS / IAM / ECR / DynamoDB infrastructure with OIDC-based GitHub Actions deploy roles across separate dev and prod environments. The whole thing — design, six services, IaC, four CI pipelines — went from nothing to running in an intense five-day sprint (peak: 30 commits from the team in a single day, debugging RDS SSL and IAM roles live).

Sustained collaboration on a 181-commit, multi-author HPC cluster automation project owned by a research-computing colleague. Independently designed and built two Ansible roles from a blank page — cobbler_pxe (full PXE/DHCP/DNS network-boot provisioning, written from a 228-line design doc through to an idempotent lineinfile-based config patcher and a four-file pytest suite) and nginx (reverse-proxy role plus a deployment-playbook refactor) — then went on to restore a vaultwarden secrets-manager role and write a "pxe-boot-testing" Copilot automation skill. Across 42 commits that's roughly +44,996 / −408 lines of real changes accepted into someone else's review process — the strongest evidence of working to another team's conventions rather than a green-field of one's own.

A green-field infrastructure-automation project: 11 Ansible roles and 11 playbooks orchestrating a complete self-hosted GitLab stack — libvirt/KVM VMs provisioned from Ubuntu cloud images via cloud-init, Docker-based GitLab fronted by Traefik with automatic TLS routing, WireGuard VPN peer management, and an iVentoy PXE server, optionally deployable to Kubernetes via Helm instead. It's organised across a properly layered multi-environment inventory (homelab/dev/prod) with Ansible Vault for secrets and .example.yml templating. ~2,300 lines of original Ansible YAML, written solo over two months with no scaffolding to lean on, then refactored twice through self-reviewed PRs.

Two tracked remotes of the same Python infrastructure tool (a mentor's project, inspired by Docker Compose / Vagrant, with a libvirt provider). On the fork, independently designed and built an entire OCI container-registry subsystem from scratch — a five-module images/ package handling inspection, caching, metadata, resolution and CLI, plus push support, backed by ~330 lines of dedicated tests and a 350-line tutorial. On the upstream-tracking clone, wrote a 296-line PXE-boot design document first, then implemented it end-to-end as a new BareVM provider with its own test files, and separately delivered libvirt-specific OCI push support plus a 380-line LaTeX/Beamer presentation introducing the tool. +16,274 lines combined — a clean design-doc-first → implementation → tests → docs workflow, repeated twice.