Applied AI in Operations

Audits autonomous LLM incident triage across 368 support cases and 787 runs under ISO/IEC 42001 and 5259 controls, proving that self-reported verbalized confidence carries no discriminative signal (AUROC 0.460 / 0.518) and retracting an earlier n=30 confidence-inversion claim.

Agent 1 perspective: proves multi-task sentiment collapses to MCC -0.018 under 20:1 imbalance.

Compares four frontier language models on the same production B2B support queue, measuring where reasoning models separate from fast models and enforcing a local 29-ms monotonic privacy redaction firewall before egress.

De-confounds observational cloud database release telemetry by separating account-concentration workload shifts from software regressions using coarsened stratification and prior-weighted e-BH control.

fsync T5: Drain masks digit-bearing tokens and groups by length and leading tokens, so a variable-length op phrase is masked or splits templates. WTL's source-derived catalog with manual-force rows (e.g. 'Time spent sync call for <*>') is the lever if the S1 op is ever split at template level.

Zhu et al. (ICSE-SEIP 2019, logparser & Loghub-2k) benchmark 13 automated log parsers across 16 open-source systems (2,000 lines per system) using Grouping Accuracy (GA: fraction of log lines whose induced partition class equals the ground-truth template's partition class) and demonstrate the necessity of domain-specific regex preprocessing (IPs, block…

Zeng, Hui, Shen, Pavlo, McKinney & Zhang (PVLDB 2023) empirically evaluate Parquet and ORC columnar internals (row-group sizing, dictionary/RLE encoding, zone maps, and predicate pushdown).

WER is the canonical industrial field-failure corpus: automatic opt-in crash reports bucketed by heuristic signatures at billion-report scale, with explicit treatment of bucketing error and selective data collection.

Chen et al. (EuroSys 2024) present RCACopilot for automated cloud incident root-cause analysis at Microsoft, matching incidents to diagnostic handlers and predicting root-cause categories with explanatory narratives.

Zhou et al. (PVLDB 2024) introduce D-Bot, an LLM-based database diagnosis system combining offline document knowledge extraction, tool-assisted metric/log retrieval, and tree-search root-cause analysis.

Staggered rollout of a GPT-based assistant to 5,179 support agents. Resolutions per hour rose about 15% on average and about 30% for novices; top performers saw no speed gain and a small quality decline. Customer sentiment improved and attrition fell.

Wei et al. (EuroSys 2023) introduce LogGrep, exploiting static template decomposition and runtime variable patterns for fast log storage and querying without full decompression.

Survey of 466 Apache/Eclipse/Mozilla developers and reporters: developers rank steps to reproduce, stack traces and test cases as most useful, while these are the items reporters find hardest to provide; the mismatch (not reporter effort) is the core quality gap. CUEZILLA scored report quality from text features.

Embedded in-process OLAP engine: vectorized execution over columnar vectors, MVCC, single-file storage. Per-operator timing reflects vector-at-a-time operators; host-language boundary (Python UDF, result conversion) is a first-class cost.

Fail-slow at scale (FAST'18): 101 incidents; slow hardware ranges from transient blips to permanent slowdowns, and stop-start/transient stalls are common. Frequency matters operationally because transient blips precede permanent faults, so counts of short stalls are a leading indicator, not an impact measure.

Gray failure = differential observability: the system's failure detector sees health while clients see failure. Directly matches the fsync POC: leader heartbeat does not touch Log::Sync, so a stalled leader keeps its lease (DB-22970); peers and YBA see a healthy node.

Postmortem study of ~150 high-severity Microsoft Teams incidents across detection, root-causing and mitigation, identifying process gaps (manual steps, cross-team dependencies) that delay response.

597 public outage reports from 32 cloud services (2009-2015): storage and upgrade/config faults are leading root causes, redundancy often fails to mask them, and many outages last hours. Context for treating cloud volume freezes as expected events that the database must tolerate.

Dremel introduced columnar striping of nested records via repetition and definition levels, the model Parquet adopted. For a profiler: nested/repeated columns carry level streams whose cost grows with nesting depth; SizeStatistics level histograms expose this.

Evaluates LLMs fine-tuned on 40k+ historical Microsoft incidents to recommend root cause and mitigation; useful as a draft judged by on-call engineers, weaker on novel incidents. For the gallery: AI-note hypotheses must be compared to the human CE resolution class, not to LLM self-scores.

Empirical study of configuration errors in a commercial storage vendor's customer support cases and open-source systems: misconfigurations are a large share (about a quarter) of customer-reported issues and over-represented among high-severity cases.

Four Java profilers disagreed on the hottest method due to yieldpoint sampling bias; random-timer sampling and causality checks (perturb a method, observe runtime) validate profiles. Python analog: in-process signal profilers sample only between bytecodes on the main thread; out-of-process samplers (py-spy, Austin, Tachyon) avoid this.

Fleet-wide profiling shows a datacenter tax (malloc, memmove, compression, serialization) spread thin across binaries. For Rust profiling: sum allocator and memcpy frames across the flat profile rather than only the top frame; allocator choice is a fleet-level NOTE.

Profiling-methodology: Top-Down hierarchy (Frontend/Bad Spec/Backend[Mem,Core]/Retiring); drill into dominant node; exposed as perf stat --topdown / -M TopdownL1 and toplev; better than raw IPC thresholds.

CHI 2024 human-LLM collaborative annotation: LLM labels plus self-explanations shown to verifiers raise accuracy when the LLM is right but induce over-reliance when it is wrong; recommends verification only where the LLM is uncertain. For audit-grade reference labels, humans should label blind to LLM output.

fsync T5-T7: Perseus (Alibaba, FAST'23) detects fail-slow drives from 15 s latency/throughput samples. A per-node polynomial regression of latency on throughput gives an adaptive upper bound (95%/99.9% prediction bounds); slowdown ratio SR = latency / upper bound;

Stabilizer re-randomizes code/stack/heap layout during execution so layout bias becomes averageable noise, enabling sound tests. For profiler agents: small deltas (a few percent) are within layout noise unless randomized or heavily repeated.

Jiang et al. (ISSTA 2024, Loghub-2.0) evaluate 15 log parsers across 14 systems totaling 50.4M annotated log lines and 3,488 ground-truth templates using GA, PA, FGA (Fine-grained Grouping Accuracy), and FTA (Fine-grained Template Accuracy), revealing three watershed findings: (1) when scaling from 2k to 50.4M lines, 9 of 15 parsers fail to finish…

Selection bias as conditioning on a common effect (collider). Bundles are uploaded because tickets are hard, and universes become visible because a ticket ships a bundle; first-observed universes (unknown source version) carry 32% adverse upgrade outcomes vs 13% for known-source paths, a collider pattern in the uppath card.

Coz virtual speedups answer which line, if faster, speeds the program end to end, for concurrent code where the hottest frame is not the bottleneck. In rust-profiler: use coz-rs progress points only when already in the code; Linux-only; complements samply/perf hotspot ranking.

Durner, Leis, and Neumann (PVLDB '23) establish the quantitative performance blueprint for analytical queries over Cloud Object Storage (S3/GCS): object stores exhibit high per-request TTFB latency (~15-50 ms per HTTP GET) and reach peak bandwidth only at request sizes >= 8-16 MiB across high concurrency.

GWP establishes low-overhead, always-on sampling (perf/pprof style) as the default for fleet servers. For YB tserver/master, the analogue is short, rate-limited perf record -F 99 windows plus tcmalloc sampled heap profiles (YB sets 1 MiB sampling period), aggregated across nodes rather than one-off long captures on a single node.

Parquet dictionary encoding + DELTA_BINARY_PACKED timestamps + BYTE_STREAM_SPLIT doubles + ZSTD on the coordinator provides SQL-native rate queries without requiring a custom Gorilla decoder.

Motivates yb-collector's deterministic node tiering and uniform per-node OS/HTTP collection so hagen-tools can run peer outlier queries (promdump outlier_node, sysstat disk_peers, sql_stats snapshot_coverage).

Synthetic control constructs a weighted combination of untreated units that matches the treated unit's pre-period path, with placebo-in-space inference. Placebo tests on other queues give an honest sense of whether the post-period TTR shift is unusual.

FastLanes file format (PVLDB 18): lightweight data-parallel (SIMD/GPU) cascaded encodings instead of general-purpose block compression; reports better ratio than Parquet with much faster decoding. ALP from the same group became a Parquet encoding (Preview, parquet-format 2.14).

PM seat (council 5): Error budgets need a policy and multiwindow burn-rate alerts; GF1 has a per-event bound and VOLUME.md proposes a 30-day budget, but the fs_asks card still adopts GF1/GF2 only as acceptance criteria with no budget policy.

Derives exact Bayes' rule adjustment of classifier posterior probabilities under shifted class priors in log-odds space. Provides the mathematical foundation for keeping base-rate priors out of the LLM prompt and applying prior-shift correction additively in log-odds space.

AutoFDO: sampling-based feedback-directed optimization from production profiles, with iterative and source stability concerns. Go PGO (default.pgo, -pgo=auto since Go 1.21) explicitly follows this AutoFDO-style loop (build, collect production CPU pprof, rebuild); Go docs cite 2-14% gains on Go 1.22 benchmarks.

Wild cluster bootstrap-t gives far better coverage than percentile/pairs cluster bootstrap with few clusters (5-30). Relevant: relrate cells rest on 11-17 orgs.

Yuan et al. (ICSE 2012) empirically study logging practices and evolution across four large open-source C/C++ systems (Apache, OpenSSH, PostgreSQL, Squid), finding that logging is pervasive (~1 log line per 30 LOC), undergoes high churn (18% of all committed patches modify log statements, with 39% of modifications altering static text or format…

CLP (OSDI 2021) parses text logs into schema-less dictionary/non-dictionary variables for compressed grep search.

Delta transaction log with per-file stats enables data skipping before Parquet row-group skipping; verify files scanned vs total.

Defines discourse patterns for observed behaviour (OB), expected behaviour (EB) and steps to reproduce (S2R) in bug descriptions and detects missing EB/S2R with pattern-based and ML detectors. Directly operationalises 'detect missing information at report time and ask for it'.

JOB benchmark on IMDB: all tested cardinality estimators (incl. PostgreSQL) routinely err by orders of magnitude on multi-join queries, errors growing with join count; cardinality errors dominate cost-model errors for plan quality; PostgreSQL non-index nested-loop joins under underestimates cause timeouts (disabling them removed all timeouts).

fsync T5-T7: IASO (Nutanix, ATC'19) detects fail-slow nodes from timeouts that PEERS observe: per peer pair, timeout/response ratio per 5 s epoch against a 10% threshold, additive score increase, decay only after minTTR = 2 min without timeouts (hysteresis), outlier detection over the N x (N-1) score table, one node quarantined at a time;

Beschastnikh et al. (ESEC/FSE 2011) infer temporal Ordering invariants (`a always precedes b`, `a never followed by b`) from system logs in Synoptic.

fsync T5: LLM parsing with an adaptive cache for template consistency; not needed for five source-known stall templates. Contrast to catalog-based parsing.

Until then the dashboard should show no quality level, only uncalibrated check trends, and every rate (e.g. the ready-to-close reopen proxy, 17.3% vs 14.9%, n=810/478, difference not significant) should carry an interval.

Twelve rules for reporting performance: report variability and CIs, use medians/nonparametric stats for skewed data, document the full environment, distinguish deterministic from noisy improvements. Basis for a shared report schema across profiler agents.

Flame graphs are the presentation layer for perf/eBPF stacks collected from yb-tserver and postgres backends; they are only as good as the stack walker: YB release binaries are LTO+PGO(+BOLT on x86) clang builds, so fp vs dwarf unwinding and BOLT-split .cold fragments change frame merging.

Dean and Barroso argue that in fan-out services, rare per-server latency hiccups dominate end-to-end latency: if 1 in 100 requests to one server is slow, a request that fans out to 100 servers is slow 63% of the time.

Upgrade failures in distributed systems are common and data-format driven. Council-5 DocDB lens: two 2025.2 field crash classes are upgrade/feature-promotion shaped (DB-24840 PITR sequences_data compaction after upgrade; DB-23719 AutoFlag promotion with in-flight transactions), so field crash rates should be split by days-since-upgrade.

Microsoft Azure study of high-severity production incidents: many come from timing, fault handling and inter-component data problems, and detection often relies on customers. Analogue for YB: field incidents surface via tickets, so ticket-derived rates are a lower bound.

RE'17 study at IBM: support analysts rely on customer context (history, account temperature) to anticipate escalations; formalised into features that predict Critical Situations. The Bundle Insights gallery has no customer-history features for CE prediction (prior CEs per org, recurrence), only bundle signatures;

Mines open-source JMH suites and finds common bad practices that bias results: dead-code elimination (results not returned or sunk into a Blackhole), constant folding of inputs (non-@State finals), loops inside @Benchmark methods, and setup at the wrong @Setup Level (Invocation). Measures the resulting distortion.

DocDB analogue: block/key layout and encoding changes, not more prefetches.

Change-induced incidents at a large online service provider: changes (code, config, deployment) are a leading incident trigger and linking incident to change under concurrent changes is the hard part.

Shows that language models suffer from majority-label, recency, and common-token biases that cause high variance across prompt formats, and corrects them via post-hoc affine calibration of output probabilities using content-free probes.

fsync T5-T7: Panorama turns interacting components into in-situ observers whose timeouts/errors are evidence about the observed component, reported with its identity. Peers' 'Fail or stepdown of leader <uuid> detected' lines are such observer reports and must be joined to the stalled node by its permanent uuid, not by time alone.

Revisits Mytkowicz PLDI'10 with modern samplers (async-profiler, Honest Profiler, JFR, YourKit, JProfiler, VisualVM) on DaCapo/Renaissance. Profilers are fairly self-consistent across ~30 runs and usually agree on the top method, but disagree beyond it and on per-method shares.

Aho and Corasick (CACM 1975) construct a finite-state string-matching automaton from a finite set of keyword needles by combining a prefix trie (goto function), suffix failure links (failure function), and output sets, finding all overlapping occurrences of all needles in a text of length n in a single O(n + m) pass independent of dictionary size.

CUPED reduces the variance of a treatment-effect estimate by regressing the outcome on a pre-period covariate (typically the same metric measured before exposure); variance falls by a factor of 1-rho^2, so the gain is entirely governed by how well pre-period data predicts the outcome.

Analyzes 600 bug reports from Mozilla and Eclipse to categorize developer follow-up questions across 8 information need categories, proving that when developers must ask follow-up questions for missing T0 context, issue resolution time roughly doubles and many questions go unanswered or delayed.

fsync T7: a limping component stalls a scale-out system because detectors see only fail-stop. Matches DB-22970 (the stalled leader keeps its lease); 'peer detected' and 'node lost leadership' columns measure that gap.

Le and Zhang (ICSE 2022) re-evaluate deep-learning log anomaly detectors under realistic conditions (chronological splits, parsing errors, class imbalance, unstable logs) and find reported performance largely does not hold up.

Morsel-driven scheduling with elastic parallelism. In DuckDB the morsel is effectively a row group (122,880 rows), so scans with fewer row groups than threads under-parallelize; check row groups vs threads.

Grounds what_the_log (wtl2 Wave-6e, 8d3b028) Schema v8 statistical RLE conservation law: because consecutive log events within 60s windows are folded into bursts with repetition_count >= 1 and 12-field args_summary moments/quantiles, downstream consumers (hagen-tools workload_signatures.py and doctor) must aggregate SUM(COALESCE(repetition_count, 1))…

Ken Thompson's classic 1968 CACM paper introducing linear-time O(m*n) regular expression matching by compiling a regular expression of length m into a non-deterministic finite automaton (NFA) and stepping the active state set lockstep across each input character in O(m) work per byte, avoiding exponential backtracking.

Protocol-aware recovery: storage faults in one replica's persisted Raft state (log, term, vote) must be repaired using the protocol, not by crashing or trusting local state. Council-5 DocDB lens: the 'Unexpected new leader in same term' FATAL family (GitHub #21808, volume remount under a running tserver;

Introduces DaCapo Chopin and argues Java performance analysis must report heap size (as a multiple of minimum heap), user-experienced latency (simple and metered), and total GC cost rather than pause time; demonstrates misleading conclusions from fixed-heap, single-metric evaluations.

Adaptive external hash join in DuckDB with runtime compression and unified memory mgmt across operators. Spill is graceful; measure peak temp dir size and temp_directory location; memory_limit and threads interact.

Of 198 user-reported failures in Cassandra/HBase/HDFS/MapReduce/Redis, most catastrophic ones came from incorrect handling of explicitly signalled non-fatal errors, and many reproduce on 3 nodes.

Ch.12 Effective Troubleshooting: triage, examine, diagnose, test/treat; ask what changed (recent changes are the most common cause), what, where, why; correlation is not causation; negative results are valuable. For support tooling: the case-open panel should answer what/where/when and what changed, and every empty answer must say whether data was absent.

fsync T5-T7: a drive is slow when its latency exceeds 2x the median of its RAID-group peers at the same time; disks were slow 0.2% and SSDs 0.6% of the time; 1.5%/2.2% of RAID groups had at least one slow drive.

Java microbenchmarks often never reach steady state; developer warmup settings are frequently wrong. For profiler agents: 'discard the first run' is not a warmup criterion; check iteration stability or report non-steady behavior.

Alert storms: many redundant low-value alerts at once; proposes clustering and representative-alert selection. For YBA the opposite failure also matters: on Comply 10-13 every alert raise failed to deliver (failedRaise) and no disk-latency check existed, so alert quality must include delivery and coverage, not only noise.

fsync T6: in a cloud block store the guest sees only I/O that does not complete; with AWS's recommended nvme_core.io_timeout maximum this explains silent kernel logs during EBS-class stalls. Host-log processors are low-yield for the acute class; CSP-side evidence (VolumeStalledIOCheck) and in-process YB logs carry the signal.

Randomized field experiment at Alibaba Taobao: 647 service workers, about 680k chats, August 2024. In treatment, agentic AI handled eligible chats under human supervision. Chat duration fell; retrial rates moved little; ratings on AI-eligible chats fell substantially.

ARES trains lightweight judges for context relevance, faithfulness and answer relevance on synthetic data, then uses a few hundred human annotations with prediction-powered inference to produce confidence intervals on system scores.

AutoMix: small model answers, a zero-shot self-consistency-based surrogate verifier decides when to escalate to a stronger model, giving better cost-quality tradeoffs than always-strong.

BOLT appends a second executable PT_LOAD segment and moves hot code, so symbolizers that assume one text segment (e.g. absl::Symbolize used by YB /pprof/heap_snapshot and yb_backend_heap_snapshot) can resolve frames ~4 KiB off on x86_64 BOLTed yb-server/postgres (GH yugabyte-db #35533).

BtrBlocks motivates cascading lightweight type-specific encodings before general compression; in ybc's Parquet writer on the coordinator, we apply the Parquet equivalents (dictionary on labels, DELTA_BINARY_PACKED on timestamps/offsets, BYTE_STREAM_SPLIT on metric doubles) ahead of ZSTD level 3.

Dehghani, Z. (2022). _Data Mesh: Delivering Data-Driven Value at Scale_. Sebastopol, CA: O'Reilly Media.

Supports yb-collector's multi-sample metrics, on-node sadf -x XML conversion, and vmstat UTC offset file so hagen-tools can align OS sar timelines with DB metrics in UTC.

Check config at startup so latent errors fail early. YBA analogue: warn at setup/health check when no default alert destination exists and when disk throughput exceeds the instance EBS baseline.

FrugalGPT's cascade beats single-model GPT-4 on their benchmark at up to 98% cost reduction on some tasks, with scoring-calibrated escalation — the canonical cost argument for a cheap first pass.

Gandalf (Microsoft Azure, NSDI 2020) uses fleet-wide telemetry for safe deployment. It correlates failure signals with rollouts in time and space and decides to stop or continue a rollout automatically. For a database vendor it is the peer pattern for the highest-value use of field data: release safety.

A 7B/8B generative verifier beating GPT-4 as a critic of a larger generator's candidate solutions is the direct template for h2o-lightning-4b scoring qwen38-flash-next's candidate vulnerability reports: small-model-as-scorer is a supported deployment, small-model-as-generator of exploit chains is not.

Facebook's Configerator/Gatekeeper treat config as code: review, automated validation, canaried staged rollout and fast rollback. Yardstick for YBA gflag edits, which apply universe-wide without a validator or canary; the gallery sees a gflag change in the observed week before 4.8% of bundled tickets but cannot say which changes were risky.

Establishes the principle of literal-substring prefiltering (extracting mandatory literal tokens from regular expressions and scanning buffers via SIMD/fast substring search before invoking NFA/DFA automata).

Gartner (2025-2026). _Lack of AI-Ready Data Puts AI Projects at Risk (60% Abandonment Forecast Through 2026) & Gartner Data and Analytics Summit 2026 Findings on Governed Semantic Layers for AI Agents_. Stamford, CT: Gartner, Inc.

Lance v2 paper (arXiv 2504.15247): structural encoding (repetition/validity) dominates random-access cost on NVMe; reports correctly configured Parquet (small pages, page index, no/fast compression) can reach >60x better random access than defaults at the cost of scan speed and RAM.

LogJack demonstrates that attacker-controlled strings in application/database logs can hijack LLM debugging agents via indirect prompt injection.

Proves that even when LLMs accept 100k+ token inputs, retrieval and multi-hop reasoning degrade sharply in a U-shaped curve when relevant facts sit in the middle of long contexts (>32k tokens).

Demonstrates that compact specialized fact-checking models verifying atomic claims against grounding documents match or exceed frontier LLMs at a fraction of the cost.

Multitask principal-agent model. When an agent spreads effort across several tasks and only some tasks are measurable, strong incentives on the measured tasks pull effort away from the unmeasured ones. The optimal response is often weaker incentives, or a job design that groups tasks so that they are equally measurable.

fsync T7: Xinda (NSDI'25) finds slow-fault impact non-monotonic in magnitude (a small slow fault can hurt more than a large one; slow followers can cost about 1.5x more than slow leaders) and ships the ADR runtime detector.

fsync T7: PELT considered for episode boundaries and rejected: stall lines are sparse events carrying their own duration, not a sampled series; interval union on [ts - real, ts] is exact and auditable, while changepoints need a sampled signal and a penalty.

Reference-free RAG metrics computed by an LLM: faithfulness (share of answer claims supported by retrieved context), answer relevance, and context relevance; later library versions add context precision/recall.

Provides RCAEval (Zenodo 14590730, 5.16 GB compressed, CC-BY-4.0), comprising 735 fault-injection cases across Online Boutique, Sock Shop, and Train Ticket in three suites (RE1, RE2, RE3) with synchronized multi-service logs, metrics, and traces.

Injecting single-node file-system faults (corruption, read/write errors) into 8 distributed stores shows redundancy rarely saves them: local faults propagate, crash nodes or lose data. Supports a YB DST campaign that injects WAL fsync delay and EIO per node, not only process kills.

Self-consistency gains come from *diverse* sampled reasoning paths; its benefit shrinks when samples share biases.

Service loyalty is driven more by reducing customer effort (repeat contacts, channel switching, re-explaining) than by delight. For B2B support dashboards: the operational proxies of effort are reopen/re-contact rate, number of customer replies to resolution, and handoffs/reassignments; a CSAT tile with n of a few dozen per year is not a substitute.

arXiv 2406.04291 carries the shorter title ``Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation''

Consistency-aware durability lets replicas acknowledge before fsync while reads stay strong. Council-5 DocDB lens: YugabyteDB's default buffered WAL (async sync at 0.6 s, forced at 1 s) is a related latency-for-durability trade;

Grounds yb-collector's P0 priority placement of server.conf, varz.json, cmdline.txt, certs.json, and OS sysctl/ulimit/cgroup configuration.

Change-before-ticket rates need a referent window from the same unit. A case-crossover design compares the hazard window (7 days before a ticket) with control windows from the same universe, giving an odds ratio rather than a bare share. Precondition: the exposure must be observable in control windows.

Pang, Allman, Paxson, and Lee (SIGCOMM CCR 2006) detail the operational lessons of packet and log trace anonymization: multi-layer scrubbing must reconcile syntax-aware field parsing with defense-in-depth scrubbing, verify that truncation or malformed records do not leak unredacted suffixes, and emit explicit audit metadata describing what was transformed.

ESEC/FSE 2018 ecosystem-scale empirical study of Regular Expression Denial of Service (ReDoS) across npm and PyPI, demonstrating that backtracking regex engines suffer catastrophic super-linear CPU stalls in production and motivating linear-time automata engines (RE2, Go regexp, Rust regex) in production telemetry agents.

TRIPOD-LLM: reporting checklist for studies of LLMs incl. evaluation-only studies: model identity/version, prompts, leakage/contamination, human evaluation protocol and agreement, LLM-as-evaluator validation.

Demonstrates that numeric variables dominate the post-parsing entropy of system logs and introduces numeric token parsing and delta/elastic coding for lossless log compression (26.2 MB/s).

Demonstrates lightweight formal specifications in Rust paired with property-based conformance checking to validate Amazon S3's ShardStore key-value storage node (SOSP 2021). Directly parallels Report 08's architecture of checking production storage/log-engine implementations against executable reference models.

LLM-as-judge criteria drift once graders see outputs; evaluators must be aligned to a small human-labelled set and re-checked as criteria evolve.

ACORN: predicate-agnostic filtered HNSW traversing only predicate-passing nodes with expanded neighbor lists. pgvector instead post-filters plus iterative scans; low-selectivity filters remain a recall/latency risk to measure per selectivity.

Acto tests cloud-system operators end to end by generating state transitions and checking operation correctness, finding many bugs in popular operators. A method for YBA operator/helm QA that field data could seed with the real upgrade paths and storage classes seen in bundles.

Shows the Linux block layer enforces ordering and durability with Transfer-and-Flush (wait for DMA completion, then flush the device cache), which dominates fsync cost on flash; a barrier command removes it.

Measures AWS EBS gp2/gp3/io1/io2: latency stays low until I/O pressure approaches the provisioned IOPS/throughput 'contract', then varies sharply; an LSM that rate-limits itself inside the contract avoids spikes (P99 <550 us). Relevance: throttling at a provisioned or instance cap turns into queueing latency for every I/O on the volume, including WAL fsync.

Chaos engineering principles: define steady state, hypothesise it holds, inject real-world faults, try to disprove. Adapted to a data lake: steady state is a card headline and its CI; faults are duplicates, dropped partitions, null keys, schema drift and truncation on copies of pinned inputs.

CockroachDB diagnostics reporting sends anonymised cluster and feature-usage counters by default (opt-out) and documents the payload. The installed-base feature frame that bundle-based feature cards lack.

Proposes LBO (lower-bound overhead): compare total CPU/wall cost of a production GC against an idealised baseline derived from the cheapest collector per benchmark, capturing barrier, allocation and mutator-locality costs that pause-time metrics miss. Finds concurrent collectors (ZGC, Shenandoah) carry substantial total-cost overhead at tight heaps.

Alibaba EBS evolution (TOS extension of the FAST'24 EBS paper): average latency is dominated by hardware/network, tail by software processing and background-task contention on block servers; availability events are localised by correlating many VMs. Guest-side the only symptom is I/O that does not complete.

Linked bug-fix samples are biased (by severity, experience); fixtest finds only 37% (DB 25%) of field-linked issues in commit messages, so its shares describe the linked subset.

Large share of config parameters in mature systems are rarely or never set by users, and a minority of knobs carry most of the misconfiguration risk. For the Bundle Insights gflag cards this argues for ranking flags by hazard and deviation from default, not by how many orgs set them; the gfrisk card is the right direction but 'any set' conditions (e.g.

Shows the Wald interval for a binomial proportion has erratic, often badly low coverage even at moderate n and especially near 0 or 1, and recommends Wilson or Jeffreys for n<40 and Agresti-Coull for larger n. Applied to support SLA tiles: per-customer P1 attainment (n=1-10 pages) and BFCM event windows need Wilson/Jeffreys bounds;

Kapoor et al. (NeurIPS 2024): prompting alone (verbalized or zero-shot logit) gives poor uncertainty estimates; fine-tuning a small LoRA/probe head on ~1,000 graded examples yields well-calibrated, generalizing uncertainty, cheaply.

Deployed enterprise customer-support BPM system that uses copilot accept/correct feedback and UI traces to train a policy plus an abstention critic. It auto-executes only high-confidence steps and hands uncertain ones to operators. Reported 45% of sessions automated and 39% lower average handling time without degraded quality (authors' claim).

Introduces PROBE-SWE (FSE 2026) to test 8 prompt-induced cognitive biases (including anchoring and overconfidence) in SE decision support. Shows standard CoT and self-debiasing prompts fail to significantly reduce bias sensitivity, whereas axiomatic reasoning cues reduce bias sensitivity by 51%.

Introduces NSVIF (Su, Xu, Gao, Yang, Li, Yang, & Xu, 2026), a neuro-symbolic framework for verifying LLM instruction following by compiling constraints into deterministic logic checks paired with semantic predicates.

Introduces PAVE and OpenRCA 2.0 (500 instances across 3 microservice systems and 27 fault types). Across 11 frontier LLMs, exact root-cause recovery averages 20.7%, and although agents name at least one correct service in 76.0% of cases (AnySvc), only 61.5% satisfy Path Reachability (PR), exposing a 14.5 pp ungrounded diagnosis gap.

PARDA (VMware): hosts sharing a SAN array adapt their queue depth to observed latency to get proportional shares, the basis of vSphere Storage I/O Control. Implies that on a shared SAN, latency is a fabric-wide property: one host's burst raises every host's fsync latency, matching the Softbank 20107 all-hosts w_await pattern.

Trains lightweight preference-calibrated routers (matrix factorization, BERT, causal LLM) between strong and weak models, achieving 85% cost reduction on MT-Bench at 95% GPT-4 quality. In our local Security-1/Security-2 pipeline, unclamped 2-option choice routing (tau=0.28) with symmetrized pairwise runoff serves as a domain-specific epistemic router.

Communications of the ACM (2021) synthesis of how Rust resolves the longstanding systems programming trade-off between safety (historically requiring GC) and low-level resource control (zero-cost abstractions, deterministic drop, no runtime GC pauses).

Proves that scaling test-time compute (via verifier-guided beam/lookahead search or iterative revision) can outperform a 14x larger model at matched FLOPs on hard tasks.

Broadwell, Harren, and Sastry (USENIX Security 2003) demonstrate in Scrash that raw core dumps indiscriminately mix sensitive application heap data (passwords, keys, customer rows) with non-sensitive execution state (call stacks, symbol tables, control registers), and show how to strip sensitive memory regions prior to crash transmission.

Agent-computer interface quality dominates raw model choice — a good interface lets a mid-size model outperform larger ones on the same task.

Lozi et al. show that four bugs in CFS load balancing (group imbalance, scheduling-group construction, overload-on-wakeup, missing domains) leave cores idle while runnable threads wait. They found them with a sanity checker for the invariant 'no idle core while threads wait' and with heatmap visualizations.

Gregg's ACM Queue article contrasts anti-methods (streetlight, random change, blame-someone-else) with methods. The main one is the USE method: for every resource, check utilization, saturation and errors, using a resource list built from a functional block diagram (CPUs, memory, network interfaces, storage devices, controllers, interconnects).

Weak supervisor (GPT-2-class) finetuning a much stronger model only partially elicits its capability (logistic-regression-style gap): the weak model's labels define a ceiling on the strong model's behavior if it is trained/selected on them.

tau-bench: simulated user plus tool-using agent under domain policy; success judged on final state, reliability by pass^k (all k trials succeed). GPT-4o under 50% pass^1 and under 25% pass^8 in retail.