E ISSN: 2583-049X
logo

International Journal of Advanced Multidisciplinary Research and Studies

Volume 6, Issue 6, 2026

Evidence-Constrained Performance Evaluation of GPU-as-a-Service Platforms for AI Workloads: A Cross-Release Analysis of MLPerf Inference Benchmarks



Author(s): Sahal Ilyas Dadabhoy, Sameer Abbasi

Abstract:

The increasing use of GPU-as-a-Service (GPUaaS) platforms for AI inference has intensified the need for reliable performance evaluation. However, identical GPU models and accelerator counts do not necessarily establish whole-system comparability, potentially limiting the interpretation of published benchmark rankings. This study proposes an evidence-constrained framework to assess the technical comparability and ranking robustness of cloud-associated MLPerf Inference submissions. The dataset for the primary analysis was MLPerf v6.1 and the data set for cross-release methodological replications was MLPerf v6.0. Published throughput results have been matched based on workload, inference scenario, performance units, GPU model, and the number of accelerators. Candidate cross-provider comparisons were subsequently evaluated against eight reported hardware and software metadata fields. Progressive ablation, hypothetical ranking perturbations, scenario comparisons, and submitter-selection sensitivity analyses were performed. In v6.1, 77 eligible throughput records yielded 18 comparison groups and 34 candidate provider pairs. Only four pairs (11.8%), all involving Nebius–Oracle 72-GPU GB300 configurations, satisfied the eight-field matching criterion, with reported throughput differences ranging from 2.73% to 8.12%. Under an assumed ±1% adverse score perturbation, 13 of 18 initial group leaders retained their rankings. An external replication in v6.0 found 36 eligible records, and 4 candidate pairs, with no records that matched well enough for the absolute matching requirement. The findings illustrate that there is a significant practical compromise between the level of technical comparability and level of comparison coverage. The proposed framework will allow clear, evidence-based reading of published performance rankings for GPUs.


Keywords: GPU-as-a-Service, MLPerf Inference, GPU Cloud Computing, AI Inference, Performance Benchmarking, Technical Comparability

Pages: 719-731

Download Full Article: Click Here