Test the same buyer-priority titles and visitor conditions wherever the suppliers overlap, then test each supplier’s unique coverage separately. A comparison of unrelated games under different conditions cannot isolate the delivery-service difference.
First obtain permission for the proposed testing. GameDistribution’s standard agreement expressly requires prior written consent for monitoring availability, performance or functionality and for benchmarking. Do not assume that ordinary publishing access authorises automated research. (static.gamedistribution.com)
What belongs in the test matrix?
This is a proposed acceptance framework, not a report of tests already completed:
| Dimension | Minimum record | Why it matters to the decision |
|---|---|---|
| Game identity | Supplier ID, canonical ID, configuration and endpoint | Confirms a like-for-like comparison |
| Device/browser | Real device model, OS and browser version | Makes findings reproducible |
| Location | Authorised test location and timestamp | Bounds the availability claim |
| Network/cache | Connection conditions; cold or warm state | Separates repeat-load behaviour from first use |
| Privacy state | Applicable consent choice, opt-out signal and storage settings | Tests the intended visitor experience |
| Authentication | Fresh, expired and invalid launch credentials where permitted | Tests safe failure and recovery |
| Readiness | Defined start event, ready event and timeout | Separates wrapper loading from playable loading |
| Gameplay | First permitted interaction, orientation and return from background | Tests beyond initial display |
| Denied access | Message, fallback and outbound behaviour | Prevents blank or misleading pages |
| Support | Ticket, acknowledgement, substantive response and resolution | Measures the escalation process |
Which metrics are decision-useful?
Use these proposed definitions consistently:
- Launch success: accepted launches divided by all attempted launches, with failure categories shown.
- Time to interactable game: elapsed time from the agreed start event to verified readiness.
- Timeout rate: attempts exceeding the agreed limit divided by all attempts.
- Configuration accuracy: reviewed records matching the evidenced game configuration.
- Coverage fulfilment: accepted must-have titles divided by the buyer’s must-have list.
- Operational effort: recorded minutes spent onboarding, correcting, diagnosing and escalating.
Report sample size, dates and environments beside every result. Show timeouts separately rather than quietly excluding them from success claims. Use percentiles only with an adequate and disclosed sample; a few launches should be presented as observations, not a dependable service-level estimate.
An HTTP success response, platform uptime claim and successful game session are different measures. Require the supplier’s proposed SLA to define its measurement point, exclusions and remedies, then compare those definitions with the buyer’s acceptance criteria.