▤The Test Group experiment running Open the partner account
paid
Affiliate disclosure. The partner link in the masthead and in the band beside the copy on this page is a sponsored link to a partner operator, and this site may be paid if you open an account through it, at no extra cost to you. It carries rel="sponsored noopener" and opens in a new tab. A desk about how a screen is measured and tested should not leave its own funding unsaid: one link funds the site, no operator and no product is named, rated or recommended anywhere on it, and this site runs no analytics of its own on its readers.
The Test Group / The record
The log of experiments nobody outside the operator can read

The record a reader can never see

Every experiment produces a record: what was changed, what was measured, how big the sample was, what the result was and what was decided. It is the most informative document a gambling operator holds about its own interface, and on the samples 3 of 40 records mentioned a guardrail and none was published.

Desk spec
records kept
40
naming a guardrail
3
published
0
naming the metric
31
the eventOne interaction, written down. A click carries a name, a time, an account, a session and what was on the screen, and it is kept whether or not the reader chose to be measured.
the funnelThe order the steps happen in. A landing visit becomes an account, an account becomes a deposit page, and 100,000 visits end in 2,074 first bets - a 2.1% path the whole loop is aimed at.
the armsTwo versions of one screen shown at the same time, and a rate for each. 4.30% against 5.16% is a 0.86-point lift, and the interval decides whether it is a result or a coincidence.
Direct answer

An experiment record is the note an operator keeps about its own test: the hypothesis, the metric, the sample, the result and the ship decision. On the samples 40 records were kept, 31 named the metric, 3 named a guardrail and none was published anywhere a reader could find it - which is why the interface can change without explanation.

What a record contains

The record is written before the test and closed after it, and its value is that the two halves are in the same document. Without the pre-registered half, a result can be chosen afterwards from whichever number moved.

Sample I - what the 40 records contain, field by field
FieldRecords with itWhat it fixes
what changed40 of 40the one thing the test is about
the primary metric31 of 40the number the ship decision reads
the sample required24 of 40how long the test runs before it is read
a guardrail and its threshold3 of 40the number that can stop a ship
the decision and its reason36 of 40why the result was acted on
a re-check date6 of 40whether the effect survived
published anywhere0 of 40-
the useful fields3 of 40the guardrail is the rarest and the most important
sample I - how much of a decision is on the record records that fix the metric before the test = 31 / 40 = 77.5% records that fix the sample before the test = 24 / 40 = 60.0% records that can stop a ship at all = 3 / 40 = 7.5% records that will be re-checked = 6 / 40 = 15.0% so 92.5% of decisions have no threshold that could have stopped them, which is the same statement as: the direction of every change is set before any of them is measured.

Why none of it is published

The record is commercially sensitive in the ordinary sense, and that is a real reason. It is also the document that would show a reader that a screen was changed to move them, which is a reason no operator publicises it. Both can be true at once, and the samples assume both are.

Sample I - who could see which part of the record
Reader of the recordWhat they seeOn the samples
the team that ran the testthe whole recordyes, by definition
the operator's compliance functionthe record on requestsometimes, and not routinely
the regulatorthe record if it asksrarely asked, at this level of detail
an auditor of the licence conditionsthe guardrail, if it existsin principle, where a condition names it
the readernothing0 of 40 published
the gapthe whole documentthe only party who cannot read it is the one it was tested on
sample I - the substitution a reader can make the record says: which step, which metric, which sample, which guardrail a reader can see: whether the step changed at all, and when a reader can hold: their own figures, taken before and after e.g. a deposit at 40.00 on 3 March and the same deposit taking two more screens on 17 March that is not the metric, and it is a fact the reader owns against 0 of 40 records published and 3 of 40 guardrails named, a dated note of what the interface did is the only comparable the reader will ever have.
The counts are invented. Whether a named operator keeps a record like this, and whether any part of it is available to a reader, is an internal fact; the desk's point is that the loop produces a document, that the document is what makes the loop accountable, and that its absence is the ordinary case rather than the exception.
How a reader can build their own comparable
  • Note the date whenever the deposit or withdrawal flow looks different.
  • Keep your own statement lines, because they are the operator's record and not a dashboard.
  • Ask in writing which metric the deposit page is tested against, and keep the reply.
  • Ask whether a session replay of your own account exists, and for how long it is kept.
  • Take a screenshot of a changed step when you notice it; the date is the part that cannot be recovered later.

Read next

The limitsWhat the loop cannot fix even with a full recordThe holdoutThe only internal check on the seriesThe metricThe field the record most often omitsThe consent lineWho was measured in the first place