Skip to content
Back to Academia

Test: from an ad click to a paid order

18.04Assessment and testing
On this page

The contractor sends a report: a click costs 100 ₸, and app installs are cheap too. The owner wants to raise the budget without knowing how much money is left after the orders. In that meeting, your job is to carry the calculation through to the purchase and say where the data you have runs out.

The numbers below are illustrative. The shop and the app are two separate problems with their own conditions. Write down your calculations before reading the walkthrough: this is a written self-check, and the site does not score it.

Problem 1: the shop's cost per customer

A campaign got 500 clicks at 100 ₸ each. 2% of clicks became leads, and 10% of leads became new paying customers. In this problem, each person submitted at most one lead and made one purchase, and the agreed window for processing leads has closed.

An order is worth 50,000 ₸. After variable costs for the goods and fulfilment, 30% of revenue remains, before advertising. The problem has no other costs.

Calculate ad spend, the number of leads and customers, the ad-only cost per new customer and the first order's contribution after advertising. Then write one sentence to the owner about the result. And name the data missing for judging the business over a longer period.

Problem 2: an install or a useful action

A different project compares two mobile campaigns over the same period. Every user has had seven full days since installing. The team calls the first useful action activation, defined the same way for both groups; a person repeating it does not add another activated user.

MetricCampaign ACampaign B
Ad spend100,000 ₸150,000 ₸
Installs1,000500
Users activated within 7 days100150

Calculate CPI, the activation rate and the cost per activated user. The manager proposes keeping only B because its activation rate is higher. Is this data enough for that decision? Name the result you would ask the app team for next.

Problem 3: a new creative increased CTR

CTR rose after an ad was replaced. Someone already wants to write "The new creative brings in more profit" in the report, although nobody has reconciled spend, payments and refunds.

Describe what you actually observed, which data a conclusion about money needs, and how to separate the creative's effect from other changes. Your answer should include the next check an analyst or media buyer can run.

Shop walkthrough: go through both steps

The clicks cost 500 × 100 = 50,000 ₸. They produced 500 × 2% = 10 leads. Of those, 10 × 10% = 1 person paid. The ad-only cost per new customer is 50,000 / 1 = 50,000 ₸.

The calculation also contains 5,000 ₸: that is 50,000 / 10, the cost per lead. Label it CAC and the nine people who left a lead and never paid vanish from the report. Every metric has its own denominator.

The first order's contribution before advertising is 50,000 × 30% = 15,000 ₸. After advertising, that leaves 15,000 - 50,000 = -35,000 ₸. Variable costs were already deducted when contribution was calculated, so do not subtract the cost of the goods from that 15,000 ₸ a second time.

From clicks to the first paid purchase

Processing window complete; at most one lead and purchase per person

  1. 500 clicks

    At 100 ₸ each: 50,000 ₸ spent

  2. 10 leads

    500 × 2%

  3. 1 new customer

    10 × 10%

    Advertising per customer: 50,000 ₸

First-order contribution 15,000 ₸ minus advertising 50,000 ₸ = -35,000 ₸. Variable costs are already included in contribution.

500 clicks become 10 leads and one customer; the first order's 15,000 ₸ contribution does not cover 50,000 ₸ advertising.

You could write to the owner: "The first order contributed 15,000 ₸ against 50,000 ₸ of ad spend, so the ads did not pay back over this horizon." That is the first order's result after advertising. Full CAC would also include other agreed acquisition costs the problem does not give, and the company's financial result includes the remaining expense lines.

A longer horizon needs repeat purchases from this same group, their contribution, refunds, retention costs and when the money arrives. A future purchase does not go into actual results just because someone plans to send the customer a reminder. If it is still too early to expect repeat purchases, show the current result and, separately, a scenario with its assumptions stated plainly.

App walkthrough: the same cost per activation

An install costs 100,000 / 1,000 = 100 ₸ in A and 150,000 / 500 = 300 ₸ in B. The activation rates are 100 / 1,000 = 10% and 150 / 500 = 30%.

MetricCampaign ACampaign B
CPI100 ₸300 ₸
Activation within 7 days10%30%
Cost per activated user1,000 ₸1,000 ₸

B's install is three times as expensive, but its activation rate is three times higher. Per activated user the cost is the same: 100,000 / 100 and 150,000 / 150 both give 1,000 ₸. Choose on CPI alone and you miss the quality of the next step; choose on activation rate alone and you miss the spend. So the data is not enough for the manager's decision: on cost per activation, the campaigns are equal.

Next, ask for purchases and contribution from these same groups over the same completed window. Activation may sit close to a purchase, but first make sure the action is tied to why the person installs the app. You also need the same rules for crediting results to a campaign, and comparable audiences.

Even if one group turns out to have better economics, moving the whole budget needs its own justification. Extra money may bring in a different audience, and the next customer may cost more. The report can reasonably propose a limited budget increase with conditions for judging it, instead of promising the same cost at any volume.

Creative walkthrough: separate the observation from the cause

CTR rose: in this data, clicks make up a larger share of impressions than before. To talk about money, you need spend, new customers, paid orders, variable costs and refunds over an agreed period. Those records must belong to one comparable group. A week of clicks and a month of all the shop's sales do not go into one calculation.

A before-and-after comparison leaves other explanations open: the audience, bids, the offer, timing. The analyst can first compare the settings and the make-up of the groups, then propose an experiment that isolates the creative change. Plan the data volume around an effect large enough to make the team change its decision. The single customer from the first problem lets you calculate the money already spent exactly, but it is far too little for a confident forecast of the next campaign.

Exercise: a calculation for your own report

Take one acquisition source in your project and an agreed period. Build a row with spend, leads, new customers and the contribution of their first orders. Note which costs went in, how you removed duplicates and how long you waited for payment. If data is missing, leave the cell empty and write down who will provide the export.

As a check, change one shop condition: 20% of the ten leads now pay, and every other number stays the same. Recalculate the result and state what you can now tell the owner.

Two people paid. The ad-only cost per customer fell to 50,000 / 2 = 25,000 ₸, and total contribution before advertising rose to 2 × 15,000 = 30,000 ₸. After advertising, -20,000 ₸ remains. That is 15,000 ₸ better, but the first orders still did not cover the spend, and that is exactly what to tell the owner.

The write-up for your own project should show the same path from source records to the conclusion. If payments are still maturing or contribution is unknown, leave that spot open and say so. The reader should see which calculation is finished and what information the next decision is missing.

Need help applying this to your analytics?