Skip to content
Back to Academia

Test: analytics and the tech stack

18.02Assessment and testing
On this page

A manager sees a new lead in the CRM, a developer shows a record in the dataLayer, but analytics has no event. The team already wants to move sending to a server. Hold on: first find where the current path broke. Otherwise the same error simply moves into the new stack.

Write down your decisions for the three exercises and how you would confirm each one, then compare them with the walkthroughs. The site does not grade these notes. The data are illustrative; lead_sent and lead_submit were invented for this exercise, and the Start track uses different event names.

Situation 1: the event appeared, the request did not leave

After a successful form submission, the site records lead_sent in the dataLayer. The required tag is configured for lead_submit. In debug mode the event is visible, the tag does not fire, and no network request goes to the destination.

Choose an action and explain what you expect to see after it:

  • A: wait for the data to appear in the report;
  • B: agree on the event name, fix the mismatch and submit the form again;
  • C: immediately move sending to a server.

The event exists; no request was sent

  1. dataLayer

    The form records lead_sent

  2. Tag condition

    Configured for lead_submit

    Names differ; the tag did not fire

  3. Request to the destination

    No network request

    Check the agreed name with the developer

The form records lead_sent, but the tag waits for lead_submit. The names differ, so the tag never fires and no request goes out.

Situation 2: the ad report and CRM disagree

The ad account shows 40 conversions, while the CRM shows 25 paid orders for the same calendar period. The owner concludes that the system lost 15 sales. But the ad account counts a form submission as a conversion, and the CRM counts payments.

What can you say straight away? Which definitions, dates and records will you need before looking for lost data? Write a reconciliation sequence that does not start by trying to force the two totals to match.

Situation 3: revenue grew after a join

In the orders table, order 101 has a total of 10,000 ₸. The line-item table has two products for that order. Joining the tables on order ID produces two rows, and each repeats the full 10,000 ₸ total. The report sums that field and shows 20,000 ₸.

Explain what happened to the row and which result is needed. Then add order 102: it also totals 10,000 ₸ but has one line item. How many rows will the join produce, what will the old calculation return, and how much revenue was in the original orders? Would summing only distinct monetary values help?

Walkthrough 1: fix the unconfirmed transition

The right option is B. The GTM data layer passes data and events that tags respond to. The lead_sent record confirms only that step. For data to go further, the trigger condition has to be met and the right tag has to fire.

Why the tag is silent is already known: the name in the condition does not match the event name. Ask the developer which name was agreed. If the site correctly sends the agreed lead_sent, fix the trigger. If the agreement was lead_submit, the site code needs fixing. Renaming the event at random in one place is risky: other tags and reports may rely on the old name.

After the fix, submit the form again. See whether the tag fired, whether a request went to the right destination, and whether the parameters match what you entered in the form. Then confirm the destination received the event and that it appeared in the report after processing. Each step answers its own question: "tag fired" does not tell you whether the address and the data are right or whether the event reached the report.

Option A misses the main point: there is no request, and waiting will not create one. Option C rebuilds the system although the break has already been found in the firing condition. Server-side sending solves other problems, and the name fix can be tested in the current setup without rebuilding anything.

Add a reverse check: a form with an error must not produce a successful-lead event, and reopening the thank-you page must not turn one lead into two. If you find such cases, fix the cause before publishing. Otherwise the event count will drift from what you agreed to call a lead.

Walkthrough 2: agree on what is being counted

The arithmetic 40 - 25 = 15 is correct, but "15 lost sales" does not follow from it. A form and a paid order are different stages. One person may have submitted several forms, some people are still deciding, and some paid the following month.

For each report, write down the event, the source, the rules for crediting a result to advertising, the date, the time zone and how the system handles duplicates. Find out which time the period is built on: form submission, ad interaction or payment. Then choose a comparable definition and a window long enough for leads to reach payment. Take the window length from the project's sales process: if managers close a deal in two weeks, a one-week window will undercount payments.

To check the link, you need lead and order IDs you are allowed to use. Trace several records from the form to the CRM: was a lead created, did a payment follow, was it cancelled, which period did the record land in? Do not copy names, phone numbers or other customer data the reconciliation does not need into the shared write-up.

If one payment has no linked lead, that is a separate discrepancy, with a place and a cause to find. Do not automatically file the rest of the difference under the same error. Reconciliation is there to explain what the difference is made of. Afterwards the reports may still show different totals if they keep counting different things.

Walkthrough 3: return the calculation to order level

In the original table, one row was an order. After the join, one row is a line item. The full order total repeated on every line item, and summing it directly multiplied the money by the number of rows.

OrderOrder totalLine itemsContribution to the incorrect sum
10110,000 ₸220,000 ₸
10210,000 ₸110,000 ₸
Total20,000 ₸330,000 ₸

Order 101 still brought in 10,000 ₸. The correct total for both orders is 10,000 + 10,000 = 20,000 ₸, while the old method gives 2 × 10,000 + 1 × 10,000 = 30,000 ₸. The join created no new payments.

Count revenue from one confirmed record per unique order. There is another route: if line items carry their own amounts and you know how discounts, delivery and returns are handled, sum the items and reconcile the result with the order total. But a full order total copied into every row does not become a line-item amount.

SUM(DISTINCT amount) will not save you. In the example, different orders have the same amount, so removing the repeated value leaves 10,000 ₸ instead of 20,000 ₸. Orders need to be told apart by ID, not by amount. This is worth checking when an assistant writes the query too: when analysing tables with AI, a syntactically valid query may count the wrong row unit.

Exercise on your own project

Choose one event and one report from your project. For the event, write down when it should be sent, its name, required parameters and destination. For the report, define what one row means and which business action goes into the total. At each stage, keep an observation for one test record you are allowed to use.

If you have no access to a client site, practise the event and tag in the training sandbox with the names from this exercise. Then calculate the two orders above without hints and explain why the ID has to survive the join.

Your finished write-up should let someone else walk the same route. "Analytics is broken" gives the person doing the fix nothing, while "the event exists, the right tag did not fire, the name in the condition differs" says exactly what to fix. For the report, attach the original and resulting totals and explain where the row count came from. After the fix, repeat the same example, including the case where no event should be sent. Close the task on a result you saw yourself, and mark any stage you could not check as unchecked.

Need help applying this to your analytics?