AI in marketing: what a finished result really costs

The content team at an online appliance store in Almaty used to write a product page in an hour, and the editor, Lyazzat, spent another twenty minutes checking it. With AI, the draft appears in ten minutes. But now Lyazzat spends fifty minutes checking it against the manufacturer's specifications and fifteen on corrections. The rollout deck said "Six times faster". The store owner asked: faster at what - the draft, or a page you can put on the site?

The answer is more modest than the deck. At the same quality, the full cycle went from 80 to 75 minutes, a saving of five minutes, or 6.25%. The numbers are made up, and we are not evaluating any particular model. A rollout is judged on the whole path: from the model's first answer to an accepted page.

When you measure this yourself, log time by stage and include the failed attempts: a draft that had to be rewritten or thrown away still ate minutes.

Pick work that someone signs off

"Introduce AI" is a task with no end. A product page has one: Lyazzat received the text in the right format, checked the specifications against the source and approved it for publication. Reviewing customer enquiries has a different sign-off: every theme leads back to source records, the count can be reproduced, and an unknown cause is not passed off as fact.

For your own task, write down the input, the output, how often it happens, who does it and who accepts it. Then list the errors that send the result back. A nice headline will not save a wrong fridge price. A page that leaves out an important product limitation can be short and clear and still unusable.

For a first pilot, choose narrow work where the reviewer can open the source. If the manual process changes every week and nobody has agreed what counts as acceptable quality, you have nothing to compare against. Agree on the result first, then make it faster.

Map the current process

Write down the path from assignment to accepted version: gathering facts, drafting, review, corrections, handover. Put time spent waiting in a queue on one line and working time on another. Two hours on the calendar can turn out to be ten minutes of work and an hour and fifty minutes of waiting. Delivery gets shorter when you cut the waiting, and workload is counted in minutes of work.

Measure both ordinary pieces and hard ones: a missing condition, conflicting sources, two languages, a poor style example. If the manual work got the difficult pages and the AI got the easy ones, you compared pages, not ways of working.

Do not throw away the old method. You need it for the comparison and as a fallback. In a move to a new process you decide up front what changes and when the old mandatory work can stop. Add a generator without changing the review, and Lyazzat simply gets a second queue.

Keep tuning apart from evaluation

On one set of pages the team refines the instructions: which facts to use, what format to return, how to flag missing information. A second set is put aside for evaluation, and the instructions are not adjusted to it. If you tweak the request after every answer on the evaluation set and then call it an independent test, you have tested your own tweaking.

The assessor is better off not knowing how a text was produced, so they are not harder on one method. They use the same criteria for both, note serious errors and the time it took to bring the text to acceptance. Run some inputs twice: one good answer does not mean the next one will be as good.

The NIST AI RMF ties evaluation to the conditions of use and to monitoring the system after deployment. For a pilot, that means keeping the tested inputs, instruction versions, the method used and the limitations you found. If the model, the sources or the product rules change, repeat the part of the evaluation they affect.

Give the editor a way to check

"A person will look at it" hides several conditions. Lyazzat needs subject knowledge, time, a current source and the right to send the page back. She checks the price against the current price list, recalculates any figures and finds a quote in its original source. She also opens any link the model suggested and looks at what is actually there.

You can ask a second model to hunt for errors as one more pass. But two answers agreeing do not replace a source. If both systems repeat the wrong warranty period, the customer still gets a wrong promise.

Split the permissions. Drafting an email, choosing recipients and sending it are three actions with different consequences. A successful text pilot is no reason to let the system change audiences or publish without an editor right away. The process owner writes down who releases the result and what happens when there is doubt.

The Russian and Kazakh versions of a page are signed off by someone who knows the language and the terms of the offer. A good Russian text says nothing about the Kazakh one. An unfamiliar term or an ambiguous phrase is better sent back for review than waved through for speed.

Count team time and new spending

Apply the 80 and 75 minutes to 120 accepted product pages a month. Assume the time includes every attempt, the review and the corrections, and the quality is the same. Manual work comes to 9,600 minutes, or 160 hours. With AI, 9,000 minutes, or 150 hours. Ten hours are freed.

To turn hours into tenge, take an internal value of 5,000 ₸ an hour. The manual process uses 800,000 ₸ of team time, the new one 750,000 ₸. The difference is 50,000 ₸, but it includes neither maintenance nor the tool.

The pilot budget sets aside 30,000 ₸ a month for the tool and other payments. Maintenance takes five hours a month: investigating errors, updating the instructions, re-running checks. At the same rate, that is 25,000 ₸ of time.

Under these assumptions, the AI process costs 5,000 ₸ a month more. And that is before one-off pilot setup, which goes on its own line. The opposite mistake happens too. If maintenance is already counted in the measured 150 hours, do not add the five hours a second time.

The table compares team time, not money in the bank. Salaries stay the same, so the 50,000 ₸ does not come back to the till. The team frees ten hours of drafting, spends five of them on maintenance, and the store pays 30,000 ₸ in real money for the tool. The owner still has to decide what work Lyazzat will do in the remaining five hours and whether that work pays for the new expense.

Decide what data the service may get

For each input, decide whether the task needs it and who has the right to pass it to the chosen service. Storage, access and data-use terms depend on the product, the workspace and its settings. Before launch, the person responsible for that environment checks them. A marketing page for a different plan does not count as confirmation.

For the first runs, prepare examples without customer data but with the same language and task difficulty, or the test will be too easy. Removing the name is not enough: a text keeps other details that can identify a person. So someone reviews what goes into the input as well.

Keep a clear way back to manual work. If a source is unavailable or quality drops, the employee should know where the task goes, what to tell the requester and where to log the failure. One good draft cannot tell you how well this part of the process works.

Exercise: only the draft got faster

After an instruction change, the draft takes five minutes instead of ten. Review and corrections stay at 50 and 15 minutes. Assume new inputs confirmed this timing and the same quality. Recalculate the full cycle and the month with 120 pages, five hours of maintenance and 30,000 ₸ for the tool and payments.

The full cycle is now 5 + 50 + 15 = 70 minutes. Against the manual 80 minutes, that is ten minutes, or 12.5%. Across 120 pages, it adds up to twenty hours, 100,000 ₸ at the internal rate. Subtract five hours of maintenance and 30,000 ₸ of payments, and the AI process comes out 45,000 ₸ a month ahead, before one-off setup. Salaries still do not change, so this is a valuation of team time. It will not show up as a line in the profit report.

Make the same record for your own pilot: accepted volume, quality, total working time, new payments, maintenance and the task the freed hours will go to. Decide whether to expand the pilot from these numbers and from fresh evaluation. How fast the first answer arrives tells you nothing for that decision.