Awareness programmes should measure whether staff verify the unexpected request

Many security awareness programme owners are now facing a decision is whether the next campaign should keep measuring how well users spot a flawed email, or start measuring how reliably they confirm an unexpected request through another channel. Microsoft's disclosure on 22 September, reported by Axios, makes it harder to keep postponing that change.

Microsoft said it had taken down digital infrastructure tied to a cybercrime platform that, in the company's description, "relied on AI tools throughout its operations." Microsoft identified the platform as EvilTokens and described how AI was used across multiple stages of the attack chain. What matters is the phrase itself: the platform did not use AI for one narrow task, such as drafting a lure or generating a voice note. It used AI across the workflow.

The easy takeaway is that a large vendor disrupted a cybercrime operation. The more useful one is that the operation was built like a production line, and production lines lower the cost of every convincing request. Axios put the wider point plainly: financially motivated hackers are already using existing AI tools "to turn stolen corporate network access into opportunities for fraud." That cost shift is what should change a training decision.

The training assumption this overturns

For years, awareness training tested the telltale misspelling. A stray capital, an awkward phrase or a mismatched domain was the exercise, and the desired behaviour was to report it. That test was already weakening before this week. The platforms that now produce fraudulent requests run on models that write clean, idiomatic prose in the language of the target. When the grammar is no longer the signal, the thing to test is whether a person, faced with a plausible request to change a payment, share a document or approve a purchase, stops to confirm the request through a channel the requester does not control.

In a mid-sized organisation, the second channel is often missing because it is not written down. Finance may have an informal rule to confirm a supplier's changed bank details by phone. HR may assume payroll alterations need a verbal sign-off. When one person owns awareness alongside supplier reviews and audit evidence, those informal rules rarely become training content or a measurable test. The result is a campaign that reports a healthy click rate on obvious fakes while the request most likely to cause loss arrives outside the simulation.

Verification has to scale with the attacker

The shift is economic. The platform's value came from turning stolen access into fraudulent outcomes with less human effort. A leaked credential was never the end of the story; it is the raw material for a series of requests that look legitimate enough to act on. The less human effort each step requires, the more requests the attacker can afford to send, and the less each one needs to be individually convincing. The defence has to scale in the same way.

That is where a managed programme earns its keep. With Managed Cyber Awareness Training, from obvious spelling and language errors to request-verification exercises without rebuilding the whole programme, and the reporting shows who falls for which type of request rather than who can spot a typo. The finance team should see simulated supplier-bank-change requests without a misspelling in sight and be measured on whether they pick up the phone. The legal team should see plausible document-share requests from near-identical domains. The point is to train the verification habit against the kind of request the platform was designed to automate, rather than against the errors it no longer makes.

When a user presses the Report Phish button, the mail needs somewhere to go. Managed detection and response gives the button a destination and an analyst, which closes the loop instead of leaving reports in a shared mailbox.

A check for this quarter's campaign

Before the next awareness session is signed off, ask whoever supplies or builds the content one question: how many of the simulations test confirmation of an unexpected request, rather than recognition of a flaw? If the answer is a percentage, ask to see the actual simulations behind that number. If none of the simulations test verification, that is your finding. Then take the two processes that matter most in your organisation—supplier bank-detail changes and payroll alterations are the usual pair—and write the second channel for each, as a rule people can be tested against. A voice call to a known number, a separate approval step in the finance system, or a physical confirmation from the requester all count. A reply to the original email does not.

The disruption will probably draw another takedown or an indictment and then the news cycle will move on. What remains is the change in cost: the requests that matter are now cheap to produce at quality and expensive to distinguish by eye. Programmes that only test the eye will keep reporting a healthy pass rate while the expensive mistake travels through a channel nobody trained.