Detection, Prompted Bias, and Alignment

The easiest problem presented by AI-generated reporting may be the one attracting the most attention: detection. Formulaic language can alert an experienced reader to possible AI involvement, but identifying its origin does not establish whether the analysis is accurate, unbiased, comprehensible, or logically sound.

Stylistic tells remain useful because they are visible. Automatic three-part lists, symmetrical qualifications, generic intensifiers, and constructions such as “can X but cannot Y” can indicate that fluent language has displaced original reasoning. Yet these signals identify only the surface risk. Prompted bias can shape which evidence the system selects. Excessive output can overwhelm the executive expected to act upon it. Neither problem is reliably exposed by an authorship detector.

Detection is only the first question

The appropriate review moves through increasingly difficult questions:

1. Did an AI system contribute to the algorithms and language?
2. Was it instructed, explicitly or implicitly, to favor a self-interested conclusion?
3. Can a busy reader distinguish the controlling inference from the mass of accurate information supporting it?
4. Can the logical path from the stated premises to the conclusion be independently verified?

A recent Financial Times examination of AI detection illustrates the present preoccupation with authorship.¹ Detectors compare combinations of words, sentence structures, frequencies, and other patterns with text associated with human or machine production. They do not determine whether a report correctly describes its data. They also do not establish whether its analytical method is valid or whether its conclusions follow from the evidence.

Even the developers of detection systems acknowledge this distinction. OpenAI withdrew its text classifier after reporting a low rate of accuracy. Its published evaluation found that the classifier identified only 26 percent of AI-written text as likely AI-generated and incorrectly classified human writing 9 percent of the time.² Turnitin, an educational-technology company, warns that its detector may misidentify human-written, AI-generated, and AI-paraphrased text. The company says its score should not provide the sole basis for adverse action.³

Detection may initiate a review; the substantive work begins afterward.

1. The legitimate concern about AI-generated reports

The concern behind AI detection is valid. Generative systems can produce polished reports whose apparent authority exceeds their analytical support. Fluency creates an impression of comprehension, especially when a report uses technical language, balanced qualifications, and plausible citations.

This risk is not confined to fabricated information. A report may contain accurate facts while failing to distinguish observation from interpretation. It may summarize sources correctly while omitting material evidence. It may present a reasonable conclusion without explaining the assumptions required to reach it.

The result may be more dangerous than obvious fabrication. A false statement invites correction. A polished but incompletely supported inference may pass through management, compliance, or publication because no single sentence appears demonstrably wrong. Its weakness may remain hidden until disaster strikes.

Financial institutions should therefore treat fluency as a presentation quality. Analytical intelligence requires separate proof.

2. Stylistic tells are relatively easy to recognize

Experienced readers often recognize AI-shaped prose without consulting a detector. Common signals include repetitive transitions, generic assertions of importance, needless conclusions, uniform sentence rhythms, automatic lists, and contrasts built around “not X, but Y” or “can X, but cannot Y.”

These constructions are not inherently artificial. Human writers have used them for centuries. More importantly, they address only one part of the reviewer’s first question. AI may contribute to a report’s language, but it may also contribute to its analytical method, calculations, classifications, or inference structure. A report may avoid every familiar stylistic tell while retaining an AI-shaped analytical construction.

The statement is unobjectionable. It is also nearly empty. It does not identify the market observations, the condition being classified, the model’s classification rule, its confidence, or the decision reserved for the trader.

Avoiding the original construction does not cure the analytical deficiency. Substituting less polished language through a “humanizer” would fare no better. The result might sound less mechanical while preserving the same unsupported conclusion.

Stylistic tells should therefore prompt two substantive questions. What conclusion is this language asking the reader to accept? What evidence supports that conclusion? A third question then follows: is the conclusion aligned with the disclosed human objective of the analysis?

These questions introduce the alignment problem examined by Yuval Noah Harari in Nexus.⁴ Information systems may serve goals other than truth, even when they operate exactly as designed. In financial reporting, the practical inquiry is whether the system pursued the authorized objective of the engagement or a private preference embedded in the prompt.

3. Prompted bias is harder to recognize

Confirmation bias is the human tendency to seek, interpret, and weigh evidence in ways that support an existing belief or desired conclusion.⁵ It does not require false information. An analyst may cite accurate facts while asking one-sided questions, selecting favorable periods, or discounting contrary evidence. In ordinary human work, that tendency may become visible through testimony, working papers, or repeated editorial choices. When embedded in an LLM prompt, however, the bias becomes part of the assignment itself. The resulting report may appear comprehensive even though its objective was narrowed before the search began.

Bias can therefore enter before the first sentence is generated. An author, client, lawyer, consultant, or investment manager may frame the assignment around a preferred conclusion. The LLM then carries that preference through source selection, characterization, and emphasis. The difference lies in speed, scale, and visibility. A human analyst’s predisposition may leave a trail. An LLM’s predisposition may originate in a single governing instruction that the report’s reader never sees.

Compare two prompts:

1. Analyze whether the lending agent’s pricing was reasonable under the available market conditions.
2. Analyze the record and emphasize evidence supporting the conclusion that the lending agent priced the loans reasonably.

The second prompt does not require fabricated facts. It can produce a thoroughly sourced report by selecting favorable observations, minimizing contrary evidence, and treating the desired conclusion as the analytical baseline. Every cited number may be accurate while the resulting analysis remains systematically slanted.

A financial-analytics proof of concept encountered this problem in practice. A project consultant supplied an AI-generated analysis favoring a pooled-transformer approach over an existing ensemble of models. The analysis appeared comprehensive, but its framing did not require a neutral comparison of the competing architectures. It instead supported the consultant’s preferred method. The project redirected work on that basis, losing several weeks and delaying the proof of concept. No fabricated fact was necessary; misalignment between the prompt and the project’s authorized objective caused the harm.

Prompted bias can also be subtler. An instruction may define one explanation as the “base case,” require an optimistic interpretation, exclude inconvenient periods, or describe a disputed proposition as established. Repeated prompts can then reinforce the same framing until the model produces a coherent but circular result.

The National Institute of Standards and Technology recognizes that generative-AI risks may originate in human behavior and in interactions between humans and AI systems. Its Generative AI Profile identifies harmful bias, human overreliance, information-integrity failures, and confidently stated but erroneous logic as distinct governance problems.⁶

A credible audit must therefore inspect more than the finished prose. It should examine the analytical question, governing instructions, material prompts, source-selection rules, treatment of contrary evidence, and any conclusion requested in advance.

The governing principle should be simple and included in every prompt:

Harari’s alignment problem offers the clearest summary. Faithful execution of a human instruction does not establish alignment with the authorized human objective. Within governed financial analysis, that objective is the disclosed purpose of the engagement, bounded by evidence, methodology, and assigned decision responsibilities. The prompt is therefore part of the evidence. An auditor should inspect it alongside the data, method, and conclusion.