Traditional application security asks familiar questions: is the code reviewed, are dependencies patched, is the infrastructure hardened. An AI system built on a pre-trained or fine-tuned model introduces a supply chain those questions were never designed to reach — the model itself, and everything that went into producing it.What the Application-Layer Review Cannot SeeA model file is, from a conventional review's perspective, an opaque artefact. It cannot be read the way source code can, its behaviour is not fully determined by its size or its declared purpose, and a subtle change introduced during fine-tuning — a shift in what the model will and will not do — is invisible to every control designed around static code and network traffic.Questions Worth Adding to the ReviewProvenance: where the base model came from, under what licence, and whether that source has any history of compromise or dataset dispute.Fine-tuning data lineage: what data was used to adapt the model, whether it included anything the organization was not entitled to use, and who approved that decision.Version control for models, not just code: a model that can be silently swapped or retrained without a recorded change has no audit trail behind its behaviour.Input and output handling: what happens to data sent to the model and returned from it — whether either is logged, retained, or exposed to a third-party provider in ways the organization has not accounted for.Abuse and manipulation testing: whether the system has been tested against inputs designed specifically to make it behave outside its intended purpose, not just against ordinary functional cases.“A security review that never asks what a model was trained on is reviewing the building and skipping the foundation.”Bringing It Into the Existing ProgrammeNone of this requires a separate AI security function. It requires the existing security review process to add a small set of model-specific questions at the point a new AI use case is proposed, and to treat a 'we don't know' answer to any of them as a finding rather than a gap the review quietly stepped around.Share this articleX (Twitter)LinkedInFacebookCopy Link
ISO 42001Aug 27, 2026Preparing for ISO 42001 Without Overbuilding the AI Management SystemMost organizations approaching ISO/IEC 42001 for the first time either bolt AI governance onto an existing ISMS as an afterthought, or build a second management system nobody has the resourcing to run twice.
Responsible AIJul 16, 2026Responsible AI Starts With Who Can Say No to a DeploymentA responsible-AI principles document with no one empowered to block a launch is a statement of values, not a control.
NIST AI RMFAug 13, 2026Using the NIST AI RMF as a Map, Not a ChecklistThe framework's four functions read like a checklist an auditor can tick. Treated that way, an organization can complete every one of them and still not know whether a specific AI system is safe to deploy.
