What changed — version 1.1 compared with 1.0
Only what changed since the approved version (1.0) goes back to a reviewer. Here that is 85 of 156 items — 54% of the package, about 21.25 hours instead of 39. The list below says why each document is or is not reviewed again.
85
items to reviewDocuments, rules and attack defences a reviewer has to look at again because something they depend on changed.
156
items if reviewed from scratchEvery document, rule and attack defence in the package — what a full re-review reads.
54%
of the package
21.25h
instead of 39hAn assumption: how long a reviewer spends on each item. The item counts are measured; this rate is yours to change. Currently 15 minutes per item.
Hours are the item counts, which are measured, times 15 minutes per item, which is an assumption. Change it above and the arithmetic follows.
What changed, and who said so
What the files themselves show changed
Training dataTraining or fine-tuning data, its labels, or its sources.Model weightsModel parameters — retraining, fine-tuning, or a new base checkpoint.
What the submitter said changed
Training dataTraining or fine-tuning data, its labels, or its sources.Model weightsModel parameters — retraining, fine-tuning, or a new base checkpoint.
Some changes leave no trace in the package at all — no file moves when a new attack is published, or when the model is moved to new hosting. Those we only know about because the submitter said so, which is why the two lists are kept apart.
DocumentsThe documents in a package — test results, data lists, design descriptions. Each has a source, a collection date and an expiry.
changedA different file from last time, so somebody has to look. 5old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. 24unchangedThe same file as last time, and still in date. Nobody needs to read it again. 33
| Document | State | Why |
|---|---|---|
| Model checkpoint fingerprintCryptographic hash of the exact weights file under accreditation. | changedA different file from last time, so somebody has to look. | The file is different from the one in the approved version. |
| Corpus manifestEnumeration of every dataset in the corpus with content hashes. | changedA different file from last time, so somebody has to look. | The file is different from the one in the approved version. |
| Data source attestationOrigin, licence, and rights basis for each corpus source. | changedA different file from last time, so somebody has to look. | The file is different from the one in the approved version. |
| Label provenanceWho or what produced the labels, under what instructions. | changedA different file from last time, so somebody has to look. | The file is different from the one in the approved version. |
| Weights provenanceWhere the weights came from, who produced them, and on what run. | changedA different file from last time, so somebody has to look. | The file is different from the one in the approved version. |
| Artifact malware scanScan of serialised artifacts for embedded executable content. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Backdoor screen resultTesting of the trained model for backdoor behaviour on trigger inputs. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data and model weights changed — so it describes the old model. |
| Base model attestationIdentity, source, and content hash of any foundation or pre-trained base. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Base model selection rationaleWhy this base model was chosen, and on what basis it was trusted. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Classification determinationClassification of the model, its weights, and its corpus. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data and model weights changed — so it describes the old model. |
| Corpus classification reviewClassification determination for the corpus and what it reveals. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data changed — so it describes the old model. |
| Corpus disclosure reviewWhat the corpus composition would reveal if disclosed. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data changed — so it describes the old model. |
| Corpus drift monitorContinuous monitoring of corpus composition and label quality over time. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data changed — so it describes the old model. |
| Deployment integrity checkVerification that the fielded artifact still matches the accredited hash. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Extraction evaluationHow much of the model can be reconstructed from its query interface. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Inversion evaluationWhat training imagery can be reconstructed from model outputs. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Label quality auditSampled re-adjudication of labels against the annotation standard. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data changed — so it describes the old model. |
| Membership inference evaluationWhether an adversary can determine that a given image was in training. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data and model weights changed — so it describes the old model. |
| Operational test recordPerformance in representative operational conditions, not benchmark conditions. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Physical attack evaluationPerformance against physically realisable attacks — patches, camouflage, decoys. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Poisoning screen resultScreening of corpus and labels for poisoning indicators. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data changed — so it describes the old model. |
| Public overlap analysisExtent to which the corpus overlaps publicly available datasets. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data changed — so it describes the old model. |
| Releasability determinationForeign-disclosure and releasability posture for the artifact. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data and model weights changed — so it describes the old model. |
| Retraining recordWhat was retrained, when, from which baseline, and why. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Retrieval timestampsWhen each external artifact was pulled, pinning it against later substitution. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Robustness evaluationPerformance under adversarially crafted inputs. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Serialisation format recordWeights file format and whether loading executes code. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Threat model recordStructured threat model mapped to the ATLAS catalogue, with a disposition and mitigation for every in-scope technique. The gated artifact. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the training data and model weights changed — so it describes the old model. |
| Transferability evaluationSuccess rate of attacks developed on a proxy and transferred to this model. | old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | Same file, but the model weights changed — so it describes the old model. |
| Adversarial test setThe adversarial examples the robustness evaluation was run against. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Analyst workload assessmentEffect of model output volume on the human review capacity behind it. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Anomalous query detectionDetection of query patterns consistent with extraction or probing. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| API access logRecord of queries against the inference interface. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Architecture disclosure reviewWhat published performance and architecture details reveal. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Architecture specModel structure, input and output shapes, and class ontology. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Artifact storage controlsControls protecting weights, corpora, and the package itself at rest. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Availability target (SLO)Availability commitment and its measurement for the inference service. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Collection chain attestationIntegrity of the path from sensor to training or inference input. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Decoy test setImagery of decoys and concealment measures used in that evaluation. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Denial deception assessmentAssessment against known adversary denial and deception practice. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Dependency scanKnown-vulnerability scan across that bill of materials. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Distribution recordWho holds a copy of the weights, under what agreement. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Downstream dissemination mapWhich products and users receive this model's outputs. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Drift detection recordDetection of distribution shift — new sensors, geographies, seasons. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| False positive monitorFalse-positive rate in the field, and its trend. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Framework version pinExact pinned versions of training and inference frameworks. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Hardware provenanceProvenance of accelerators and collection hardware in the chain. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Human review procedureWhere a human sits in the chain and what they are accountable for. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Impact assessmentMission consequence of model failure, in the terms the decision is made in. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Ontology release reviewReview of what the class ontology reveals. For an overhead detector the ontology approximates a statement of collection priorities. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Output disclosure policyWhat the model's outputs may reveal and to whom they may go. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Output granularity reviewWhether output precision leaks more capability than intended. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Package access logRecord of access to the accreditation package itself. The package aggregates everything an adversary would want in one place, which makes it a target in its own right. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Performance monitorFielded performance instrumentation between accreditation events. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Provenance of input imageryProvenance chain for imagery the fielded model scores. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Publication review recordPrepublication or export review of programme papers and briefings. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Query rate controlsRate limiting and quota controls on the inference interface. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Software bill of materials (SBOM)Software bill of materials for the training and serving stack. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Synthetic imagery detectionAbility to detect generated or manipulated imagery presented as collection. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Trigger test setCandidate backdoor triggers the model was tested against. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Vulnerability monitoring recordTracking of newly published attacks against this architecture family. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
| Weights access logRecord of who accessed model weights and when. | unchangedThe same file as last time, and still in date. Nobody needs to read it again. | Same file, still in date. Not reviewed again. |
What does not need reading again
| To review | Carried overUnchanged and still in date since the approved version, so carried over without being read again. | |
|---|---|---|
| RulesThe rules a package has to satisfy. These are illustrative sample rules; a real criteria set loads in the same way, as data. | 15 | 13 |
| Attack defencesThe AI attack techniques that apply to this model, from MITRE ATLAS. Each needs a recorded defence or an accepted risk. | 41 | 25 |
A rule is only carried over when every document under it is unchanged and still in date. Anything else puts it back in front of a reviewer.
What each label means
| changedA different file from last time, so somebody has to look. | A different file from last time, so somebody has to look. |
| old modelThe same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. | The same file, and out of date anyway: something it depended on has changed, so it describes a model that no longer exists. This is the one a plain file comparison misses. |
| past its dateThe same file, but past the date our policy says it stays good for. | The same file, but past the date our policy says it stays good for. |
| newIt was not in the approved version, so nobody has ever reviewed it. | It was not in the approved version, so nobody has ever reviewed it. |
| goneIt was in the approved version and it is not here. | It was in the approved version and it is not here. |
| unchangedThe same file as last time, and still in date. Nobody needs to read it again. | The same file as last time, and still in date. Nobody needs to read it again. |