Key outcomes from NIST AI 600-1: Generative AI Profile (Draft)

Document scope: Implementation profile drafted by the U.S. National Institute of Standards and Technology to operationalize the AI RMF 1.0 for generative AI model developers, deployers, and evaluators.

Govern function

• Establish enterprise generative AI policies that cover system inventory, training data rights, IP protections, and incident disclosure.
• Assign accountable executives and cross-functional governance boards to oversee generative AI risk tolerances, escalation paths, and funding.
• Embed supply chain due diligence for foundation models, APIs, and third-party fine-tuning services, including contractual controls and transparency requirements.
• Ensure workforce competence through training on prompt security, copyright, safety evaluation, and content moderation workflows.

Map function

• Document intended use, prohibited uses, context-of-use assumptions, and user populations for each generative AI capability.
• Capture data lineage for pre-training, fine-tuning, reinforcement learning, and retrieval-augmented sources; confirm licensing and privacy compliance.
• Model system architecture, including tool integration, plug-ins, safety guardrails, retrieval connectors, and human checkpoints.
• Assess downstream impacts across safety, security, privacy, civil rights, and misinformation risks in the deployment environment.

Measure function

• Define evaluation plans with quantitative and qualitative metrics for hallucination rates, toxicity, bias, factual accuracy, watermark robustness, and jailbreak resilience.
• Conduct systematic red-teaming, adversarial prompting, and automated stress testing before deployment and after significant updates.
• Monitor output logs, guardrail triggers, and user feedback to detect drift, misuse, or emerging harms; integrate with SOC/SIEM workflows.
• Validate third-party model components and safety filters through independent assurance and reproducible testing.

Manage function

• Implement preventive and detective controls such as content filtering, rate limiting, human-in-the-loop review, provenance tagging, and revocation mechanisms.
• Maintain incident response runbooks covering jailbreaks, data leakage, model theft, hallucination-driven harm, and rapid rollback of unsafe updates.
• Establish continuous improvement loops that feed evaluation results, user reports, and regulatory changes into retraining and policy updates.
• Provide downstream users with transparency artifacts—model cards, system cards, user documentation, and end-user disclosures—aligned to AI RMF expectations.

Operational expectations for federal agencies

• Agencies should require contractors and grantees to demonstrate profile alignment through acquisition language, program reviews, and reporting to the U.S. AI Safety Institute.
• Adopt shared evaluation infrastructure from USAISI to standardize red-teaming, reporting, and continuous monitoring across agencies.
• Integrate profile outcomes into capital planning, cybersecurity strategies, privacy impact assessments, and mission assurance metrics.
• Mandate provenance tagging and synthetic content disclosure in citizen-facing services while ensuring accessibility and civil rights protections.