4 Adding Safety Guardrails
gpu-accelerationtool-callingretrieval-augmented-generationllm-inferencetensorrtnvidia-generative-ai-exampleslarge-language-modelsmicroservicetriton-inference-serverLLMdata-flywheelragnemo
Export
Part IV. Adding Safety Guardrails
This notebook covers the following -
[3]
[4]
[5]
Entity Store, Customizer, Evaluator, Guardrails endpoint: http://nemo.test NIM endpoint: http://nim.test
Load the Custom Model
Specify the customized model name that you got from the finetuning notebook to the following variable.
[6]
The following code checks if the NIM endpoint hosts the models properly.
[7]
List of available models: ['meta/llama-3.2-1b-instruct', 'xlam-tutorial-ns/llama-3.2-1b-xlam-run1@cust-FarcM8gwhL1XFDXQ57qGLL']
Step 1: Adding a Guardrails Configuration to the Microservice
A default guardrail configuration with a simple self-check content moderation is enabled in the NeMo Microservice Helm Chart. But for this tutorial, we will use the content-safety model as a part of a new guardrails configuration (this model needs to either be deployed in the cluster or available through build.nvidia.com).
Start by running the following command which creates a config.yml file with the model deployed in the guardrails microservice
[19]
Created guardrail config: default/toolcalling
Created at: 2025-06-20 17:26:00.582557
Config data:
{
"models": [
{
"engine": "nim",
"type": "content_safety",
"api_key_env_var": "NIM_ENDPOINT_API_KEY",
"mode": "chat",
"model": "nvidia/llama-3.1-nemoguard-8b-content-safety",
"parameters": {},
"reasoning_config": {
"end_token": "</think>",
"remove_reasoning_traces": true,
"remove_thinking_traces": null,
"start_token": "<think>"
}
}
],
"actions_server_url": null,
"colang_version": "1.0",
"custom_data": {},
"enable_multi_step_generation": false,
"enable_rails_exceptions": false,
"instructions": [
{
"content": "Below is a conversation between a helpful AI assistant and a user. The bot is designed to generate human-like text based on the input that it receives. The bot is talkative and provides lots of specific details. If the bot does not know the answer to a question, it truthfully says it does not know.",
"type": "general"
}
],
"lowest_temperature": 0.001,
"passthrough": null,
"prompting_mode": "standard",
"prompts": [
{
"task": "content_safety_check_input $model=content_safety",
"content": null,
"max_length": 16000,
"max_tokens": 50,
"messages": [
{
"content": "Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n\n<BEGIN UNSAFE CONTENT CATEGORIES>\nS1: Violence.\nS2: Sexual.\nS3: Criminal Planning/Confessions.\nS4: Guns and Illegal Weapons.\nS5: Controlled/Regulated Substances.\nS6: Suicide and Self Harm.\nS7: Sexual (minor).\nS8: Hate/Identity Hate.\nS9: PII/Privacy.\nS10: Harassment.\nS11: Threat.\nS12: Profanity.\nS13: Needs Caution.\nS14: Other.\nS15: Manipulation.\nS16: Fraud/Deception.\nS17: Malware.\nS18: High Risk Gov Decision Making.\nS19: Political/Misinformation/Conspiracy.\nS20: Copyright/Trademark/Plagiarism.\nS21: Unauthorized Advice.\nS22: Illegal Activity.\nS23: Immoral/Unethical.\nS24: Deleting Logs.\n<END UNSAFE CONTENT CATEGORIES>\n\n<BEGIN CONVERSATION>\nuser: {{user_query}}\n<END CONVERSATION>\n\nProvide your safety assessment for the above conversation in the below output JSON format:\n{\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n\nDo not include anything other than the output JSON in your response.\nOutput JSON:",
"type": "system"
},
{
"content": "{{ user_input }}",
"type": "user"
}
],
"mode": "standard",
"models": null,
"output_parser": "nemoguard_parse_prompt_safety",
"stop": null
}
],
"rails": {
"actions": {
"instant_actions": null
},
"config": null,
"dialog": {
"single_call": {
"enabled": false,
"fallback_to_multiple_calls": true
},
"user_messages": {
"embeddings_only": false,
"embeddings_only_fallback_intent": null,
"embeddings_only_similarity_threshold": null
}
},
"input": {
"flows": [
"content safety check input $model=content_safety"
]
},
"output": {
"apply_to_reasoning_traces": false,
"flows": [],
"streaming": {
"chunk_size": 200,
"context_size": 50,
"enabled": false,
"stream_first": true
}
},
"retrieval": {
"flows": []
}
},
"sample_conversation": "user \"Hello there!\"\n express greeting\nbot express greeting\n \"Hello! How can I assist you today?\"\nuser \"What can you do for me?\"\n ask about capabilities\nbot respond about capabilities\n \"As an AI assistant, I can help you with a wide range of tasks. This includes question answering on various topics, generating text for various purposes and providing suggestions based on your preferences.\"\nuser \"Tell me a bit about the history of NVIDIA.\"\n ask general question\nbot response for general question\n \"NVIDIA is a technology company that specializes in designing and manufacturing graphics processing units (GPUs) and other computer hardware. The company was founded in 1993 by Jen-Hsun Huang, Chris Malachowsky, and Curtis Priem.\"\nuser \"tell me more\"\n request more information\nbot provide more information\n \"Initially, the company focused on developing 3D graphics processing technology for the PC gaming market. In 1999, NVIDIA released the GeForce 256, the world's first GPU, which was a major breakthrough for the gaming industry. The company continued to innovate in the GPU space, releasing new products and expanding into other markets such as professional graphics, mobile devices, and artificial intelligence.\"\nuser \"thanks\"\n express appreciation\nbot express appreciation and offer additional help\n \"You're welcome. If you have any more questions or if there's anything else I can help you with, please don't hesitate to ask.\"\n"
}
The following nemo_client.guardrail.configs.list() call lists the available guardrails configurations. You should be able to see the toolcalling configuration -
[20]
Found 4 guardrail configurations: - Config: default/self-check Description: self-check guardrail config Created: 2025-06-20 17:25:12.056077 Files URL: file:///app/services/guardrails/config-store/self-check - Config: default/abc Description: abc guardrail config Created: 2025-06-20 17:25:12.058581 Files URL: file:///app/services/guardrails/config-store/abc - Config: default/default Description: default guardrail config Created: 2025-06-20 17:25:12.060298 Files URL: file:///app/services/guardrails/config-store/default - Config: default/toolcalling Description: None Created: 2025-06-20 17:26:00.582557
[21]
To send a test query to the guardrailed chat API endpoint, create the following helper object:
[22]
Let's look at the usage example. Begin with Guardrails OFF and run the above unsafe prompt with the same set of tools.
2.1: Unsafe User Query - Guardrails OFF
[23]
Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='chatcmpl-tool-dba4e2bfe74646e1a7a8a9ceb9273420', function=Function(arguments='{"type": "loot"}', name='live_giveaways_by_type'), type='function')]), stop_reason=None)
Now Let's try the same with Guardrails ON The content-safety NIM should block the message and abort the process without calling the Tool-calling LLM
2.2: Unsafe User Query - Guardrails ON
[24]
Guardrails safety check: blocked Not a safe input, the guardrails has resulted in status as blocked. Tool-calling shall not happen
Let's try the safe user query with guardrails ON. The content-safety NIM should check the safety and ensure smooth running of the fine-tuned, tool-calling LLM
2.3: Safe User Query - Guardrails ON
[25]
Guardrails safety check: success
Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='chatcmpl-tool-38ea6f9de5244a17bb1ce5e161c69c60', function=Function(arguments='{"type": "beta,game"}', name='live_giveaways_by_type'), type='function')]), stop_reason=None)