In the realm of AI, agents rely on shared registries to select tools based on natural-language descriptions. However, there is a critical gap in the verification process of these descriptions, leaving room for potential risks and vulnerabilities.
The discovery of this gap came to light when a submission was made to the CoSAI secure AI tooling repository, highlighting the need for attention to selection-time threats and execution-time threats separately. This emphasized the fact that tool registry poisoning is not just a single vulnerability but rather a series of vulnerabilities that persist throughout the tool’s life cycle.
While existing software supply chain controls like code signing, SBOMs, SLSA, and Sigstore have been developed over the years to enhance security, they primarily focus on artifact integrity rather than behavioral integrity. This distinction is crucial as it ensures that tools not only match their descriptions but also behave as expected without any malicious intent.
To address this gap, a runtime verification layer known as MCP (Model Context Protocol) has been proposed. This layer acts as a verification proxy between the agent and the tool, performing validations such as discovery binding, endpoint allowlisting, and output schema validation to ensure that tools behave as intended and do not deviate from their expected behavior.
By incorporating behavioral specifications into the verification process, tools are monitored for any unauthorized actions or changes in behavior, mitigating risks such as tool impersonation, schema manipulation, behavioral drift, description injection, and transitive tool invocation. This multi-layered approach combines provenance checks with runtime verification to provide comprehensive security coverage.
To implement this solution without hindering developer velocity, a phased approach is recommended. Starting with endpoint allowlisting at deployment time, organizations can gradually introduce output schema validation, discovery binding for high-risk tools, and full behavioral monitoring based on the risk profile of each tool category.
In conclusion, while provenance checks play a crucial role in ensuring the integrity of tools in AI agent registries, incorporating runtime verification is essential to address behavioral integrity. By adopting a holistic approach that combines both provenance and runtime verification, organizations can enhance the security of their AI tooling pipelines effectively.