Caveat verdict

azure-ai-evaluation-py

88
🟢 Trusted
No high-risk patterns surfaced by the deep scan — automated capability review, not behavioral proof.

Azure AI Evaluation SDK for assessing generative AI quality and safety; reads environment variables for Azure OpenAI credentials to make authorized API calls to the user's own Azure resources, which is the expected behavior for this type of SDK.

⚠ Flagged for review — coarse, uncorroborated signal, not a confirmed exploit. Review the config yourself before installing.

Automated static analysis — not a human review. Caveat flags capabilities, not confirmed intent, and can produce false positives. Disagree with this verdict? Use Dispute below.

63
security
30
transparency
70
maintenance

Permission integrity

Installs packages at runtime — transitive dependencies are not auditable

package_install

Findings (2)

Pattern match critical

Possible hardcoded credential

references/built-in-evaluators.md · code · api_key="<your-api-key>

Pattern match low

Python os.environ.get — reads environment variable

scripts/run_batch_evaluation.py · prose · downgraded · os.environ.get(

Permissions & capabilities

No declared permissions — minimal attack surface.

package_installnetwork_incredential_accesscredential_store

Is this flag fair?

Check another skill Browse the registry Auditing your own skills or configs? Use the API