"Model Permission" filter do not apply to MarketPlace nor playground

[davihern]

Select Topic Area

Bug

Body

When enabling specific models to use in GitHub Models inside a GitHub Organization, so we can limit what Models the company want to developers to use.

We can see that when a developers is within a repository, the configured filter gets applied, and only the selected models appear in the combo box.

But if the user enters into the marketplace or the play ground. The filter do not apply.

https://github.com/marketplace/models
https://github.com/marketplace/models/azureml-xai/grok-3-mini/playground

[Akash1134]

Hi @davihern,

Thank you for reporting this issue with GitHub Models permissions. Based on the documentation and current implementation, I can confirm this is a limitation in the current public preview of GitHub Models.

Current behavior:

• When you configure organization-level model permissions to restrict which models developers can use, these restrictions are correctly applied within repository contexts (as shown in your first screenshot).
• However, these same restrictions are not currently applied when users access the GitHub Models Marketplace or Playground directly.

Technical explanation:

GitHub Models is still in public preview, and the organization-level permission controls appear to be designed primarily to restrict model usage within repositories. According to the GitHub Models documentation, the permission filters are intended to control which models can be used in your organization's repositories.

The Marketplace and Playground environments are global GitHub features that currently operate outside the scope of these organization-level restrictions. This seems to be by design currently, allowing developers to explore and experiment with models in the playground even if those models are restricted for use in actual repositories.

Recommendation:

Since GitHub Models is in public preview and explicitly noted as "subject to change" in the documentation, I recommend:

1. Continue using the organization-level permissions to restrict model usage within repositories
2. Report this as desired functionality through GitHub feedback channels
3. Consider implementing additional controls (like internal policies) until this feature gap is addressed

You may want to also explore GitHub Copilot enterprise-level policies if you need more comprehensive AI-related controls across your organization.

Thank you for helping improve GitHub Models by identifying this inconsistency in how permissions are applied!

[KateCatlin]

Hey @davihern! Yep, confirming what Akash said: model filters intentionally don’t apply in the non-repo-specific playground (https://github.com/marketplace/models), since you’re outside of any organization context. That’s similar to if someone clicked out to try models on a different website, we don’t have the repo-level settings to reference.

Would love to learn more about your use case here! Which models are you restricting right now? And just to make sure it's on your radar, all the hosted models run through Azure, so no data goes back to the model providers.

[davihern]

Thanks for the response! The worries come from having a way to limit access only to the models approved by the company (that will change over time).
We want to avoid developers to use models that are not approved, to avoid rework and also we want to avoid sending data to LLMs that are not approved by security. Good to know that all models are hosted on Azure with the same level of security!