title | ms.topic | ms.custom | description | ms.assetid | ms.date | monikerRange |
---|---|---|---|---|---|---|
Create and manage agent pools |
conceptual |
devx-track-azurecli |
Learn about organizing agents into pools for builds and releases in Azure Pipelines and Team Foundation Server |
BD5478A8-48CF-4859-A0CB-6E1948CE2C89 |
08/30/2024 |
<= azure-devops |
[!INCLUDE version-lt-eq-azure-devops]
::: moniker range="= azure-devops"
An agent pool is a collection of agents. Instead of managing each agent individually, you organize agents into agent pools. When you configure an agent, it is registered with a single pool, and when you create a pipeline, you specify the pool in which the pipeline runs. When you run the pipeline, it runs on an agent from that pool that meets the demands of the pipeline.
Note
Managed DevOps Pools agent pools are managed in the Azure Portal. If you're using Managed DevOps Pools, see Create your first Managed DevOps Pool.
::: moniker-end
::: moniker range="= azure-devops"
In Azure Pipelines, pools are scoped to the entire organization; so you can share the agent machines across projects.
::: moniker-end
::: moniker range=">= azure-devops-2019 < azure-devops"
In Azure DevOps Server, agent pools are scoped to the entire server; so you can share the agent machines across projects and collections.
::: moniker-end
::: moniker range="<=azure-devops"
Note
Agent pool jobs run a job on a single agent. If you need to run a job on all agents, such as a deployment group for classic release pipelines, see Provision deployment groups.
::: moniker-end
::: moniker range=">= azure-devops-2019"
If you are an organization administrator, you create and manage agent pools from the agent pools tab in admin settings.
::: moniker-end
[!INCLUDE agent-pools-tab]
::: moniker range=">= azure-devops-2019"
If you are a project team member, you create and manage agent pools from the agent pools tab in project settings.
::: moniker-end
[!INCLUDE agent-queues-tab]
The following agent pools are provided by default:
- Default pool: Use it to register self-hosted agents that you've set up.
::: moniker range="azure-devops"
-
Azure Pipelines hosted pool with various Windows, Linux, and macOS images. For a complete list of the available images and their installed software, see Microsoft-hosted agents.
[!NOTE] The Azure Pipelines hosted pool replaces the previous hosted pools that had names that mapped to the corresponding images. Any jobs you had in the previous hosted pools are automatically redirected to the correct image in the new Azure Pipelines hosted pool. In some circumstances, you may still see the old pool names, but behind the scenes the hosted jobs are run using the Azure Pipelines pool. For more information, see the Single hosted pool release notes from the July 1 2019 - Sprint 154 release notes.
By default, all contributors in a project are members of the User role on hosted pools. This allows every contributor in a project to author and run pipelines using Microsoft-hosted agents.
::: moniker-end
:::moniker range=">=azure-devops-2019"
To choose a Microsoft-hosted agent from the Azure Pipelines pool in your Azure DevOps Services YAML pipeline, specify the name of the image, using the YAML VM Image Label from this table.
pool:
vmImage: ubuntu-latest # This is the default if you don't specify a pool or vmImage.
To use a private pool with no demands:
pool: MyPool
For more information, see the YAML schema for pools.
:::moniker-end
To choose a pool and agent in the classic editor, navigate to the pipeline settings, select the desired Agent pool, and then the desired image from the Agent Specification drop-down. The default Agent Specification is windows-2019. For more information about the software installed on the Microsoft-hosted images, see the corresponding entry in the Classic Editor Pool column from this table.
:::image type="content" source="media/agent-pool-classic.png" alt-text="Screenshot of selecting and Agent pool and choosing the desired agent.":::
::: moniker range=">= azure-devops-2019" If you are an organization administrator, you create and manage agent pools from the agent pools tab in admin settings. ::: moniker-end
[!INCLUDE agent-pools-tab]
::: moniker range=">= azure-devops-2019" If you are a project team member, you create and manage agent pools from the agent pools tab in project settings. ::: moniker-end
[!INCLUDE agent-queues-tab]
To delete a pool, go to the Agent pools list, choose More options, Delete.
:::image type="content" source="media/agent-pool-delete.png" alt-text="Screenshot of deleting an agent pool.":::
::: moniker range="azure-devops"
List agent pools | Show agent pool details | List agent queues | Show agent queue details
Note
At this time you can view information about agent pools and queues, but not edit them, using the Azure CLI.
If this is your first time using az devops pipelines
commands, see Get started with Azure DevOps CLI.
az pipelines pool list [--action {manage, none, use}]
[--detect {false, true}]
[--org]
[--pool-name]
[--pool-type {automation, deployment}]
- action: Filter the list with user action permitted. Accepted values: manage, none, use
- detect: Automatically detect organization. Accepted values: false, true
- org or organization: Azure DevOps organization URL. You can configure the default organization using az devops configure -d organization=ORG_URL. Required if not configured as default or picked up via git config. Example:
https://dev.azure.com/MyOrganizationName/
. - pool-name: Filter the list with matching pool name.
- pool-type: Filter the list with type of pool. Accepted values: automation, deployment
The following example lists all pools in table format. This example uses the following default configuration: az devops configure --defaults organization=https://dev.azure.com/fabrikam-tailspin project=FabrikamFiber
az pipelines pool list --output table
ID Name Is Hosted Pool Type
---- ------------------------------- ----------- -----------
1 Default False automation
2 Hosted True automation
3 Hosted VS2017 True automation
4 Hosted Windows 2019 with VS2019 True automation
5 Hosted Windows Container True automation
6 Hosted macOS True automation
7 Hosted macOS High Sierra True automation
8 Hosted Ubuntu 1604 True automation
9 Azure Pipelines True automation
10 MyAgentPool False automation
az pipelines pool show --id
[--action {manage, none, use}]
[--detect {false, true}]
[--org]
- id or pool-id: (Required) ID of the pool to list the details.
- action: Filter the list with user action permitted. Accepted values: manage, none, use
- detect: Automatically detect organization. Accepted values: false, true
- org or organization: Azure DevOps organization URL. You can configure the default organization using az devops configure -d organization=ORG_URL. Required if not configured as default or picked up via git config. Example:
https://dev.azure.com/MyOrganizationName/
.
The following example displays pool details for the Hosted Windows 2019 with VS2019 pool. This example uses the following default configuration: az devops configure --defaults organization=https://dev.azure.com/fabrikam-tailspin project=FabrikamFiber
az pipelines pool show --id 4
{
"agentCloudId": 1,
"autoProvision": true,
"autoSize": null,
<Some properties omitted for space>
"poolType": "automation",
"properties": null,
"scope": "941fcaeb-be37-4309-b7b0-5cf156e1236e",
"size": 1,
"targetSize": 1
}
You can also use --output table
that returns the same information as the list
command.
az pipelines pool show --id 4 --output table
ID Name Is Hosted Pool Type
---- ------------------------------- ----------- -----------
4 Hosted Windows 2019 with VS2019 True automation
az pipelines queue list [--action {manage, none, use}]
[--detect {false, true}]
[--org]
[--project]
[--queue-name]
- action: Filter the list with user action permitted. Accepted values: manage, none, use
- detect: Automatically detect organization. Accepted values: false, true
- org or organization: Azure DevOps organization URL. You can configure the default organization using az devops configure -d organization=ORG_URL. Required if not configured as default or picked up via git config. Example:
https://dev.azure.com/MyOrganizationName/
. - project or p: Name or ID of the project. You can configure the default project using
az devops configure -d project=NAME_OR_ID
. Required if not configured as default or picked up via git config. - queue-name: Filter the list with matching queue name regex, e.g., ubuntu for queue with name 'Hosted Ubuntu 1604'.
The following example lists all queues in table format. This example uses the following default configuration: az devops configure --defaults organization=https://dev.azure.com/fabrikam-tailspin project=FabrikamFiber
az pipelines queue list --output table
This command group is in preview. It may be changed/removed in a future release.
ID Name Pool IsHosted Pool Type
---- ------------------------------- --------------- -----------
11 Default False automation
12 Hosted True automation
13 Hosted VS2017 True automation
14 Hosted Windows 2019 with VS2019 True automation
15 Hosted Windows Container True automation
16 Hosted macOS True automation
17 Hosted macOS High Sierra True automation
18 Hosted Ubuntu 1604 True automation
19 Azure Pipelines True automation
az pipelines queue show --id
[--action {manage, none, use}]
[--detect {false, true}]
[--org]
[--project]
- id or queue-id: ID of the agent queue to get information about.
- action: Filter the list with user action permitted. Accepted values: manage, none, use
- detect: Automatically detect organization. Accepted values: false, true
- org or organization: Azure DevOps organization URL. You can configure the default organization using az devops configure -d organization=ORG_URL. Required if not configured as default or picked up via git config. Example:
https://dev.azure.com/MyOrganizationName/
. - project or p: Name or ID of the project. You can configure the default project using
az devops configure -d project=NAME_OR_ID
. Required if not configured as default or picked up via git config.
The following example displays queue details for the Hosted Windows 2019 with VS2019 queue. This example uses the following default configuration: az devops configure --defaults organization=https://dev.azure.com/fabrikam-tailspin project=FabrikamFiber
az pipelines queue show --id 14
{
"id": 14,
"name": "Hosted Windows 2019 with VS2019",
"pool": {
"id": 4,
"isHosted": true,
"isLegacy": true,
"name": "Hosted Windows 2019 with VS2019",
"poolType": "automation",
"scope": "941fcaeb-be37-4309-b7b0-5cf156e1236e",
"size": 1
},
"projectId": "16836457-4ce1-4e77-b97a-e7e0c6508e84"
}
::: moniker-end
[!INCLUDE temp]
Pools are used to run jobs. Learn about specifying pools for jobs.
If you've got a lot of self-hosted agents intended for different teams or purposes, you might want to create additional pools as explained below.
Here are some typical situations when you might want to create self-hosted agent pools:
::: moniker range="azure-devops"
-
You're a member of a project and you want to use a set of machines owned by your team for running build and deployment jobs. First, make sure you've got the permissions to create pools in your project by selecting Security on the agent pools page in your Project settings. You must have Administrator role to be able to create new pools. Next, select Add pool and select the option to create a new pool. Finally install and configure agents to be part of that agent pool.
-
You're a member of the infrastructure team and would like to set up a pool of agents for use in all projects. First, make sure you've got the permissions to create pools in your project by selecting Security on the agent pools page in your Organization settings. Next create a New agent pool and select the option to Auto-provision this agent pool in all projects while creating the pool. This setting ensures all projects have access to this agent pool. Finally install and configure agents to be part of that agent pool.
-
You want to share a set of agent machines with multiple projects, but not all of them. First, navigate to the settings for one of the projects, add an agent pool, and select the option to create a new pool at the organization level. Next, go to each of the other projects, and create a pool in each of them while selecting the option to Use an existing agent pool from the organization. Finally, install and configure agents to be part of the shared agent pool. ::: moniker-end
::: moniker range=">azure-devops-2019 < azure-devops"
-
You're a member of a project and you want to use a set of machines owned by your team for running build and deployment jobs. First, make sure you've got the permissions to create pools in your project by selecting Security on the agent pools page in your Project settings. You must have Administrator role to be able to create new pools. Next, select Add pool and select the option to create a new pool. Finally install and configure agents to be part of that agent pool.
-
You're a member of the infrastructure team and would like to set up a pool of agents for use in all projects. First, make sure you've got the permissions to create pools in your project by selecting Security on the agent pools page in your Project collection settings. Next create a New agent pool and select the option to Auto-provision this agent pool in all projects while creating the pool. This setting ensures all projects have access to this agent pool. Finally install and configure agents to be part of that agent pool.
-
You want to share a set of agent machines with multiple projects, but not all of them. First, navigate to the settings for one of the projects, add an agent pool, and select the option to create a new pool at the organization level. Next, go to each of the other projects, and create a pool in each of them while selecting the option to Use an existing agent pool from the organization. Finally, install and configure agents to be part of the shared agent pool. ::: moniker-end
::: moniker range="=azure-devops-2019"
-
You're a member of a project and you want to use a set of machines owned by your team for running build and deployment jobs. First, make sure you're a member of a group in All Pools with the Administrator role. Next create a New project agent pool in your project settings and select the option to Create a new organization agent pool. As a result, both an organization and project-level agent pool will be created. Finally install and configure agents to be part of that agent pool.
-
You're a member of the infrastructure team and would like to set up a pool of agents for use in all projects. First make sure you're a member of a group in All Pools with the Administrator role. Next create a New organization agent pool in your admin settings and select the option to Auto-provision corresponding project agent pools in all projects while creating the pool. This setting ensures all projects have a pool pointing to the organization agent pool. The system creates a pool for existing projects, and in the future it will do so whenever a new project is created. Finally install and configure agents to be part of that agent pool.
-
You want to share a set of agent machines with multiple projects, but not all of them. First create a project agent pool in one of the projects and select the option to Create a new organization agent pool while creating that pool. Next, go to each of the other projects, and create a pool in each of them while selecting the option to Use an existing organization agent pool. Finally, install and configure agents to be part of the shared agent pool. ::: moniker-end
Understanding how security works for agent pools helps you control sharing and use of agents.
Roles are defined on each agent pool, and membership in these roles governs what operations you can perform on an agent pool.
Role on an agent pool in organization settings | Purpose |
---|---|
Reader | Members of this role can view the agent pool as well as agents. You typically use this to add operators that are responsible for monitoring the agents and their health. |
Service Account | Members of this role can use the organization agent pool to create a project agent pool in a project. If you follow the guidelines above for creating new project agent pools, you typically do not have to add any members here. |
Administrator | In addition to all the above permissions, members of this role can register or unregister agents from the organization agent pool. They can also refer to the organization agent pool when creating a project agent pool in a project. Finally, they can also manage membership for all roles of the organization agent pool. The user that created the organization agent pool is automatically added to the Administrator role for that pool. |
The All agent pools node in the Agent Pools tab is used to control the security of all organization agent pools. Role memberships for individual organization agent pools are automatically inherited from those of the 'All agent pools' node. By default, TFS and Azure DevOps Server administrators are also administrators of the 'All agent pools' node when using TFS or Azure DevOps Server.
Roles are also defined on each project agent pool, and memberships in these roles govern what operations you can perform on an agent pool at the project level.
Role on an agent pool in project settings | Purpose |
---|---|
Reader | Members of this role can view the project agent pool. You typically use this to add operators that are responsible for monitoring the build and deployment jobs in that project agent pool. |
User | Members of this role can use the project agent pool when authoring pipelines. |
Administrator | In addition to all the above operations, members of this role can manage membership for all roles of the project agent pool. The user that created the pool is automatically added to the Administrator role for that pool. |
::: moniker range="azure-devops"
Pipeline permissions control which YAML pipelines are authorized to use an agent pool. Pipeline permissions do not restrict access from Classic pipelines.
You can choose from the following options:
-
Open access for all pipelines to use the agent pool from the more options at top-right corner of the Pipeline permissions section in security tab of an agent pool.
-
Lock down the agent pool and only allow selected YAML pipelines to use it. If any other YAML pipeline refers to the agent pool, an authorization request gets raised, which must be approved by an agent pool Administrator. This does not limit access from Classic pipelines.
:::image type="content" source="media/agent-pools-pipeline-permissions.png" alt-text="Screenshot of the pipeline permissions user experience for an agent pool.":::
Pipeline permissions for the Azure Pipelines agent pool cannot be configured, as the pool is accessible, by default, to all pipelines.
::: moniker-end
::: moniker range="=azure-devops-2019"
The All agent pools node in the Agent pools tab is used to control the security of all project agent pools in a project. Role memberships for individual project agent pools are automatically inherited from those of the 'All agent pools' node. By default, the following groups are added to the Administrator role of 'All agent pools': Build Administrators, Release Administrators, Project Administrators.
::: moniker-end
::: moniker range="azure-devops"
The Security action in the Agent pools tab is used to control the security of all project agent pools in a project. Role memberships for individual project agent pools are automatically inherited from what you define here. By default, the following groups are added to the Administrator role of 'All agent pools': Build Administrators, Release Administrators, Project Administrators.
::: moniker-end
If no window is scheduled, then the agents in that pool will not run the maintenance job.
You can configure agent pools to periodically clean stale working directories and repositories. This should reduce the potential for the agents to run out of disk space. Maintenance jobs are configured at the organization level in agent pool settings.
To configure maintenance job settings:
[!INCLUDE agent-pools-tab]
Choose the desired pool and choose Settings to configure maintenance job settings for that agent pool.
Important
You must have the Manage build queues permission to configure maintenance job settings. If you don't see the Settings tab or the Maintenance History tab, you don't have that permission, which is granted by default to the Administrator role. For more information, see Security of agent pools.
:::moniker range="<=azure-devops"
:::image type="content" source="media/maintenance-job-settings.png" alt-text="Maintenance job settings":::
:::moniker-end
Configure your desired settings and choose Save.
Select Maintenance History to see the maintenance job history for the current agent pool. You can download and review logs to see the cleaning steps and actions taken.
:::moniker range="<=azure-devops"
:::image type="content" source="media/maintenance-job-history.png" alt-text="Maintenance job history":::
:::moniker-end
The maintenance is done per agent pool, not per machine; so if you have multiple agent pools on a single machine, you may still run into disk space issues.
Typically, a maintenance job gets "stuck" when it's waiting to run on an agent that is no longer in the agent pool. This happens when, for example, the agent has been purposefully taken offline or when there are issues communicating with it.
Maintenance jobs that have been queued to run will wait seven days to run. Afterward, they'll be automatically set to failed state if not run. This time limit cannot be changed.
The seven-day limit is different from the maintenance job timeout setting. The latter controls the maximum number of minutes an agent can spend doing maintenance. The timer starts when the job starts, not when the job is queued on an agent.
I'm trying to create a project agent pool that uses an existing organization agent pool, but the controls are grayed out. Why?
On the 'Create a project agent pool' dialog box, you can't use an existing organization agent pool if it is already referenced by another project agent pool. Each organization agent pool can be referenced by only one project agent pool within a given project collection.
::: moniker range="azure-devops"
Ask the owner of your Azure DevOps organization to grant you permission to use the pool. See Security of agent pools.
::: moniker-end
::: moniker range="azure-devops"
A: The Azure Pipelines pool provides all Azure DevOps organizations with cloud-hosted build agents and free build minutes each month. If you need more Microsoft-hosted build resources, or need to run more jobs in parallel, then you can either:
::: moniker-end