Support Monitoring of HttpClient with HttpComponentsClientHttpRequestFactoryBuilder

[danielrohe]

In Spring Boot 3.4 the HttpComponentsClientHttpRequestFactoryBuilder used by RestClient does not provide the capability to add monitoring to the HttpClient's connection pool using micrometers PoolingHttpClientConnectionManagerMetricsBinder. This leads to blindness for utilization of the underlying connection pool.

[nosan]

does not provide the capability to add monitoring to the HttpClient's connection pool

You can configure your own ClientHttpRequestFactoryBuilder and PoolingHttpClientConnectionManagerMetricsBinder beans.

@Configuration(proxyBeanMethods = false)
public class ClientHttpConfiguration {

    @Bean
    PoolingHttpClientConnectionManager poolingHttpClientConnectionManager() {
        return PoolingHttpClientConnectionManagerBuilder.create().useSystemProperties().build();
    }

    @Bean
    ClientHttpRequestFactoryBuilder<?> httpComponentsClientHttpRequestFactoryBuilder(
            PoolingHttpClientConnectionManager poolingHttpClientConnectionManager) {
        return ClientHttpRequestFactoryBuilder.httpComponents()
                .withHttpClientCustomizer(
                        (builder) -> builder.setConnectionManager(poolingHttpClientConnectionManager)
                                .setConnectionManagerShared(true));
    }

    @Bean
    MeterBinder poolingHttpClientConnectionManagerMetricsBinder(
            PoolingHttpClientConnectionManager poolingHttpClientConnectionManager) {
        return new PoolingHttpClientConnectionManagerMetricsBinder(poolingHttpClientConnectionManager,
                "httpComponents.pool");
    }

}

...
  
    "httpcomponents.httpclient.pool.route.max.default",
    "httpcomponents.httpclient.pool.total.connections",
    "httpcomponents.httpclient.pool.total.max",
    "httpcomponents.httpclient.pool.total.pending",
...

Unfortunately, you'll need to manually configure all the settings for PoolingHttpClientConnectionManager, such as SocketConfig and TlsSocketStrategy.

[nosan]

The above code configures a shared PoolingHttpClientConnectionManager for all RestClient. If this is not the desired behavior, consider using the following configuration:

@Configuration(proxyBeanMethods = false)
class ClientHttpConfiguration {

    private final AtomicInteger connectionPoolCounter = new AtomicInteger();

    private final MeterRegistry meterRegistry;

    ClientHttpConfiguration(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
    }

    @Bean
    ClientHttpRequestFactoryBuilder<?> clientHttpRequestFactoryBuilder() {
        return ClientHttpRequestFactoryBuilder.httpComponents()
                .withHttpClientCustomizer(
                        (builder) -> builder.setConnectionManager(getPoolingHttpClientConnectionManager()));
    }

    private PoolingHttpClientConnectionManager getPoolingHttpClientConnectionManager() {
        PoolingHttpClientConnectionManager connectionManager = PoolingHttpClientConnectionManagerBuilder.create()
                .useSystemProperties()
                .build();
        new PoolingHttpClientConnectionManagerMetricsBinder(connectionManager,
                "pool-" + this.connectionPoolCounter.incrementAndGet()).bindTo(this.meterRegistry);
        return connectionManager;
    }

}

{
  "name": "httpcomponents.httpclient.pool.total.connections",
  "description": "The number of persistent and leased connections for all routes.",
  "measurements": [
    {
      "statistic": "VALUE",
      "value": 20
    }
  ],
  "availableTags": [
    {
      "tag": "state",
      "values": [
        "available",
        "leased"
      ]
    },
    {
      "tag": "httpclient",
      "values": [
        "pool-2",
        "pool-1"
      ]
    }
  ]
}

[nosan]

I am curious if the HttpComponentsClientHttpRequestFactoryBuilder could be enhanced with the following method:

public HttpComponentsClientHttpRequestFactoryBuilder withConnectionManagerPostConfigurer(
		Consumer<PoolingHttpClientConnectionManager> connectionManagerPostConfigurer) {
	...
}

In that case, it would be possible to have something like this:

@Configuration(proxyBeanMethods = false)
class ClientHttpConfiguration {

    private final AtomicInteger counter = new AtomicInteger();

    private final MeterRegistry meterRegistry;

    ClientHttpConfiguration(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
    }

    @Bean
    ClientHttpRequestFactoryBuilder<?> clientHttpRequestFactoryBuilder() {
        return ClientHttpRequestFactoryBuilder.httpComponents()
                .withConnectionManagerPostConfigurer(this::bindToMeterRegistry);
    }

    private void bindToMeterRegistry(PoolingHttpClientConnectionManager connectionManager) {
        new PoolingHttpClientConnectionManagerMetricsBinder(connectionManager, "pool-" + this.counter.getAndIncrement())
                .bindTo(this.meterRegistry);
    }

}

I’ve prototyped some changes: main...nosan:spring-boot:44643

This solution will not work if someone overrides PoolingHttpClientConnectionManager via Consumer<HttpClientBuilder> httpClientCustomizer:

ClientHttpRequestFactoryBuilder.httpComponents()
                .withHttpClientCustomizer(
                        (builder) -> builder.setConnectionManager(new PoolingHttpClientConnectionManager()));

[nosan]

I’ve prototyped some changes: main...nosan:spring-boot:44643

I am not sure about HttpClientMetricsAutoConfiguration; it seems a bit fragile. It works only
when no one overrides the PoolingHttpClientConnectionManager through HttpClientBuilder.
Otherwise, HttpComponentsClientHttpRequestFactoryBuilderMetricsPostProcessor binds the wrong
PoolingHttpClientConnectionManager to the MeterRegistry which leads to NaN metrics.