spring.datasource.hikari.data-source-class-name cannot be used as a driver class name is always required and Hikari does not accept both

[michael-simons]

Given a valid javax.sql.DataSource implementation I expect to be able to used it as type in

spring.datasource.type

to configure my application.

However, this fails:

registry.add("spring.datasource.type", MyDataSource.class::getCanonicalName);
registry.add("spring.datasource.driver-class-name", MyDriver.class::getCanonicalName);
registry.add("spring.datasource.url", () ->
	"jdbc:vendor://localhost"
);
registry.add("spring.datasource.username", () -> "foo");
registry.add("spring.datasource.password", () -> "bar");

with

Caused by: org.springframework.boot.jdbc.UnsupportedDataSourcePropertyException: Unable to find suitable method for url
	at app//org.springframework.boot.jdbc.UnsupportedDataSourcePropertyException.throwIf(UnsupportedDataSourcePropertyException.java:36)
	at app//org.springframework.boot.jdbc.DataSourceBuilder$ReflectionDataSourceProperties.getMethod(DataSourceBuilder.java:581)
	at app//org.springframework.boot.jdbc.DataSourceBuilder$ReflectionDataSourceProperties.set(DataSourceBuilder.java:563)
	at app//org.springframework.boot.jdbc.DataSourceBuilder.build(DataSourceBuilder.java:184)
	at app//org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration.createDataSource(DataSourceConfiguration.java:59)

I can of course not omit the URL, as this is required for the connection.

However, setUrl is not defined part of the DataSource interface, yet the builder insists to propagate all non-null properties from the datasource properties in DataSourceBuilder#build.

It will not fail on Hikari because there's dedicated config class.

Reproducer is attached.

@SpringBootTest
class DemoApplicationTests {

	@DynamicPropertySource
	static void postgresqlProperties(DynamicPropertyRegistry registry) {
		registry.add("spring.datasource.type", MyDataSource.class::getCanonicalName);
		registry.add("spring.datasource.driver-class-name", MyDriver.class::getCanonicalName);
		registry.add("spring.datasource.url", () ->
			"jdbc:vendor://localhost"
		);
		registry.add("spring.datasource.username", () -> "foo");
		registry.add("spring.datasource.password", () -> "bar");
	}

	@Test
	void contextLoads() {
	}

}

demo.zip

[michael-simons]

Hm. It seems the way forward is to actually just instantiate any custom datasource I want, and maybe using the properties to hold common stuff. (*thinking loud here)

What I actually want to do is this in configuration:

@TestConfiguration
	static class F {

		@Bean
		public DataSource neo4jDataSource(
			DataSourceProperties dataSourceProperties,
			@Value("${spring.datasource.hikari.maximum-pool-size}") int maximumPoolSize) {

			var neo4jDataSource = new Neo4jDataSource();
			neo4jDataSource.setUrl(dataSourceProperties.getUrl());
			neo4jDataSource.setPassword(dataSourceProperties.getPassword());
			neo4jDataSource.setUser(dataSourceProperties.getUsername());

			// Create pool
			var hikariConfig = new HikariConfig();
			hikariConfig.setMaximumPoolSize(maximumPoolSize);
			hikariConfig.setDataSource(neo4jDataSource);

			return new HikariDataSource(hikariConfig);
		}
	}

aka creating a nested data source.

So I can just add setUrl to our datasource, no problem.

And I can use spring.datasource.hikari.data-source-class-name to tell Hikari the data source I want to use .

But, the Spring Boot Datasource builder requires the driver and will apply it to the Hikari config, and that is not valid (Hikari will complain about using both driver class and data source)

[wilkinsona]

Thanks for the report, @michael-simons. This use case – using a connection pool with a custom nested DataSource – is something that we haven't really considered thus far.

As you've observed, the auto-configuration really wants to set a driver class name on the connection pool. This logic is in determineDriverClassName() of DataSourceProperties and there's no way at the moment to get it to return null and prevent the driver class name from being set as it'll throw an exception when it's unable to determine one.

The problem's related to the configuration properties being applied in two passes such that the first pass (that applies general properties) knows nothing about the second pass (that applies Hikari-specific properties). This makes it hard for it to know to ignore a missing driver class name because a data source class name is going to be applied.

I wondered if we could work around the problem by using a bean post-processor to clear the driver class name when the data source class name has been set. Unfortunately that does not work as Hikari throws an exception when the driver class name is set to null or an empty string.

The least bad option that I've found thus far is to use a bean post-processor to set the DataSource when the data source class name has been set:

@Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
	if (bean instanceof HikariDataSource dataSource) {
		String dataSourceClassName = dataSource.getDataSourceClassName();
		if (dataSourceClassName != null) {
			try {
				Class<? extends DataSource> dataSourceClass = (Class<? extends DataSource>) ClassUtils
					.forName(dataSourceClassName, getClass().getClassLoader());
				dataSource.setDataSource(BeanUtils.instantiateClass(dataSourceClass));
			}
			catch (Exception ex) {
				throw new RuntimeException(ex);
			}
		}
	}
	return bean;
}

I don't think this is something you should do in your app (I think it would be better just to configure the data source manually), but perhaps it'll help us to come up with a better way to support this out of the box.

[wilkinsona]

We think we might be able to change the code that creates the Hikari DataSource to be more lenient about the need for a driver class name, possibly by passing a flag to the private createDataSource method. We may need to catch an exception that's thrown when calling getDriverClassName() on the connection details or overload that method to change how it behaves.

[michael-simons]

Hey Andy. I was on that code as well… Prior to the checks on the driver name, you mentioned here

As you've observed, the auto-configuration really wants to set a driver class name on the connection pool. This logic is in determineDriverClassName() of DataSourceProperties and there's no way at the moment to get it to return null and prevent the driver class name from being set as it'll throw an exception when it's unable to determine one.

I think it would be a good compromise.

WRT Bean postprocessor: I'd rather not do this in my app either :) I'd rather just wrap my datasource than in Hikari manually.

Thanks a lot for the feedback / input. 🙇

[wilkinsona]

This test, added to HikariDataSourceConfigurationTests, fails because a suitable driver class could not be determined:

@Test
@ClassPathExclusions({ "h2-*.jar", "hsqldb-*.jar" })
void configureDataSourceClassNameWithNoEmbeddedDatabaseAvailable() {
	this.contextRunner
		.withPropertyValues("spring.datasource.url=jdbc:example//",
				"spring.datasource.hikari.data-source-class-name=example.ExampleDataSource")
		.run((context) -> {
			HikariDataSource ds = context.getBean(HikariDataSource.class);
			ds.getConnection();
		});
}

This test fails because Hikari does not support setting both a driver class name and a DataSource class name:

@Test
void configureDataSourceClassNameToOverrideUseOfAnEmbeddedDatabase() {
	this.contextRunner
		.withPropertyValues("spring.datasource.url=jdbc:example//",
				"spring.datasource.hikari.data-source-class-name=example.ExampleDataSource")
		.run((context) -> {
			HikariDataSource ds = context.getBean(HikariDataSource.class);
			ds.getConnection();
		});
}

A test where the driver class name can be determined from the JDBC URL would also fail.

[wilkinsona]

@michael-simons with the changes made here and for #44994, you should now be able to configure Neo4jDataSource using properties alone:

spring.datasource.url=jdbc://anything-you-want
spring.datasource.hikari.data-source-class-name=org.neo4j.jdbc.Neo4jDataSource
spring.datasource.hikari.data-source-properties.serverName=localhost
spring.datasource.hikari.data-source-properties.portNumber=12345

spring.datasource.url is still needed to prevent an attempt to determine the URL.

Should a setUrl method be added to Neo4jDataSource, something like this should then work:

spring.datasource.url=jdbc:neo4j://localhost:12345
spring.datasource.hikari.data-source-class-name=org.neo4j.jdbc.Neo4jDataSource

[michael-simons]

That is wonderful. I indeed added such an extension already.

FWIW the background of this, I do maintain now the JDBC driver for Neo4j for a while and I chose to use the DataSource for users wanting to customise their driver, i.e. for example by adding tracing.

I will add those bits in https://github.com/neo4j/neo4j-jdbc/pull/904/files#diff-142a855adfd1b5cf732debc1bd3c150d33f68b10d2d7160bb522797d73f393eb than, too.