Elasticsearch starter forces use of Log4j2, breaking logging in apps that try to use Logback

[staleks]

I'm trying to run spring-boot-starter-data-elasticsearch in latest Milestone 2.0.0.M5.

I've used project template generated from start.spring.io.
Here is the GitHub repo url: https://github.com/staleks/spring-boot-2.0.M5-ES

Run

1. $ ./gradlew clean build
2. $ ./gradlew bootRun

stale the process of loading application context.
Please check the attached image: https://github.com/staleks/spring-boot-2.0.M5-ES/blob/master/images/staled-bootRun-process.png

On the other hand, if I switch back to 1.5.8.RELEASE version
(GitHub repo url: https://github.com/staleks/spring-boot-1.5.8-ES)

Run

1. $ ./gradlew clean build
2. $ ./gradlew bootRun

application context is loaded, and web application is started.
Please check the attached image: https://github.com/staleks/spring-boot-1.5.8-ES/blob/master/images/bootRun-process.png

Can someone please verify this?

Thank You

[wilkinsona]

Thanks for the sample. The application starts successfully, it's just some logging that's lost. That appears to be due to the fact that you have both Logback and Log4j2 on the classpath. Things work as expected if you exclude Log4j2:

compile('org.springframework.boot:spring-boot-starter-data-elasticsearch') {
    exclude group: 'org.apache.logging.log4j'
}

I'm not sure why Spring Data Elasticsearch pulls in all of Log4j2 rather than just a logging API. That should either be changed in Spring Data Elasticsearch or we should consider adding some exclusions in our starter.

@olivergierke @mp911de Can you please advise?

[odrotbohm]

That's a bug in the POM. Feel free to exclude in the starter for the upcoming milestone but we're gonna get this fixed with the next bugfix release for Kay.

[staleks]

Confirmed!
If log4j is excluded from starter-data-es then application context is loaded correctly.

Thank You for clarification and help.

Note: I'm trying to use new spring-data-elasticsearch v3.0.0.RELEASE in order to connect to ES 5.x.
Therefore I'm experimenting with spring-boot-starter 2.0.0.M5.

[staleks]

"Two cents" more on this topic :), as it seams not solved just by exclusion.

Try to add ElasticSearch Configuration class (JavaConfig), e.g.

@Configuration
public class ESConfig {
	
	@Bean
	public Client client() throws UnknownHostException {
		TransportClient esClient = new PreBuiltTransportClient(Settings.EMPTY);
		esClient.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9200));
		return esClient;
	}
	
	public ElasticsearchOperations esTemplate() throws UnknownHostException {
		return new ElasticsearchTemplate(client());
	}
	
}

Now, starting application will "complain" about missing log4j.

Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.Level
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_131]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_131]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[na:1.8.0_131]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_131]
	... 45 common frames omitted

In order to help you debug this, please checkout GitRepo:
https://github.com/staleks/spring-boot-2.0.M5-ES and switch to development branch.

Note: org.apache.logging.log4j.Logger introduced in
org.elasticsearch.common.component.AbstractComponent of elasticsearch-5.5.3.jar

[philwebb]

@staleks Can you post the full stack trace please.

[staleks]

@philwebb sure,

Here is complete DEBUG log, on application context loading:
https://github.com/staleks/spring-boot-2.0.M5-ES/blob/development/logs/spring-boot-2.0.0.M5-ES-debug.txt

You can find project at:
https://github.com/staleks/spring-boot-2.0.M5-ES/tree/development

[odrotbohm]

I guess that's the reason, the Log4j dependencies made it into the compile scope of Spring Data ES in the first place. I tried to remove the dependencies completely as we pull in Logback in the Spring Data builds during tests anyway but that causes the tests to fail with a similar exception. It looks like ES doesn't properly run without Log4j being used.

[wilkinsona]

I believe that can be avoided with a runtime dependency on org.apache.logging.log4j:log4j-to-slf4j.

@staleks For some reason that module is not part of Log4j2's bom (so Boot doesn't currently manage its version). However, you can align its version with the version of the log4j-api module (which Boot does manage) like this:

runtime("org.apache.logging.log4j:log4j-to-slf4j:${dependencyManagement.managedVersions['org.apache.logging.log4j:log4j-api']}")

In summary, I think the long term fix for this is for Spring Data Elasticsearch to exclude any Log4j2 implementation dependencies and pull in org.apache.logging.log4j:log4j-to-slf4j instead. As a temporary measure for Boot M6, we can create the same arrangement by updating spring-boot-starter-data-elasticsearch.

[wilkinsona]

I've opened LOG4J2-2093 to see if log4j-to-slf4j could be added to Log4j 2's bom.

[odrotbohm]

Thanks, Andy. I’ll proceed as suggested and ping you once the fix is available.

[spencergibb]

@wilkinsona This change caused a test to fail in spring-cloud-gateway. See https://jenkins.spring.io/blue/organizations/jenkins/spring-cloud-gateway-2.0.x-ci/detail/spring-cloud-gateway-2.0.x-ci/1019/pipeline

Adding the following exclusion to the pom fixed the error. Seems a bit bizarre to me:

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.apache.logging.log4j</groupId>
					<artifactId>log4j-to-slf4j</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

[wilkinsona]

It appears that the test failures are due to a bug in ModifiedClassPathRunner which is causing Provider to be loaded by two different class loaders. Let me know if fixing Spring Cloud’s copy of the runner doesn’t help and we can take another look.

[spencergibb]

Adding changes you made to the boot version (excluding log4j jars) fixed the problem.

[wilkinsona]

Aah, I'd forgotten I'd had to do that. Sorry. Could have saved us all some cycles if my memory was a bit better…

[odrotbohm]

The reductions of the scope of the log4j dependencies for Spring Data Elasticsearch have now been applied to the Lovelace and Kay release train branches. Snapshots should be available shortly.