Make auto-renew use built-in renew function #3392

stevecrozz · 2023-12-15T00:27:49Z

As a complete newcomer to this project, this is my attempt to fix certificate auto-renew issues #1916 where manual renew works, but automatic renew does not. This changes proposes we replace the functionality inside automatic renewal with a call to the same function used for manual renewal.

Specifically, I have replaced the certbot 'renew everything' command with a DB-driven one-by-one renewal. Determining which certificates need renewal is a pretty naive time based DB query, and access control is completely bypassed with Promise.resolve({ permission_visibility: 'all' } because I assume it isn't needed for a background job.

I'm not sure yet how to best contribute to this project, and I'm sure there are some issues with this so please do guide me as I'm brand new here.

stevecrozz · 2023-12-15T00:29:53Z

backend/internal/certificate.js

@@ -30,6 +30,7 @@ const internalCertificate = {
 	intervalTimeout:    1000 * 60 * 60, // 1 hour
 	interval:           null,
 	intervalProcessing: false,
+	renewBeforeExpirationBy: [7, 'days'],


According to the certbot docs, renew (which is what was being used prior) will find all certificates that expire within 30 days. Maybe I should change this to 30 days?

I would prefer 30 days to avoid expiration notification emails from LE.

flightsupport · 2024-01-02T20:46:06Z

I tested this image. I think there are still some issues. I have 3 certificates, all needed to be replaced.
Only 1 certificate got replaced.
Then I restarted the container, then the next certificate was replaced.
Here is the log output after the first restart (2 remaining certificates):

[1/2/2024] [8:38:15 PM] [SSL      ] › ℹ  info      Renewing Let'sEncrypt certificates via Route 53 (Amazon) for Cert #3: sub.domain.tld
[1/2/2024] [8:38:15 PM] [SSL      ] › ℹ  info      Command: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-3' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-3" --disable-hook-validation --no-random-sleep-on-renew
[1/2/2024] [8:38:15 PM] [SSL      ] › ℹ  info      Renewing Let'sEncrypt certificates via Route 53 (Amazon) for Cert #4: sub2.domain.tld
[1/2/2024] [8:38:15 PM] [SSL      ] › ℹ  info      Command: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-4' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-4" --disable-hook-validation --no-random-sleep-on-renew
[1/2/2024] [8:38:16 PM] [SSL      ] › ✖  error     Command failed: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-4' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-4" --disable-hook-validation --no-random-sleep-on-renew
Another instance of Certbot is already running.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-2px2j2lx/log or re-run Certbot with -v for more details.

[1/2/2024] [8:39:07 PM] [SSL      ] › ℹ  info      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-3.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for sub.domain.tld

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded:
  /etc/letsencrypt/live/npm-3/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[1/2/2024] [8:39:07 PM] [SSL      ] › ✖  error     Cannot read properties of undefined (reading 'getUserId')`

stevecrozz · 2024-01-02T21:53:48Z

@flightsupport thanks for testing! This makes me think I'll need to run these renew calls in sequence instead of in parallel. It seems like they have to run one-by-one. I actually don't have a good test environment to verify, but I will attempt this change anyway.

flightsupport · 2024-01-07T19:28:48Z

I pulled the latest image. Now the renewals work.
I had to change the renewBeforeExpirationBy var to 86 to get my certificates renewed.

However for every certficate there's still an error (might by unrelated):

Cannot read properties of undefined (reading 'getUserId')

[1/7/2024] [7:23:06 PM] [SSL      ] › ℹ  info      Renewing Let'sEncrypt certificates via Route 53 (Amazon) for Cert #4: domain.tld
[1/7/2024] [7:23:06 PM] [SSL      ] › ℹ  info      Command: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-4' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-4" --disable-hook-validation --no-random-sleep-on-renew 
[1/7/2024] [7:23:15 PM] [SSL      ] › ℹ  info      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-4.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for domain.tld
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded: 
  /etc/letsencrypt/live/npm-4/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[1/7/2024] [7:23:15 PM] [SSL      ] › ✖  error     Cannot read properties of undefined (reading 'getUserId')

flightsupport · 2024-01-10T09:57:57Z

@stevecrozz did you have a look at the error to see if it is related to this PR?
Other than that, I hope this PR gets merged soon. Thanks for your work 👍

stevecrozz · 2024-01-10T16:42:52Z

@stevecrozz did you have a look at the error to see if it is related to this PR? Other than that, I hope this PR gets merged soon. Thanks for your work 👍

I think it probably is related to my work here unfortunately. I need to step through with a debugger because I don't see the issue yet. One simpler thing you could try if you feel like it is to remove that 'catch' clause and see if you can get a real stack trace.

jc21 · 2024-01-10T22:17:18Z

I can confirm that this error is being caused by this change and because it's throwing, some expected code paths are not being taken. I'll leave notes on the affected file soon.

jc21 · 2024-01-10T22:19:44Z

backend/internal/certificate.js

@@ -46,58 +47,45 @@ const internalCertificate = {
 			internalCertificate.intervalProcessing = true;
 			logger.info('Renewing SSL certs close to expiry...');


Can you please change this line to:

logger.info('Renewing SSL certs expiring within ' + internalCertificate.renewBeforeExpirationBy[0] + ' ' + internalCertificate.renewBeforeExpirationBy[1] + ' ...');

backend/internal/certificate.js

nginxproxymanagerci · 2024-01-11T04:31:13Z

Docker Image for build 5 is available on DockerHub as jc21/nginx-proxy-manager:github-pr-3392

Note: ensure you backup your NPM instance before testing this PR image! Especially if this PR contains database changes.

flightsupport · 2024-01-11T20:16:09Z

Test with build 5.
Everything seems fine now. The renewed certs even show up in the audit log. That was missing before.

[1/11/2024] [8:11:01 PM] [SSL      ] › ℹ  info      Renewing Let'sEncrypt certificates via Route 53 (Amazon) for Cert #2: sub1.domain.tld
[1/11/2024] [8:11:01 PM] [SSL      ] › ℹ  info      Command: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-2' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-2" --disable-hook-validation --no-random-sleep-on-renew 
[1/11/2024] [8:11:10 PM] [SSL      ] › ℹ  info      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-2.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for sub1.domain.tld

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded: 
  /etc/letsencrypt/live/npm-2/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[1/11/2024] [8:11:10 PM] [SSL      ] › ℹ  info      Renewing Let'sEncrypt certificates via Route 53 (Amazon) for Cert #3: sub2.domain.tld
[1/11/2024] [8:11:10 PM] [SSL      ] › ℹ  info      Command: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-3' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-3" --disable-hook-validation --no-random-sleep-on-renew 
[1/11/2024] [8:11:20 PM] [SSL      ] › ℹ  info      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-3.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for sub2.domain.tld

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded: 
  /etc/letsencrypt/live/npm-3/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[1/11/2024] [8:11:20 PM] [SSL      ] › ℹ  info      Renewing Let'sEncrypt certificates via Route 53 (Amazon) for Cert #4: sub3.domain.tld
[1/11/2024] [8:11:20 PM] [SSL      ] › ℹ  info      Command: AWS_CONFIG_FILE='/etc/letsencrypt/credentials/credentials-4' certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-4" --disable-hook-validation --no-random-sleep-on-renew 
[1/11/2024] [8:11:28 PM] [SSL      ] › ℹ  info      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-4.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for sub3.domain.tld

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded: 
  /etc/letsencrypt/live/npm-4/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[1/11/2024] [8:11:29 PM] [SSL      ] › ℹ  info      Completed SSL cert renew process

stevecrozz commented Dec 15, 2023

View reviewed changes

stevecrozz mentioned this pull request Dec 15, 2023

Auto SSL certificate renewal failing using AWS Route53 DNS #1916

Open

stevecrozz force-pushed the auto-renew-uses-bulitin-renew branch from 57e70bb to c17273a Compare December 21, 2023 04:17

flightsupport approved these changes Jan 10, 2024

View reviewed changes

jc21 reviewed Jan 10, 2024

View reviewed changes

backend/internal/certificate.js Show resolved Hide resolved

jc21 reviewed Jan 10, 2024

View reviewed changes

backend/internal/certificate.js Show resolved Hide resolved

stevecrozz added 3 commits January 10, 2024 20:08

Make auto-renew use built-in renew function

fe4bd9f

Run renews sequentially

f7d1c49

Provide the token model for certificate renewal

9c54d1b

stevecrozz force-pushed the auto-renew-uses-bulitin-renew branch from d2149af to 9c54d1b Compare January 11, 2024 04:08

jc21 merged commit 1be87f4 into NginxProxyManager:develop Jan 12, 2024

poiromaniax mentioned this pull request Jun 10, 2024

Update addon based on latest NPM release hassio-addons/addon-nginx-proxy-manager#579

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make auto-renew use built-in renew function #3392

Make auto-renew use built-in renew function #3392

stevecrozz commented Dec 15, 2023

stevecrozz Dec 15, 2023 •

edited

Loading

flightsupport Jan 2, 2024 •

edited

Loading

flightsupport commented Jan 2, 2024

stevecrozz commented Jan 2, 2024

flightsupport commented Jan 7, 2024 •

edited

Loading

flightsupport commented Jan 10, 2024

stevecrozz commented Jan 10, 2024

jc21 commented Jan 10, 2024

jc21 Jan 10, 2024

nginxproxymanagerci bot commented Jan 11, 2024

flightsupport commented Jan 11, 2024

		@@ -46,58 +47,45 @@ const internalCertificate = {
		internalCertificate.intervalProcessing = true;
		logger.info('Renewing SSL certs close to expiry...');

Make auto-renew use built-in renew function #3392

Make auto-renew use built-in renew function #3392

Conversation

stevecrozz commented Dec 15, 2023

stevecrozz Dec 15, 2023 • edited Loading

Choose a reason for hiding this comment

flightsupport Jan 2, 2024 • edited Loading

Choose a reason for hiding this comment

flightsupport commented Jan 2, 2024

stevecrozz commented Jan 2, 2024

flightsupport commented Jan 7, 2024 • edited Loading

flightsupport commented Jan 10, 2024

stevecrozz commented Jan 10, 2024

jc21 commented Jan 10, 2024

jc21 Jan 10, 2024

Choose a reason for hiding this comment

nginxproxymanagerci bot commented Jan 11, 2024

flightsupport commented Jan 11, 2024

stevecrozz Dec 15, 2023 •

edited

Loading

flightsupport Jan 2, 2024 •

edited

Loading

flightsupport commented Jan 7, 2024 •

edited

Loading