-
Notifications
You must be signed in to change notification settings - Fork 205
/
Copy pathRELEASE_NOTES
3112 lines (2775 loc) · 199 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This file contains a description of the major changes to the easybuild-framework EasyBuild package.
For more detailed information, please see the git log.
These release notes can also be consulted at https://docs.easybuild.io/release-notes .
v5.0.0 (18 March 2025)
----------------------
- remove support for Python 2.7 and 3.5 (#4229, #4270, #4306, #4473, #4477, #4476, #4478, #4524, #4607, #4756, #4810, #4811)
- also run unit test suite with Python 3.12 + 3.13 (#4484, #4674)
- changed defaults in EasyBuild configuration:
- enable --trace by default (#4250, #4310, #4311, #4491)
- enable RPATH linking by default (#4448, #4515, #4779)
- disable strict RPATH sanity check by default, allow re-enabling it via `--strict-rpath-sanity-check` configuration option (#4475)
- enable `module-depends-on` by default (#4500)
- enable `module-extensions` by default (#4501)
- set default maximum level of parallellism to `16` via `max-parallel` configuration option (#4606, #4777, #4816)
- use Slurm as the default job backend (#4659)
- enable keeping of symbolic links by default when copying (set `keepsymlinks` easyconfig parameter to `True`) (#4773)
- changed behaviour:
- change default for `change_into_dir` option in `extract_file` function to to `False` (#4246)
- enforce correct `.patch(.*)` extension for patch files (#4247)
- create `lib` -> `lib64` symlink (or vice versa) *before* running `postinstallcmds` (#4435)
- don't allow unresolved templates in easyconfig parameters by default + add support for `--allow-unresolved-templates` configuration option (#4516, #4725, #4726, #4727)
- switch default checksum to `sha256` (#4523)
- only allow use of `rpath` toolchain option when `system` toolchain is used (#4585, #4808)
- use default value `$XDG_CONFIG_DIRS` from XDG basedir spec: `/etc/xdg` (instead of `/etc`) (#4591)
- reverse order for parsing files in `XDG_CONFIG_DIRS` (#4630)
- move verifying of checksums from `source` to `fetch` step, to include it with `--fetch` (#4624, #4729)
- drop support for pep8 package (was used for `--check-contrib` + `--check-style`) (#4634)
- specify changes that should be made by generated module files via `module_load_environment` attribute of `EasyBlock` class (#4653, #4754, #4761, #4774, #4799, #4784, #4801, #4802, #4803, #4809)
- let jobs retweak easyconfigs themselves by passing down `--try-*` options (#4669)
- refactor `make_extension_string` method in `EasyBlock` class (#4690)
- change `Toolchain.get_flag` so it doesn't automatically prepend a dash (`-`) to compiler flags (#4698)
- rename `Compiler.COMPILER*_FLAGS` to `Compiler.COMPILER*_OPTIONS` (#4698)
- change semantics of `--dry-run`, so it doesn't imply `--robot` (#4704)
- run sanity checks commands from an empty temporary directory (rather than the software install directory) (#4723)
- add context manager for allowing unresolved templates and make the state members private (#4735)
- also remove support for directly setting `enable_templating` and `expect_resolved_template_values`
- various enhancements:
- `run_shell_cmd` function to run shell commands which replaces both the (now deprecated) `run_cmd` and `run_cmd_qa` functions (#4284, #4314, #4321, #4322, #4327, #4334, #4335, #4336, #4351, #4356, #4378, #4380, #4383, #4390, #4422, #4423, #4427, #4428, #4430, #4431, #4432, #4441, #4443, #4444, #4453, #4454, #4471, #4504, #4509, #4544, #4612, #4617, #4664, #4721, #4728, #4731, #4755, #4757)
- detect Fortran `.mod` files for installations using `GCCcore` toolchain (#4389)
- create `env.sh` and `cmd.sh` helper scripts in `run_shell_cmd` to allow starting interactive shell to debug failing shell commands (#4486, #4611, #4662, #4666, #4685, #4792)
- add support for alternate easyconfig parameters/templates/constants (#4511, #4514, #4549, #4555)
- create reproducible tarballs for sources created via `git_config` (requires `.tar.xz` + Python 3.9+) (#4248, #4517, #4522, #4660, #4733, #4797, #4798, #4813)
- use more granular exit codes when `EasyBuildError` is raised (#4534)
- prepend to `$PYTHONPATH` or `$EBPYTHONPREFIXES` in generated module files by automatically scanning for Python site package directories (as configured via `prefer-python-search-path`) (#4539, #4686)
- copy build directory and/or log file(s) if installation failed to path specified via `--failed-install-build-dirs-path` or `--failed-install-logs-path` (#4601)
- add `--search-path-cpp-headers` configuration option to control how EasyBuild sets paths to headers at build time (#4645)
- add `module-search-path-headers` configuration option to control how modules set search paths to header files (#4655)
- add `--keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option (#4688, #4764)
- add `--search-path-linker` option to control linker options at build time (#4697)
- don't raise error when required extensions are not found when installing extensions in parallel (#4671)
- mark support for installing extensions in parallel as being mature, since it's no longer experimental (#4672)
- mark easystack support as being mature, since it's no longer experimental (#4673)
- and several additional small enhancements:
- enhance download instructions by mentioning active source path (#4459)
- add CUDA compute capability integer format templates (`cuda_int_*_sep`) (#4463)
- add new `get_cwd` function to `tools.filetools` to retrieve current working directory (#4525)
- use `dict.items()` instead of repeatedly getting the value (#4533)
- set `usedforsecurity` to `False` when calling `hashlib.md5` with Python >= 3.9 (#4550)
- add `GNU_FTP_SOURCE` template constant (#4598)
- allow using `amend/try-amend` multiple times in an easystack file entry (#4667)
- add support for `%(rpath_enabled)s` template value (#4670)
- add `resolve_template` method to `EasyConfig` class (#4677)
- allow templates in `custom_paths` & `custom_commands` sanity-check arguments (#4679)
- updates and fixes for `findPythonDeps` script (#4682, #4740)
- allow use of custom delimiter for paths in module generator (#4687)
- enhance `get_software_libdir` to return full paths if requested (#4699)
- enhance `EasyBlock` class to allow passing in `logfile` (#4707)
- avoid checking loaded module twice in `findUpdatedEcs.sh` script (#4710)
- allow nesting values in checksum dicts (#4711) Flamefire:non-dict MERGED 2024-12-02T12:14:12Z
- use `enumerate` where applicable + fix for `ModuleGenerator._generate_multi_deps_list` (#4720)
- faster `nub` function (#4737)
- enhance `apply_regex_substitutions` to support use of multi-line patterns, requiring matching all patterns in each file, and use pre-compiled regular expressions (#4758)
- ignore other classes if software specific easyblock class was found (#4769)
- also allow trailing whitespaces for `examples` and `citing` easyconfig parameters (#4796)
- enhance `get_gpu_info` to also use `amd-smi` for AMD GPUs if possible (#4805)
- various changes, improvemnents, and fixes for the supported modules tools (Environment Modules + Lmod)
- drop load storm safe guard for Environment Modules v4.2.4+ (#4373)
- run unit tests on an updated versions of Environment Modules: v4.5.3 + v5.3.1 (#4415)
- add `check_group` support for module files in Tcl syntax (#4418)
- bump minimum required Lmod to 8.0.0 (#4424)
- bump minimum required Tmod (4.x) to 4.3.0 (#4425)
- use `getenv` modulefile command with `EnvironmentModules` >= 4.2.0 (#4614)
- add module cache build support in `EnvironmentModules` class (#4615)
- derive `EnvironmentModules` class directly from `ModulesTool` rather than from to be deprecated `EnvironmentModulesTcl` (#4625)
- adapt `module show` command run to cope with non-zero exit code for non-existing module (required for Environment Modules v5.5+ and Lmod 8.7.56+) (#4739)
- various improvements for the framework test suite:
- allow test case filtering by class name (#4788)
- improve test failure output for `assertEqual` (#4807)
- fix `test_modulerc` by taking into account that wrapper module may also be shown as being loaded (#4778)
- (temporarily) use 5.0.x branch for easyblocks + easyconfigs in CI workflows (#4358)
- fix GitHub Actions for CentOS 7.9 container (#4712)
- stop using Ubuntu 20.4 in GitHub Actions workflows, use Ubuntu 22.04 instead (#4783)
- update GitHub actions workflows to use `ubuntu-24.04` where possible (#4795)
- various small bug fixes:
- silence output of included easyblocks for `--terse` (#4765)
- avoid processing the same EasyConfig multiple times (#4767)
- fix easyconfig parameter deprecation (#4479, #4480)
- switch from `ls` to `bash` in tests that are expecting this to be a binary (#4492)
- fix the checksum type check (#4578)
- fix `to_checksums` with `None` values in dicts and recursion (#4579)
- fix `test_toy_lock_cleanup_signals` (#4600)
- resolve symlink when making log dir writable (#4658)
- fix typo in `veryloose` toolchain option for RISC-V (#4668)
- fix dry-run output when using `multi_deps` (#4678)
- make `LooseVersion('1.0') == LooseVersion('1')` (#4691)
- fix FFT entry in `--list-toolchains` output for Cray toolchains (#4719)
- avoid making build directory read-only (#4736)
- deprecated functionality:
- `easybuild.tools.py2vs3` module (#4229)
- rename unclear `*run*` methods to `*install_extension*` + rename `install_extensions` to `install_all_extensions` (#4400)
- deprecate `run_cmd` and `run_cmd_qa` & co, move them to `easybuild._deprecated` module (#4433)
- deprecate support for `EnvironmentModulesC` and `EnvironmentModulesTcl` module tools (#4439)
- deprecate old checksum types (incl. md5) (#4526, #4545)
- deprecate use of `parallel` easyconfig parameter and fix updating the template value (#4580)
- convert template constant lists to dicts and export the constants by name (#4595)
- rename '`source`' step to '`extract`' (affects `skipsteps` easyconfig parameter + `--stop` option) (#4629)
- deprecate `make_module_req_guess` method in `EasyBlock` class (#4653, #4763)
- deprecate support for GC3Pie as job backend (#4659)
- add deprecation warning for `optarch` value without leading dash (#4698)
- deprecate `post_install_step` method in `EasyBlock`, was renamed to `post_processing_step` (#4715)
- remove functionality that was deprecated in EasyBuild v4.x, including:
- EasyBuild bootstrap script (#4233)
- support for YAML-based easyconfig format (.yeb) (#4237)
- `wait-on-lock` configuration setting (#4239)
- `dummy` toolchain (#4240)
- `accept-eula` configuration setting (#4242)
- `is_generic_easyblock` function from `easybuild.framework.easyconfig.easyconfig` (#4243)
- `use_git_am` option for `apply_patch` function (#4244)
- `fetch_extension_sources` method in `EasyBlock` class (#4245)
- support for 32-bit targets (#4272)
- `descr` option for `simple_option` function (#4273)
- `Toolchain.add_dependencies` method (#4274)
- `copytree`, `rmtree2` functions from `easybuild.filetools` (#4275)
- `skip_symlinks` option for `adjust_permissions` function (#4275)
- `log_error` option from `which` function (#4276)
- `skip_lower` option from `template_constant_dict` (#4277)
- `disable_templating` + `default_fallback` options in `get_easyblock_class` (#4278)
- `mod_exists_regex_template` options in `ModulesTool.exist` (#4279)
- other changes
- take into account that `VERBOSE_VERSION` imported from `easybuild.easyblocks` is now a string value (#4357)
v4.9.4 (22 September 2024)
--------------------------
update/bugfix release
- various enhancements, including:
- set $LMOD_TERSE_DECORATIONS to 'no' to avoid additional info in output produced by 'ml --terse avail' (#4648)
- various bug fixes, including:
- implement workaround for permission error when copying read-only files that have extended attributes set and using Python 3.6 (#4642)
- take into account alternate sysroot for /bin/bash used by run_cmd (#4646)
v4.9.3 (14 September 2024)
--------------------------
update/bugfix release
- various enhancements, including:
- add support for `--extra-source-urls` to fetch sources from additional URLs (#4079)
- add definition for gmpflf toolchain (#4566, #4571)
- reuse pre-computed checksums (#4569)
- add `cuda_cc_space_sep` variant that does not have periods (#4583)
- add `--skip-sanity-check` option (#4590)
- add `GNU_FTP_SOURCE` template constant (#4597)
- improve error messages for empty easyconfigs (#4603)
- improve help string for `--dep-graph` (#4610)
- only call `_sanity_check_step_extensions` if `--skip-extensions` is not set (#4620)
- add support for `--software-commit` and an associated template `%(software_commit)s` (#4628)
- various bug fixes, including:
- correctly evaluate result for `--dep-graph` (#4554)
- fix fetch progress bar showing to many files (#4568)
- resolve internal for imkl>=2021 version subdir via "latest" symlink (#4570)
- fix typo in message about including an easyblock from a commit (#4575)
- don't use special flags for `strict`, `precise`, `loose`, `veryloose` toolchain options on RISC-V (#4576)
- fix help text for `cuda_compute_capabilities` template (#4589)
- fix help message for `--http-headers-fields-urlpat` configuration option (#4594)
- fix `test_compiler_cache` in case `gcc` is available multiple times (#4599)
- handle post-install patches in check_checksums_for (#4605)
- fix `copy_file` with a folder as the target (#4609)
- allow for case where `homepage = None` when generating the docs (#4626)
- fix test_github_det_commit_status by using more recent commits (#4636)
- other changes:
- clean up code that was only there to support Python 2.6 + avoid syntax warnings when parsing py2vs3/py.p2 with Python 3.x (#3788)
- use Intel's oneAPI Fortran compiler by default for version 2024.0.0 and newer (`oneapi_fortran` toolchain option set to `True`) (#4567)
- allow using Node 16 actions in CI (#4574)
- remove a superflous check in `EasyBlock.run_all_steps` (#4623)
- remove trailing dots from backup message produced by --inject-checksums (#4632)
v4.9.2 (12 June 2024)
---------------------
update/bugfix release
- various enhancements, including:
- improve behavior when using extension which has 'nosource' enabled (#4506)
- enhance 'get_software_libdir' to return 'lib' or 'lib64' if only one of them contains library files (#4513)
- implement versions checks to avoid mixing major versions across the EasyBuild components (#4520, #4553)
- add support for easyconfig parameter 'module_only' (#4537)
- various bug fixes, including:
- fix typo in patch_step logging (#4505)
- consider both 'easybuild-framework*.tar.gz' and 'easybuild_framework*.tar.gz' in CI workflows (#4507)
- don't delete existing environment module files when using '--dump-env-script' with '--force' or '--rebuild' (#4512)
- fix resolved (template) values in case of failure (#4532)
- also consider '$CRAY_PE_LIBSCI_PREFIX_DIR' to determine installation prefix for cray-libsci (#4551)
- symlink downloaded repo at specified commit when using '--from-commit' so easyconfigs for dependencies are found (#4552)
- other changes:
- code cleanup in 'easyblock.py' (#4519)
- stop running unit tests on Python 3.5 (#4530)
v4.9.1 (5 April 2024)
---------------------
update/bugfix release
- various enhancements, including:
- make `is_rpath_wrapper` faster by only checking file contents if file is not located in subdirectory of RPATH wrapper subdirectory (#4406)
- add terse support to `--missing-modules` (#4407)
- adapt version pattern for EnvironmentModules to allow using development version (#4416)
- use `--all` option with EnvironmentModules v4.6+ to get available hidden modules (#4417)
- add support for appending to path environment variables via `modextrapaths_append` + add corresponding `allow_append_abs_path` (#4436)
- improve output produced by `--check-github` (#4437)
- add script for updating local git repos with `develop` branch (#4438)
- show error when multiple PR options are passed (#4440)
- improve `findPythonDeps` script to recognize non-canonical package names (#4445)
- add support for `--from-commit` and `--include-easyblocks-from-commit` (#4468)
- improve logging & handling of (empty) `--optarch` values (#4481)
- add `--short` option to `findUpdatedEcs` script (#4488)
- add generic GCC and Clang compiler flags for RISC-V (#4489)
- various bug fixes, including:
- clean up log file of `EasyBlock` instance in `check_sha256_checksums` (#4452)
- fix description of `backup-modules` configuration option (#4456)
- replace `'` with `"` for `printf` in CI workflow for running test suite to have bash replace a variable (#4461)
- use `cp -dR` instead of `cp -a` for shell script "extraction" (#4465)
- fix link to documentation in `close_pr` message (#4466)
- fix `test_github_merge_pr` by using more recent easyconfigs PR (#4470)
- add workaround for 404 error when installing packages in CI workflow for testing Apptainer integration (#4472)
- other changes:
- clean up & speed up environment checks (#4409)
- use more performant and concise dict construction by using dict comprehensions (#4410)
- remove superflous string formatting (#4411)
- clean up uses of `getattr` and `hasattr` (#4412)
- update copyright lines to 2024 (#4494)
v4.9.0 (30 December 2023)
-------------------------
feature release
- various enhancements, including:
- allow tweaking of easyconfigs from different toolchains (#3669)
- add support for `--list-software --output-format=json` (#4152)
- add `--module-cache-suffix` configuration setting to allow multiple (Lmod) caches (#4403)
- various bug fixes, including:
- deduplicate warnings & errors found in logs and add initial newline + tab in output (#4361)
- fix support for Environment Modules as modules tool to pass unit tests with v4.2+ (#4369)
- adapt module function check for Environment Modules v4+ (#4371)
- only install GitHub token when testing with Lmod 8.x + Python 3.6 or 3.9 (#4375)
- use `-qopenmp` instead of `-fiopenmp` for OpenMP in Intel compilers (#4377)
- fix `LIBBLAS_MT` for FlexiBLAS, ensure `-lpthread` is included (#4379)
- relax major version match regex in `find_related_easyconfigs` using for `--review-pr` (#4385)
- eliminate duplicate multideps from generated module files (#4386)
- resolve templated values in extension names in `_make_extension_list` (#4392)
- use source toolchain version when passing only `--try-toolchain` (#4395)
- fix writing spider cache for Lmod >= 8.7.12 (#4402)
- fix `--inject-checksums` when extension specifies patch file in tuple format (#4405)
- fix `LooseVersion` when running with Python 2.7 (#4408)
- use more recent easyblocks PR in `test_github_merge_pr` (#4414)
- other changes:
- extend test that checks build environment to recent `foss/2023a` toolchain (#4391)
v4.8.2 (29 October 2023)
------------------------
update/bugfix release
- various enhancements, including:
- add support for `%(sysroot)s` template value (#4359)
- add `dependency_names` method to `EasyConfig` class to get set of names of (direct) dependencies (#4360)
- various bug fixes, including:
- add CI workflow to run unit tests with Python 2 (again) (#4333)
- fix typo in help message for `--silence-hook-trigger` (#4343)
- include major version (`*majver`) templates in auto-generated documentation (#4347)
- reset `tempfile.tempdir` to `None` to avoid that tmpdir path gets progressively deeper with each easystack item (#4350)
- fix `findPythonDeps.py` script when called with an (absolute or relative) path to an easyconfig instead of a filename (#4365)
- fix broken test for `reasons_for_closing`, which fails because commit status of easyconfigs PR is no longer available (#4366)
- other changes:
- reduce number of CI jobs by testing for Lua and Tcl module syntax in a single CI job (#4192)
v4.8.1 (11 September 2023)
--------------------------
update/bugfix release
- various enhancements, including:
- add end-to-end test for running EasyBuild in different Linux distros using containers (#3968)
- suggest default title in `--review-pr` (#4287)
- add `build_and_install_loop` hooks to run before and after the install loop for individual easyconfigs (#4304)
- implement support for `cancel_hook` and `fail_hook` (#4315, #4325)
- add postiter hook to the list of steps so the corresponding hook can be used (#4316)
- add `run_shell_cmd` hook (#4323)
- add `build_info_msg` easyconfig parameter to print message during installation of an easyconfig (#4324)
- add `--silence-hook-trigger` configuration option to supress printing of debug message every time a hook is triggered (#4329)
- add support for using fine grained Github tokens (#4332)
- add definitions for ifbf and iofbf toolchain (#4337)
- add support for submodule filtering and specifying extra Git configuration in `git_config` (#4338, #4339)
- various bug fixes, including:
- improve error when checksum dict has no entry for a file (#4150)
- avoid error being logged when `checksums.json` is not found (#4261)
- don't fail in `mkdir` if path gets created while processing it (#4300, #4328)
- ignore request for external module (meta)data when no modules tool is active (#4308)
- use sys.executable to obtain path to `python` command in tests, rather than assuming that `python` command is available in `$PATH` (#4309)
- fix `test_add_and_remove_module_path` by replacing string comparison of paths by checking whether they point to the same path (since symlinks may cause trouble) (#4312)
- enhance `Toolchain.get_flag` to handle lists (#4319)
- only add extensions in module file if there are extensions (#4331)
v4.8.0 (7 July 2023)
--------------------
feature release
- various enhancements, including:
- use version suffixes in PR title instead of Python versions when using --new-pr (#4253 + #4286)
- add script to find updated ECs for installed module (#4271)
- various bug fixes, including:
- fix regex for extracting glibc version from output of '`ldd --version`' in Gentoo Linux (#4290)
- allow versionsuffix to be set to None in `det_full_ec_version` (#4292)
- other changes:
- stop running tests with Python 2.7 since it is no longer supported in GitHub Action (#4267)
- replace imp.load_source by leveraging importlib.util.module_from_spec when using Python 3 (#4280)
v4.7.2 (27 May 2023)
--------------------
update/bugfix release
- various enhancements, including:
- extend (experimental) support for generating container images with Apptainer (#3975)
- add default download timeout (#4226)
- make the RPATH section check with `readelf -d` in sanity check optional (#4249)
- various bug fixes, including:
- add `make_extension_string` and `_make_extension_list` methods to `EasyBlock` class, so easyblocks can customize them (#3697)
- fix typo in condition guarding installation of GitHub token in CI workflow to run unit tests + fix broken tests related to GitHub integration features (#4216)
- fix --check-github as gist url now includes username (#4231)
- switch to Ubuntu 22.04 in GitHub Actions workflow to run container tests (#4238)
- use "/usr/bin/env python" in test_run_cmd_script (#4241)
- other changes:
- print potential errors after running a command only once (#4065)
- change github_* template descriptions to be somewhat clearer (#4251)
- remove workflow to test bootstrap script (#4255)
v4.7.1 (March 20th 2023)
------------------------
update/bugfix release
- various enhancements, including:
- add option to make sanity_check_paths arch dependent (#3845)
- add support for %(start_dir)s easyconfig template (#4073)
- add Ubuntu friendly package naming scheme EasyBuildDebFriendlyPNS (#4115)
- allow to directly import constants from easybuild.framework.easyconfig.constants (#4144)
- accept single source as dict and single checksum in check_checksums_for (#4180)
- add pre/post extension hook that is triggered before/after individual extension installations (#4193)
- enforce absolute paths as start dir of extensions (#4196)
- print warning when non-existing `start_dir` is specified for extension (#4201)
- add support to silence deprecation warning for easyconfigs and toolchains (#4207)
- skip directories when fixing shebangs (#4209)
- add support for 'modunloadmsg' easyconfig parameter (#4223)
- various bug fixes, including:
- fix use of `locate` in `check_os_dependency` (#4166)
- add VERSION to easybuild.easyblocks namespace in test sandbox (#4171)
- also apply filter to asyncprocess test subsuite (#4172)
- don't use deprecated SafeConfigParser when running with Python 3.x (#4173)
- update CI workflows to run 'apt-get update' if installation of packages via 'apt-get install' failed, likely due to stale apt cache (#4176)
- fixes for findPythonDeps script (#4179)
- check type of versionsuffix value in det_full_ec_version, and raise useful error message if it's not a string (#4184)
- avoid GC3Pie deprecation warning (#4185)
- correctly restore sys.path in tests (#4186)
- suppress output of --skip-test-step test (#4187)
- fix flaky test_search_file due to accidental matches for `.*C++` search pattern (#4190)
- silence distutils deprecation warnings (#4204)
- improve handling of start_dir and add tests for cases where either ext_dir or initial start_dir or both are unset or None (#4206)
- restore initial environment before processing each easystack item (#4213)
- make test_help_long work on small terminals (#4218)
- other changes:
- use better test assertions by replacing use of assertFalse/assertTrue (#4170, #4205)
- remove Travis CI configuration file (#4174)
- filter out deprecation warnings for platform.dist in get_os_version and platform.linux_distribution in get_os_name (#4175)
- only give read permissions in GitHub Actions workflows (#4182)
- fix website/docs links in README (#4199)
- only print "default: " for configuration option of strlist type if default is not empty (#4220)
v4.7.0 (January 9th 2023)
-------------------------
feature release
- print deprecation warning with running EasyBuild with Python 2 (#4136)
- various enhancements, including:
- add support for checksums specified in external `checksums.json` file (#3749)
- vendor distutils.version.LooseVersion as easybuild.tools.LooseVersion (since distutils is deprecated in Python 3.10) (#3794, #4156)
- take into account custom configuration options specified in easystack file (#4057)
- add support for using --output-format=md (MarkDown) (#4117, #4155)
- add support for --filter-rpath-sanity-libs to skip RPATH sanity check for designated libraries (#4119)
- add sanity_check_load_module method to EasyBlock to provide control over when module is loaded during sanity check step (#4125)
- add eb_bash_completion_local.bash script to setup.py (#4127)
- check whether nvidia-smi/rocm-smi command is available before trying to run it in `get_gpu_info` (#4131)
- add gfbf as subtoolchain of foss (#4143)
- add support for postinstallmsgs (#4145)
- make iimkl toolchain aware of intel-compilers (#4146)
- add definition for nvofbf toolchain (#4157, #4163)
- various bug fixes, including:
- also use EasyConfig instances cache in process_easyconfig when build_specs is empty dict (#4107)
- fix build options that should have empty list as default value (#4108)
- catch easyconfig parsing failure so we can generate and post a test report (#4109)
- add explicit return when no checksums match those specified in a tuple (#4112)
- tweak findPythonDeps.py script to use canonical package names (#4118)
- add `-fdefault-double-8` to `r8` toolchain compiler option for GCC (to be consistent with Intel) (#4121)
- always start with empty list for banned/requires libraries to avoid that corresponding build option is updated in-place (#4137)
- fix container tests by using EPEL archive URL for downloading Singularity RPM (#4138)
- use -march=x86-64 -mtune=generic instead of -xSSE2 when using Intel oneAPI compilers (#4147)
- partially skip test_det_copy_ec_specs if no GitHub token is available (#4149)
- other changes:
- drop support for easystack files using 'software' top-level key (#4057)
- also run unit tests with Python 3.11 + add Python 3.11 to classifiers in setup.py (#4092, #4141)
- use new EasyBuild logo in README (#4123)
- automatically cancel Github Action workflow runs for outdated commits (#4132)
- auto-enable use of oneAPI C/C++ compilers for intel-compilers >= 2022.2.0 (#4135)
- trim CI test configurations to avoid hitting GitHub rate limits (#4148)
- various tweaks to docstrings and help messages to fix problems with auto-generated documentation in MarkDown format (#4129, #4130, #4154, #4160, #4162, #4168)
- update copyright lines for 2023 (#4161)
v4.6.2 (October 21st 2022)
--------------------------
update/bugfix release
- various enhancements, including:
- add support for easystack file that contains easyconfig filenames + implement parsing of configuration options (#4021)
- skip over unset $EB_PYTHON/$EB_INSTALLPYTHON in eb wrappeer script (#4080)
- add GITHUB_RELEASE and GITHUB_LOWER_RELEASE templates (#4084)
- add `%(cuda_cc_cmake)s` template (#4087)
- various bug fixes, including:
- make check_sha256_checksums verify all checksums if they're specified as a dict value (#4076)
- replace use of symlink with copied files in `alt_location` tests to fix failing EasyBuild installation on BeeGFS (#4083)
- fix trying to generate RPATH wrappers for Clang (#4088)
- make sure that GitPython version is a proper version before checking minimal required version (#4090, #4091)
- first look for patch in alt_location when it is specified (#4093)
- other changes:
- make scripts executable (#4081)
- make --inject-checksums inject dictionary value for checksums which maps filename to SHA256 checksum (#4085)
- update to v3 of actions/checkout and actions/setup-python in CI workflows (#4089)
- use SYSTEM template constant in dependencies instead of True in framework tests (#4094)
v4.6.1 (September 12th 2022)
----------------------------
update/bugfix release
- various enhancements, including:
- add script to find dependencies of Python packages (#3839)
- add 'ai' default module class (#4053)
- various bug fixes, including:
- fix code style issues reported by recent flake8 linter (#4049)
- stick to autopep8 < 1.7.0 with Python 2.7 (#4055)
- ensure we call EasyBlock.patch_step for postinstallpatches (#4063)
- fix leaked handles in set_columns, complete_cmd, run_cmd_qa, det_terminal_size functions + tests (#4066)
- fix `quote_str` when string with both ' and " ends with a double quote (#4068)
- fix type-checking of patches to allow dict values + correctly handle patches specified as dict values in --new-pr (#4070)
- relax toolchain test by accepting both -march=native (x86_64) and -mcpu=native (aarch64) (#4071)
- other changes:
- run python in the same process as `eb` wrapper script by using `exec` (#4048)
- add get_linked_libs_raw function, and use it from both check_linked_shared_libs and sanity_check_rpath (#4051)
- update CI workflows (except container tests) to use Ubuntu 20.04, since Ubuntu 18.04 is deprecated (#4064)
- use SYSTEM constant for dependency that uses system toolchain in dumped easyconfig (#4069)
v4.6.0 (July 8th 2022)
----------------------
feature release
- various enhancements, including:
- allow searching for sources/patches in alternative location by specifying 'alt_location' in source/patch spec (#3994)
- show URLs used for download attempts in trace output (#4026)
- add support for setting environment variables via 'pushenv' with modextravars (#4030)
- add support for OneAPI compilers using toolchain option 'oneapi' (#4031, #4032, #4039)
- make check_linked_shared_libs more robust by taking into account that 'ldd' may fail (#4033)
- fall back to sequential extension install if parallel install is not implemented (#4034)
- add support for using template values in name/version of extensions (#4036)
- various bug fixes, including:
- make sure that ARCH constant has 'aarch64' (rather than 'arm64') as value on macOS ARM (#4018)
- tweak 'eb' wrapper script to correctly handle errors when python command being considered fails to run (#4019)
- tweak is_patch_for function to make it more robust (#4028)
- other changes:
- update Lmod used to run tests to version 8.7.6 (#4027, #4030)
- tweak apply_patch to not create .orig files (by default) when applying patch files (#4038)
v4.5.5 (June 8th 2022)
----------------------
update/bugfix release
- various enhancements, including:
- add toolchain definitions for nvompi (NVHPC + OpenMPI) (#3969), nvpsmpi (NVHPC + ParaStationMPI) (#3970), gfbf (GCC, FlexiBLAS, FFTW) (#4006)
- add support for FFTW.MPI toolchain component ($FFT*DIR variables) (#4012)
- add support for customizing EasyBuild command used in jobs via --job-eb-cmd (#4016)
- various bug fixes, including:
- fix copying of easyconfig file with --copy-ec without --rebuild if module is already installed (#3993)
- ignore deprecation warnings regarding cryptography in Python 3.6 + Blowfish with Python 3.10 in test suite output (#3999)
- fix typo in debug log message in easyblock.py (#4000)
- fix printing of list of attempted download URLs when url-encoded characters are used in URL (#4005)
- set $FFT(W)_LIB_DIR to imkl-FFTW's lib path in build environment if usempi toolchain option is enabled (#4011)
- correctly identify Apple Silicon M1 as Arm 64-bit by also considering arm64 next to aarch64 (#4014)
- fix 'eb --show-system-info' on Apple M1 system (#4015)
- other changes:
- change 'eb' command to import easybuild.framework to check if EasyBuild framework is available (#3995, #3998)
v4.5.4 (March 31st 2022)
------------------------
update/bugfix release
- various enhancements, including:
- warn about potentially missing patches in --new-pr (#3759, #3966)
- add support for 'clone_into' field in git_config source spec to specify different name for top-level directory (#3949)
- add bash completion for easyconfigs from local dir but not robot search path (#3953)
- add a 'sync pr' message when the PR has a mergeable state but is showing a failed status for the test suite on the last commit (#3967)
- add gmpit toolchain definition (GCC + MPItrampoline) (#3971)
- use 'zypper search -i' to check whether specified OS dependency is installed on openSUSE + make sure that rpm is considered for checking OS dependencies on RHEL8 (#3973)
- add support for post-install patches (#3974)
- add support for 'download_instructions' easyconfig parameter key to specify some download or installation steps for user in case of complicated way of obtaining needed files (#3976, #3981)
- also try collecting AMD GPU info (via rocm-smi) for --show-system-info (#3978, #3982)
- various bug fixes, including:
- ensure --review-pr can find dependencies included in PR (#3979)
- run 'apt-get update' in GitHub Actions workflow for container tests to update container package list before installing anything (#3985)
- other changes:
- enable code linting check for all supported Python versions (#3725)
- update copyright lines for 2022 (#3986)
v4.5.3 (February 11th 2022)
---------------------------
update/bugfix release
- various enhancements, including:
- also check for git in --check-eb-deps (#3954)
- add end2end test for 'eb --containerize' (#3958)
- various bug fixes, including:
- take into account that patch files can also be zipped when checking filename extension for patches (#3936)
- initialize BACKUP_PKG_URL with empty string in install_eb_dep.sh script (#3939)
- fix get_os_name and get_os_version to avoid reporting UNKNOWN in output of eb --show-system-info (#3942)
- consistently use actions/setup-python@v2 in CI workflows (#3944)
- switch to using pip3 for installing EasyBuild in container recipes generated by EasyBuild (#3945)
- specify easybuild.io as EasyBuild homepage in setup.py (#3947)
- avoid crash in get_os_version on modern SLES-based OSs (#3955)
- other changes:
- indicate compatibility with Python 3.10 in setup.py (#3940)
v4.5.2 (January 24th 2022)
--------------------------
update/bugfix release
- various enhancements, including:
- automatically prepend value for env-for-shebang configuration setting with sysroot (#3919)
- enhance update_build_option to return original value + add support for update_build_options (#3923)
- fix compatibility with Python 3.10 (#3926, #3933)
- also take /etc/os-release into account in get_os_name and get_os_version (#3930)
- enhance get_cpu_architecture and get_cpu_family to be aware of RISC-V (#3931)
- various bug fixes, including:
- relax pattern checks in test_toy_exts_parallel (#3921)
- use sources.easybuild.io as backup URL for downloading Environment Modules sources to run test suite (#3928)
- make intelfftw toolchain component aware of GCCcore (#3929)
- other changes:
- deprecate use of (actual) patch files that don't have a filename ending with .patch (#3920)
- remove version restriction for keyring in requirements.txt to test with latest version (#3925)
- simplify requirements.txt by removing Python 2.6 support (#3927)
- only run GitHub tests when testing with Lua module syntax, to avoid hitting GitHub rate limit when running tests (#3938)
v4.5.1 (December 13th 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- also dump environment to reprod directory (#3374)
- determine which extensions can be skipped in parallel (if --parallel-extensions-install is enabled) (#3890)
- fall back to sequential installation for extensions with unknown dependencies when using --parallel-extensions-install (#3906)
- allow oversubscription in sanity check for OpenMPI-based toolchains (#3909)
- various bug fixes, including:
- don't try to ensure absolute path for path part of repositorypath (#3893, #3899)
- fix typo in EULA agreement error message (#3894)
- only remove lock if it was created in the same EasyBuild session (not if it existed already) (#3889)
- introduce EasyBlock.post_init method to correctly define builddir variable when build-in-installdir mode is enabled in easyconfig or easyblock (#3900)
- also show download progress bar when using --inject-checksums (#3905)
- pick up custom extract_cmd specified for extension (#3907)
- make test_run_cmd_async more robust against fluke failures (#3908)
v4.5.0 (October 29th 2021)
--------------------------
feature release
- various enhancements, including:
- add --review-pr-max and --review-pr-filter options to limit easyconfigs used by --review-pr + retain order of easyconfigs being compared with (#3754)
- use Rich (if available) to show progress bars when installing easyconfigs (#3823, #3826, #3833, #3835, #3844, #3864, #3867, #3882)
- see also https://docs.easybuild.io/en/latest/Progress_bars.html
- add support for checking required/optional EasyBuild dependencies via 'eb --check-eb-deps' (#3829)
- add support for collecting GPU info (via nvidia-smi), and include it in --show-system-info and test report (#3851)
- added support for --insecure-download configuration option (#3859)
- make intelfftw toolchain component aware of imkl-FFTW module (#3861)
- make sure the contrib/hooks tree is included in the source tarball for easybuild-framework (#3862)
- add check_async_cmd function to facilitate checking on asynchronously running commands (#3865, #3881)
- add initial/experimental support for installing extensions in parallel (#3667, #3878)
- see also https://docs.easybuild.io/en/latest/Installing_extensions_in_parallel.html
- filter out duplicate paths added to module files, and print warning when they occur (#3770, #3874)
- various bug fixes, including:
- ensure that path configuration options have absolute path values (#3832)
- fix broken test by taking into account changed error raised by Python 3.9.7 when copying directory via shutil.copyfile (#3840)
- ensure newer location of CUDA stubs is taken into account by RPATH filter (#3850)
- replace 'which' by 'command -v' in 'eb' wrapper script to avoid dependency on 'which' command (#3852)
- refactor EasyBlock to decouple collecting of information on extension source/patch files from downloading them (#3860)
- in this context, the EasyBlock.fetch_extension_sources method was deprecated, and replaced by EasyBlock.collect_exts_file_info
- fix library paths to add to $LDFLAGS for intel-compilers toolchain component (#3866)
- remove '--depth 1' from git clone when 'commit' specified (#3871, #3872)
- make sure correct include directory is used for FlexiBLAS toolchain component (#3875)
- explictly disable rebase when pulling develop branch to create a merge commit, since not specifying how to reconcile divergent branches is an error with Git 2.33.1 and newer (#3879)
- tweak test_http_header_fields_urlpat to download from sources.easybuild.io rather than https://ftp.gnu.org (#3883)
- other changes:
- change copy_file function to raise an error when trying to copy non-existing file (#3836, #3855, #3877)
- only print the hook messages if EasyBuild is running in debug mode (#3843)
- deprecate old toolchain versions (pre-2019a common toolchains) (#3876, #3884)
- see also https://docs.easybuild.io/en/latest/Deprecated-easyconfigs.html#deprecated-toolchains
v4.4.2 (September 7th 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- add per-extension timing in output produced by eb command (#3734)
- add definition for new toolchain nvpsmpic (NVHPC + ParaStationMPI + CUDA) (#3736)
- include list of missing libraries in warning about missing FFTW libraries in imkl toolchain component (#3776)
- check for recursive symlinks by default before copying a folder (#3784)
- add --filter-ecs configuration option to filter out easyconfigs from set of easyconfigs to install (#3796)
- check type of source_tmpl value for extensions, ensure it's a string value (not a list) (#3799)
- also define $BLAS_SHARED_LIBS & co in build environment (analogous to $BLAS_STATIC_LIBS) (#3800)
- report use of --ignore-test-failure in success message in output (#3806)
- add get_cuda_cc_template_value method to EasyConfig class (#3807)
- add support for fix_bash_shebang_for (#3808)
- pick up $MODULES_CMD to facilitate using Environment Modules 4.x as modules tool (#3816)
- use more sensible branch name for creating easyblocks PR with --new-pr (#3817)
- various bug fixes, including:
- remove Python 2.6 from list of supported Python versions in setup.py (#3767)
- don't add directory that doesn't include any files to $PATH or $LD_LIBRARY_PATH (#3769)
- make logdir writable also when --stop/--fetch is used and --read-only-installdir is enabled (#3771)
- fix forgotten renaming of 'l' to 'char' __init__.py that is created for included Python modules (#3773)
- fix verify_imports by deleting all imported modules before re-importing them one by one (#3780)
- fix ignore_test_failure not set for Extension instances (#3782)
- update iompi toolchain to intel-compiler subtoolchain for oneAPI versions (>= iompi 2020.12) (#3785)
- don't parse patch files as easyconfigs when searching for where patch file is used (#3786)
- make sure git clone with a tag argument actually downloads a tag (#3795)
- fix CI by excluding GC3Pie 2.6.7 (which is broken with Python 2) and improve error reporting for option parsing (#3798)
- correctly resolve templates for patches in extensions when uploading to GitHub (#3805)
- add --easystack to ignored options when submitting job (#3813)
- other changes:
- speed up tests by caching checked paths in set_tmpdir + less test cases for test_compiler_dependent_optarch (#3802)
- speed up set_parallel method in EasyBlock class (#3812)
v4.4.1 (July 6th 2021)
----------------------
update/bugfix release
- various enhancements, including:
- enhance detection of patch files supplied to 'eb' command with better error messages (#3709)
- add per-step timing information (#3716)
- add module-write hook (#3728)
- add ignore-test-failure configuration option to ignore failing test step (#3732)
- add toolchain definition for nvompic (NVHPC + OpenMPI) (#3735)
- warn about generic milestone in --review-pr and --merge-pr (#3751)
- various bug fixes, including:
- don't override COMPILER_MODULE_NAME, inherited from Ffmpi, in Fujitsu toolchain definition (#3721)
- avoid overwritting pr_nr in post_pr_test_report for reports with --include-easyblocks-from-pr (#3724, #3726)
- fix crash in get_config_dict when copying modules that were imported in easyconfig file (like 'import os') (#3729)
- parse C standard flags in CFLAGS for Fujitsu compiler (#3731)
- fix error message for --use-ccache (#3733)
- error out when passing a list to run_cmd* to avoid running wrong command unintended, unless shell=True is used (#3737)
- minor fixes to output of test reports when using multiple PRs (#3741)
- fix location for modules installed with intel-compilers toolchain in HierarchicalMNS by always checking toolchain compiler name against template map (#3745)
- avoid checking msg attribute of GitCommandError (#3756)
- consider sources provided as dict in EasyBlock.check_checksums_for (#3758)
- don't make changes to software installation directory when using --sanity-check-only (#3761)
- honor specified easyblock for extensions (#3762)
- other changes:
- make sure that tests requiring a github token have 'github' in the test name so that they can be easily filtered (#3730)
- deprecate EasyBuild bootstrap script (#3742)
- use temporary download folder in test_http_header_fields_urlpat (#3755)
v4.4.0 (June 2nd 2021)
----------------------
feature release
- various enhancements, including:
- enhance apply_regex_substitutions to allow specifying action to take in case there are no matches (#3440)
- performance improvements for easyconfig parsing and eb startup (#3555)
- add support for downloading easyconfigs from multiple PRs with --from-pr (#3605, #3707, #3708)
- add support for prepending custom library paths in RPATH section via --rpath-override-dirs (#3650)
- allow amending easyconfig parameters which are not the default (#3651)
- update HierarchicalMNS for Intel OneAPI compilers (#3653)
- add support for --sanity-check-only (#3655)
- add support for running commands asynchronously via run_cmd + complete_cmd functions (#3656)
- add support for using oneAPI versions of 'intel' toolchain components (#3665)
- add toolchain definition for gofbf (foss with FlexiBLAS rather than OpenBLAS) (#3666)
- add support for using ARCH constant in easyconfig files (#3670)
- honor keyboard interrupt in 'eb' command (#3674)
- add toolchain definition for Fujitsu toolchain (#3677, #3704, #3712, #3713, #3714, #3717)
- extend sanity check step to check whether specific libraries are not linked into installed binaries/libraries (#3678)
- via banned-linked-shared-libs and required-linked-shared-libs EasyBuild configuration options
- via banned_linked_shared_libs and required_linked_shared_libs methods in toolchain support
- via banned_linked_shared_libs and required_linked_shared_libs methods in easyblock
- via banned_linked_shared_libs and required_linked_shared_libs easyconfig parameters
- add locate_solib function to locate Linux shared libraries (#3682)
- add system agnostic function to locate shared libraries (#3683)
- add update_build_option function to update specific build options after initializing the EasyBuild configuration (#3684)
- allow opting out of recursively unloaded of modules via recursive_module_unload easyconfig parameter (#3689)
- check for correct version values when parsing easystack file (#3693)
- run post-install commands specified for a specific extension (#3696)
- add support for --skip-extensions (#3702)
- include PR title in output produced by --merge-pr (#3706)
- various bug fixes, including:
- avoid metadata greedy behaviour when probing for external module metadata (mostly relevant for integration with Cray Programming Environment) (#3559)
- catch problems early on if --github-user is not specified for --new-pr & co (#3644)
- re-enable write permissions when installing with read-only-installdir (#3649)
- also run sanity check for extensions when using --module-only (#3655)
- improve logging when failing to load class from exts_classmap (#3657)
- fix use of --module-only on existing installations without write permissions (#3659)
- correct help text for subdir-user-modules (#3660)
- avoid picking up easyblocks outside of sandbox in framework tests (#3680)
- use unload/load in ModuleGeneratorLua.swap_module, since 'swap' is not supported by Lmod (#3685)
- update HierarchicalMNS to also return 'Toolchain/<name>/<version>' as $MODULEPATH extension for cpe* Cray toolchains (#3686)
- make EasyConfigParser.get_config_dict return a copy rather than a reference (#3692)
- make sure that $TAPE is unset when using piped tar (#3698)
- fix extending message for changed files in new_pr_from_branch (#3699)
- enhance sched_getaffinity function to avoid early crash when counting available cores on systems with more than 1024 cores (#3701)
- correctly strip extension from filename in extract_cmd and back_up_file functions (#3705)
- other changes:
- deprecate adding a non-existing path to $MODULEPATH (#3637)
- bump cryptography requirement from 3.2.1 to 3.3.2 (#3643, #3648)
- test bootstrap script in separate workflow, and limit test configurations a bit (#3646)
- update setup.py to indicate compatibility with Python 3.8 and 3.9 (#3647)
- replace log_error parameter of which() by on_error (#3661, #3664)
- don't skip sanity check for --module-only --rebuild (#3645)
- move disable_templating function into the EasyConfig class (#3668)
- pin GitPython version for Python<3.6, don't test bootstrap script against Python 3.8/3.9 (#3675)
- tweak foss toolchain definition to switch from OpenBLAS to FlexiBLAS in foss/2021a (#3679)
- suggest missing SSH key when not able to read from remote repository in --check-github (#3681)
- drop support for Python 2.6 (#3715)
v4.3.4 (April 9th 2021)
-----------------------
update/bugfix release
- various enhancements, including:
- add support for filtering dependencies by using False as version (#3506)
- add create_unused_dir function to create a directory which does not yet exist (#3551)
- avoid running expensive 'module use' and 'module unuse' commands when using Lmod as modules tool, update $MODULEPATH directly instead (#3557, #3633)
- create CUDA cache (for JIT compiled PTX code) in build dir instead of $HOME (#3569)
- add "Citing" section to module files (#3596)
- add support for using fallback 'arch=*' key in dependency version specified as arch->version mapping (#3600)
- also check for pending change requests and mergeable_state in check_pr_eligible_to_merge (#3604)
- ignore undismissed 'changes requested' review if there is an 'approved' review by the same user (#3607, #3608)
- sort output of 'eb --search' in natural order (respecting numbers) (#3609)
- enhance 'eb' command to ensure that easybuild.main can be imported before settling on python* command to use (#3610)
- add --env-for-shebang configuration option to define the env command to use for shebangs (#3613)
- add templates for architecture independent Python wheels (#3618)
- mention easyblocks PR in gist when uploading test report for it + fix clean_gists.py script (#3622)
- also accept regular expression value for --accept-eula-for (#3630)
- update validate_github_token function to accept GitHub token in new format (#3632)
- various bug fixes, including:
- fix $BLAS_LIB_MT for OpenBLAS, ensure -lpthread is included (#3584)
- use '--opt=val' for passing settings from config file to option parser to avoid error for values starting with '-' or '--' (#3594)
- avoid raised exception when getting output from interactive command in run_cmd_qa (#3599)
- add option to write file from file-like object and use in download_file (#3614)
- make sure that path to eb is always found by tests (#3617)
- other changes:
- add pick_default_branch function to clean up duplicate code in tools/github.py (#3592)
- refactor the CI configuration to use inclusion instead of exclusion (#3616)
- use develop branch when testing push access in --check-github (#3629)
- deprecate --accept-eula, rename to --accept-eula-for (#3630)
v4.3.3 (February 23rd 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- advise PR labels in --review-pr and add support for --add-pr-labels (#3177)
- add support for using customized HTTP headers in download_file (#3472, #3583)
- also take toolchain dependencies into account when defining template values (#3541, #3560)
- add support for --accept-eula configuration option + 'accept_eula' easyconfig parameter (#3535, #3536, #3546)
- detect 'SYSTEM' toolchain as special case in easystack files (#3543)
- enhance extract_cmd function to use 'cp -a' for shell scripts (.sh) (#3545)
- allow use of alternate envvar(s) to $HOME for user modules (#3558)
- use https://sources.easybuild.io as fallback source URL (#3572, #3576)
- add toolchain definition for iibff toolchain (#3574)
- add %(cuda_cc_space_sep)s and %(cuda_cc_semicolon_sep)s templates (#3578)
- add support for intel-compiler toolchain (>= 2021.x versions, oneAPI) (#3581, #3582)
- various bug fixes, including:
- add --init and --recursive options to 'git submodule update' command that is used when creating source tarball for specific commit (#3537)
- filter out duplicate paths in RPATH wrapper script (#3538)
- don't clean up imported modules after verifying imports of included Python modules (#3544)
- avoid no-op changes to $LD_* environment variables in ModulesTool (#3553)
- fix UTF-8 encoding errors when running EasyBuild with Python 3.0.x-3.6.x (#3565)
- create lib64 symlink as a relative symlink (#3566)
- don't reuse variable name in the loop to fix adding extra compiler flags via toolchainopts (#3571)
- symlink 'lib' to 'lib64' if it doesn't exist (#3580)
- include %(mpi_cmd_prefix)s and %(cuda_*)s templates in output of --avail-easyconfig-templates (#3586)
- other changes:
- rename EasyBlock._skip_step to EasyBlock.skip_step, to make it part of the public API (#3561)
- make symlinking of posix_c.so to posix.so in test suite configuration conditional (#3570)
- use 'main' rather than 'master' branch in GitHub integration functionality (#3589)
v4.3.2 (December 10th 2020)
---------------------------
update/bugfix release
- add (experimental) support for specifying easyconfig files via an "easystack" file (#3479, #3511, #3515, #3517, #3520, #3521)
- see also https://easybuild.readthedocs.io/en/latest/Easystack-files.html
- add definition for new 'gobff' toolchain using BLIS and LibFLAME (#3505)
- various enhancements, including:
- add support for toolchain options like 'extra_cxxflags' to specify extra compiler options (#2193)
- fix combination of --copy-ec and --from-pr (#3482)
- enhance copy_files function: support single file target, error on empty input list, support verbose mode (#3483)
- cache result of fetch_files_from_pr function (mainly to speed up framework test suite) (#3484)
- add locate_files function to filetools module (#3485)
- add support for %(module_name)s template value (#3497)
- clarify input format for --cuda-compute-capabilities in 'eb --help' output (#3509)
- add support for skiping unit tests (test step) via --skip-test-step (#3524)
- various bug fixes, including:
- also ignore vsc.* imports coming from from pkg_resources/__init__.py (setuptools) in fake vsc namespace (#3491)
- don't pass username in github_api_get_request when no GitHub token is available (#3494)
- also inject -rpath options for all entries in $LIBRARY_PATH in RPATH wrappers (#3495)
- avoid TypeError being raised by list_toolchains (#3499)
- check if PR is already merged in --merge-pr (#3502)
- graciously handle wrong PR id in fetch_pr_data (#3503)
- fix regression in apply_regex_substitutions: also accept list of paths to patch (#3507)
- update installation procedure for EasyBuild in generated Singularity container recipes (#3510)
- fix GitHub Actions workflow for test suite: run outside of repo checkout + also test bootstrap script with Python 3.9 (#3518)
- bump cryptography from 2.9.2 to 3.2 for Python 2 in requirements.txt (#3519)
- fix 'eb --help=rst' when running with Python 3 (#3525)
- other changes:
- exclude test configurations with Lmod 7 and Python 3, except for Python 3.6 (#3496)
- significantly speed up parsing of easyconfig files by only extracting comments from an easyconfig file when they're actually needed (#3498)
- don't include file/ldd/readelf commands run during RPATH sanity check in --trace output (#3508)
v4.3.1 (October 29th 2020)
--------------------------
update/bugfix release
- various enhancements, including:
- further GCC toolchain fixes for aarch64 (#3433)
- take into account --include-easyblocks-from-pr when uploading test reports (#3446)
- add path to pkg-config files in sysroot to $PKG_CONFIG_PATH when --sysroot is specified (#3451)
- add support for NVHPC compiler + toolchain (based on PGI) (#3454)
- check for _VERSION and _PREFIX Cray environment variables with both software and module name (#3460)
- allow including easyblocks from multiple PRs (#3480, #3481)
- various bug fixes, including:
- avoid UnicodeDecodeError in apply_regex_substitutions when patching files that include non-UTF-8 characters (#3450)
- avoid appending lib stubs pattern to RPATH filter over and over again (#3452)
- fix missing string template on error for incorrect extension 'sources' value (#3461)
- fix compatibility with Python 3.9 by renaming fancy root logger (#3465)
- also remove empty checksums list specified in easyconfig file when using --inject-checksums (#3466)
- avoid confusing error log message when determining impi version while trying to define value for %(mpi_cmd_prefix)s template (#3474)
- unset $LD_LIBRARY_PATH when checking for OS dependencies with 'rpm' & co (#3477)
- don't change directory in download_repo function in tools.github (#3486)
- take source_urls, checksums, patches into account when extension source is specified via 'sources' (#3487)
- other changes:
- consider $EB_INSTALLPYTHON in 'eb' command to specify 'python' command to use for running EasyBuild (#3428)
- use only the sub folder name for createSubmoduleDeps script (#3464)
v4.3.0 (September 13th 2020)
----------------------------
feature release
- various enhancements, including:
- add script to create 'sources' entries for git submodules (#3369, #3436)
- add templates for CUDA compute capabilities (#3382)
- '%(cuda_compute_capabilities)s', '%(cuda_sm_comma_sep)s', '%(cuda_sm_space_sep)s'
- add EasyBuild configuration option --generate-devel-module (#3388)
- to allow disabling generating of "devel" modules via --disable-generate-devel-module
- set up a minimal build environment when using system compiler (#3399)
- by default, $CC is set to 'gcc' and $CXX is set to 'g++' when using 'system' toolchain
- minimal build environment can be customized via --minimal-build-env configuration option
- add --sysroot configuration option to specify alternative location of system root (#3419)
- this is useful when installing software in a Gentoo Prefix environment (for example)
- various bug fixes, including:
- allow including easyblocks from multiple locations, by combining --include-easyblocks and --include-easyblocks-from-pr (#3311)
- also escape backslashes in quote_py_str() (#3386)
- use one argument 'module swap' statements in Tcl modulefiles (required by Modules 4.2.3+) (#3397)
- fix copying of (non-existing) file with apply_patch (#3400)
- create symlink from lib64 to lib subdir in installation directories to avoid that GCC prefers /lib64 system directories (#3401)
- fix default value for 'lib64_fallback_sanity_check' build option (#3402)
- correctly determine commit status in --merge-pr (#3406)
- stop installing ccache wrapper for Fortran compiler (#3409)
- fix issues with applying PR patch in --from-pr (#3414)
- make RPATH wrapper script more robust by using 'python -E -s -S' to run rpath_args.py (#3422)
- don't inject -Wl,-rpath options when '-x c++-header' compiler option is used (#3424)
- fix lack of -mno-recip on aarch64 by tweaking GCC options used when 'precise' toolchain option is enabled (#3425)
- make sure self.start_dir is set in ExtensionEasyBlock (#3426, #3435)
- exclude local variables from typo check in easyconfig files (#3427)
- other changes:
- stick to cryptography 2.9.2 when using Python 2.7 to avoid broken test suite in CI (#3392)
- automatically enable --ignore-locks with --fetch (#3404)
- switch to status badge based on tests run in GitHub Actions CI in README (#3415)
- make flake8 code style checks pass (#3416, #3417)
- limit Travis to only test with Python 2.6 + Lomod 7 (#3418)
- ignore deprecation warning raised for cryptography when using Python 3.5 produced by output of test suite (#3423)
v4.2.2 (July 8th 2020)
----------------------
update/bugfix release
- various enhancements, including:
- add support for using 'sources' and 'git_config' for extensions in 'exts_list' (#3294)
- add support for software minver template (like %(pyminver)s, etc.) (#3344, #3345)
- add support for updating dictionary or tuple easyconfig parameters with self.cfg.update (#3356)
- various bug fixes, including:
- fix crash in --avail-easyconfig-constants when using --output-format=rst + ensure sorted output (#3341)
- always take into account builddependencies when generating template values, also when we're not iterating over builddependencies (#3346)
- fix running command as 'easybuild' user in generated Singularity definition file (#3347)
- allow ignoring versionsuffix in --try-update-deps (#3350, #3353)
- retain order of paths when generating prepend_path statements for module file (don't sort them alphabetically) (#3367)
- also put easyblocks used by extensions in 'reprod' directory (#3375)
- also copy template values in EasyConfig.copy method to ensure they are defined when installing extensions (#3377)
- skip lines that start with 'module-version' when determining whether a module exists in ModulesTool.exist (#3379)
v4.2.1 (May 20th 2020)
----------------------
update/bugfix release
- various enhancements, including:
- also mention CPU architecture (x86_64, POWER) in comment for test reports (#3281)
- add support for enhancing existing sanity check in easyconfigs, as opposed to overwriting paths/commands (#3288)
- clean up locks when EasyBuild session is cancelled with a signal like SIGTERM (#3291, #3321)
- add 'find_glob_pattern' function to filetools module (#3297)
- add constants for common OS dependencies (OS_PKG_IBVERBS_DEV, OS_PKG_OPENSSL_DEV, ...) (#3309, #3334)
- flesh out get_mpi_cmd_template function from Mpi.mpi_cmd_for method (#3312)
- add variable 'moddependpaths' to specify extra $MODULEPATH entry to consider for loading dependency modules (#3324)
- allow copying of tweaked easyconfigs when using --try-* with --copy-ec (#3332)
- various bug fixes, including:
- make ModulesTool.exist more robust w.r.t. module wrappers, aliases, defaults, etc. (#3216, #3337)
- clean up rst output of --list-toolchains (#3246)
- cast CPU arch name provided by archspec to a regular string (#3286)
- get pr_title and pr_descr built_options in new_pr_from_branch instead of new_pr (and commit_msg in both) (#3298)
- make pypi_source_urls more robust by using HTMLParser rather than xml.etree.ElementTree (#3303, #3329)
- fix broken test for --include-easyblocks-from-pr (#3304)
- don't use distutils.dir_util in copy_dir (#3310)
- print trace message for sanity check command before running it (#3316)
- fix problems with processing of easyconfigs using a Cray* toolchain when there are no actual external modules (#3319)