[stdlib] Collapse sequence and collection wrappers #20221

airspeedswift · 2018-11-01T19:00:47Z

Builds on top of #20175 and collapses wrappers like LazyMapSequence and LazyMapCollection into one now this is possible.

airspeedswift · 2018-11-09T00:03:27Z

@swift-ci please test

swift-ci · 2018-11-09T00:16:02Z

Build failed
Swift Test OS X Platform
Git Sha - 38a13c1ecc3670cab02e22c99a6b78f07462bbdb

swift-ci · 2018-11-09T02:10:41Z

Build failed
Swift Test Linux Platform
Git Sha - 4cf80a439192828db6666da66e6c6b58e4c5c889

airspeedswift · 2018-11-12T16:27:19Z

(test will fail but I want to track exactly where)

airspeedswift · 2018-11-12T16:27:54Z

@swift-ci please smoke test macOS platform

airspeedswift · 2018-11-13T02:32:17Z

@swift-ci please test

swift-ci · 2018-11-13T04:52:25Z

Build failed
Swift Test OS X Platform
Git Sha - e6a75987b2b9526880295002facc695c86880f82

airspeedswift · 2018-11-13T16:47:39Z

@swift-ci please test

swift-ci · 2018-11-13T17:46:38Z

Build failed
Swift Test OS X Platform
Git Sha - 712a6e748c701342d67e9e4055b2ea0a29d2c3ea

Remove split customization point Eliminate SubSequence from Sequence protocol Collapse LazyCollection Collapse LazyMapCollection Eliminate _SequenceWrapper Collapse LazyFilterCollection Collapse LazyDrop/PrefixWhileCollection Fix tests, ABI stability update Collapse FlattenSequence

airspeedswift · 2018-11-14T00:09:07Z

@swift-ci please benchmark

airspeedswift · 2018-11-14T00:09:18Z

@swift-ci please test

swift-ci · 2018-11-14T00:58:12Z

Build comment file:

Performance: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
DropLastAnySeqCntRangeLazy	9	21083	+234129.5%	0.00x
SuffixAnySeqCntRangeLazy	14	20952	+149546.5%	0.00x
DropLastAnySeqCntRange	9	516	+5632.7%	0.02x
SuffixAnySeqCntRange	14	760	+5328.2%	0.02x
DropLastAnySeqCRangeIterLazy	2277	21074	+825.5%	0.11x
SuffixAnySeqCRangeIterLazy	3027	21020	+594.4%	0.14x
PrefixAnySeqCRangeIterLazy	32	88	+175.0%	0.36x
PrefixAnySeqCRangeIter	32	88	+175.0%	0.36x
PrefixWhileAnySeqCntRange	141	363	+157.4%	0.39x
DropFirstAnySeqCRangeIter	58	93	+60.3%	0.62x
DropFirstAnySeqCRangeIterLazy	58	91	+56.9%	0.64x
PrefixAnySequence	1412	2213	+56.7%	0.64x
DropFirstAnySequenceLazy	1855	2729	+47.1%	0.68x
DropWhileAnySequence	1855	2694	+45.2%	0.69x
DropFirstAnySequence	1855	2651	+42.9%	0.70x
DropFirstAnySeqCntRange	71	92	+29.6%	0.77x
DropFirstAnySeqCntRangeLazy	71	91	+28.2%	0.78x
DropWhileAnySeqCRangeIter	75	95	+26.7%	0.79x
PrefixAnySeqCntRange	72	88	+22.2%	0.82x
PrefixAnySeqCntRangeLazy	73	88	+20.5%	0.83x
Improvement
DropWhileSequence	2204	29	-98.7%	76.00x
PrefixSequenceLazy	2267	52	-97.7%	43.60x
PrefixSequence	2258	52	-97.7%	43.42x
DropFirstSequenceLazy	2308	58	-97.5%	39.79x
DropFirstSequence	2240	58	-97.4%	38.62x
DropLastAnySequence	3932	546	-86.1%	7.20x
SuffixAnySequence	3903	568	-85.4%	6.87x
SuffixSequence	2593	521	-79.9%	4.98x
SuffixSequenceLazy	2590	521	-79.9%	4.97x
DropLastAnySeqCRangeIter	2272	518	-77.2%	4.39x
SuffixAnySeqCRangeIter	3033	825	-72.8%	3.68x
DropLastAnySequenceLazy	4030	3400	-15.6%	1.19x
PrefixWhileAnySeqCRangeIter	387	344	-11.1%	1.12x
SuffixAnySequenceLazy	3966	3533	-10.9%	1.12x
DictionaryOfAnyHashableStrings_lookup	4279	3826	-10.6%	1.12x
CharacterLiteralsLarge	108	97	-10.2%	1.11x
InsertCharacterEndIndex	163	148	-9.2%	1.10x
StringEqualPointerComparison	628	571	-9.1%	1.10x
Hanoi	3741	3460	-7.5%	1.08x
CharacterLiteralsSmall	348	325	-6.6%	1.07x

Code size: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
LazyFilter.o	9497	11193	+17.9%	0.85x
DropLast.o	24811	26467	+6.7%	0.94x
Suffix.o	25465	26361	+3.5%	0.97x
DropFirst.o	24356	25028	+2.8%	0.97x
RomanNumbers.o	5247	5375	+2.4%	0.98x
Prefix.o	23913	24297	+1.6%	0.98x
Improvement
NibbleSort.o	16456	12168	-26.1%	1.35x
LuhnAlgoLazy.o	11808	10986	-7.0%	1.07x
LuhnAlgoEager.o	11810	10988	-7.0%	1.07x
COWTree.o	13464	12642	-6.1%	1.07x
UTF8Decode.o	13098	12378	-5.5%	1.06x
SequenceAlgos.o	21535	20713	-3.8%	1.04x
Substring.o	18890	18295	-3.1%	1.03x
DictOfArraysToArrayOfDicts.o	30848	29998	-2.8%	1.03x
PrefixWhile.o	23438	22878	-2.4%	1.02x
StringMatch.o	4381	4318	-1.4%	1.01x
CSVParsing.o	32385	32057	-1.0%	1.01x

Performance: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
DropLastAnySeqCRangeIterLazy	2668	20895	+683.2%	0.13x
SuffixAnySeqCRangeIterLazy	2964	21022	+609.2%	0.14x
SuffixAnySeqCntRangeLazy	5335	20959	+292.9%	0.25x
DropLastAnySeqCntRangeLazy	5332	20900	+292.0%	0.26x
PrefixWhileAnySeqCntRange	234	347	+48.3%	0.67x
DropFirstAnySeqCntRange	15984	21264	+33.0%	0.75x
SuffixCountableRangeLazy	11	12	+9.1%	0.92x
Improvement
PrefixAnySeqCRangeIterLazy	17015	159	-99.1%	107.01x
DropFirstAnySeqCRangeIterLazy	21764	218	-99.0%	99.83x
DropWhileAnySeqCRangeIter	17877	180	-99.0%	99.32x
PrefixAnySeqCntRangeLazy	15971	176	-98.9%	90.74x
DropFirstAnySeqCntRangeLazy	15976	213	-98.7%	75.00x
DropWhileSequence	2582	35	-98.6%	73.77x
PrefixSequence	2703	70	-97.4%	38.61x
PrefixSequenceLazy	2670	70	-97.4%	38.14x
DropFirstSequenceLazy	2664	76	-97.1%	35.05x
DropFirstSequence	2648	76	-97.1%	34.84x
PrefixWhileAnySeqCRangeIter	6556	364	-94.4%	18.01x
DropLastAnySeqCntRange	5329	536	-89.9%	9.94x
SuffixAnySeqCntRange	5334	760	-85.8%	7.02x
DropLastAnySequence	4552	677	-85.1%	6.72x
DropLastAnySeqCRangeIter	2669	533	-80.0%	5.01x
SuffixAnySequence	4571	942	-79.4%	4.85x
PrefixWhileAnySequence	8048	1692	-79.0%	4.76x
SuffixAnySeqCRangeIter	2963	826	-72.1%	3.59x
SuffixSequence	2975	942	-68.3%	3.16x
SuffixSequenceLazy	2974	943	-68.3%	3.15x
PrefixAnySequenceLazy	4292	1581	-63.2%	2.71x
DropWhileAnySequence	5539	3194	-42.3%	1.73x
DropFirstAnySequenceLazy	4669	3194	-31.6%	1.46x
PrefixWhileSequence	486	345	-29.0%	1.41x
DropWhileAnySeqCntRange	205	163	-20.5%	1.26x
DropLastAnySequenceLazy	4626	3780	-18.3%	1.22x
SuffixAnySequenceLazy	4593	3833	-16.5%	1.20x
Dictionary4OfObjects	563	499	-11.4%	1.13x
Dictionary4	282	251	-11.0%	1.12x

Code size: -Osize

TEST	OLD	NEW	DELTA	RATIO
Improvement
NibbleSort.o	18752	14160	-24.5%	1.32x
StringMatch.o	4785	4281	-10.5%	1.12x
Substring.o	17250	15834	-8.2%	1.09x
PrefixWhile.o	22894	21246	-7.2%	1.08x
COWTree.o	13622	12800	-6.0%	1.06x
LuhnAlgoLazy.o	14744	13922	-5.6%	1.06x
LuhnAlgoEager.o	14746	13924	-5.6%	1.06x
UTF8Decode.o	12453	11873	-4.7%	1.05x
CSVParsing.o	32673	31513	-3.6%	1.04x
SequenceAlgos.o	23240	22418	-3.5%	1.04x
DictOfArraysToArrayOfDicts.o	31440	30590	-2.7%	1.03x
RomanNumbers.o	6094	5934	-2.6%	1.03x
StringEdits.o	12070	11950	-1.0%	1.01x

Performance: -Onone

TEST	OLD	NEW	DELTA	RATIO
Regression
DropLastAnySeqCntRangeLazy	5583	35140	+529.4%	0.16x
DropLastAnySeqCntRange	5580	34826	+524.1%	0.16x
SuffixAnySeqCntRangeLazy	5582	29478	+428.1%	0.19x
SuffixAnySeqCntRange	5581	29188	+423.0%	0.19x
StringBuilderWithLongSubstring	1980	3662	+84.9%	0.54x
StringMatch	31735	48611	+53.2%	0.65x
StringUTF16SubstringBuilder	9170	14011	+52.8%	0.65x
StringEdits	131815	178898	+35.7%	0.74x
DropFirstAnySeqCntRangeLazy	16691	22259	+33.4%	0.75x
DropFirstAnySeqCntRange	16722	22185	+32.7%	0.75x
DropWhileAnySeqCntRange	21695	25420	+17.2%	0.85x
Improvement
ArrayOfPOD	860	780	-9.3%	1.10x
DropFirstSequence	10225	9332	-8.7%	1.10x
DropFirstSequenceLazy	10357	9499	-8.3%	1.09x
PrefixSequenceLazy	8531	7865	-7.8%	1.08x
PrefixAnySeqCRangeIterLazy	18181	16799	-7.6%	1.08x
PrefixAnySequenceLazy	8792	8124	-7.6%	1.08x
PrefixSequence	8459	7836	-7.4%	1.08x

Code size: -swiftlibs

TEST	OLD	NEW	DELTA	RATIO
Improvement
libswiftsimd.dylib	266240	217088	-18.5%	1.23x
libswiftSwiftReflectionTest.dylib	49152	40960	-16.7%	1.20x
libswiftCoreAudio.dylib	28672	24576	-14.3%	1.17x
libswiftCloudKit.dylib	94208	81920	-13.0%	1.15x
libswiftDispatch.dylib	110592	98304	-11.1%	1.12x
libswiftCore.dylib	3227648	2998272	-7.1%	1.08x
libswiftFoundation.dylib	1605632	1523712	-5.1%	1.05x

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

--------------

palimondo · 2018-11-14T01:37:53Z

stdlib/public/core/Sequence.swift

+      rotated.reserveCapacity(ringBuffer.count)
+      rotated += ringBuffer[i..<ringBuffer.endIndex]
+      rotated += ringBuffer[0..<i]
+      return rotated


How does this copying approach compare to in-place rotation?

ringBuffer[0..<i].reverse() ringBuffer[i..<ringBuffer.endIndex].reverse() ringBuffer[0..<ringBuffer.endIndex].reverse()

Also, is there a potential perf. win in using ContiguousArrays here (and in dropLast) and then return Array(ringBuffer)? I've noticed that's what map and filter implementations use.

Either are worth exploring to see – they're not important for the ABI though so we can look at trying them out after this change lands.