Add support for quantification kinds to the DSL #150

natecook1000 · 2022-02-09T05:58:31Z

This adds support for specifying eager / reluctant / possessive quantification from within the DSL, like the literal quantification operators with an appended ?.

For example, with the default eager capturing, the following simple regex matches the entire string and capture the "2":

let regex = Regex {
    oneOrMore(.word, .reluctant)
    CharacterClass.digit.capture()
}
"aaa1bbb2".match(regex).match // ("aaa1", "1")

This adds support for specifying eager / reluctant / possessive quantification from within the DSL, like the literal quantification operators with an appended `?`. --- ```swift let regex = Regex { oneOrMore(.word, .reluctant) CharacterClass.digit.capture() } "aaa1bbb2".match(regex).match // ("aaa1", "1") ```

rxwei · 2022-02-09T06:08:49Z

Sources/VariadicsGenerator/VariadicsGenerator.swift

+        _ component: Component,
+        _ kind: QuantificationKind = .eager


Should kind come before component? This would make it consistent with the builder API. For the builder API we are kinda out of options since we'd have to support a trailing closure.

So it's

oneOrMore("x", .eager) oneOrMore(.eager) { "x" }

vs.

oneOrMore(.eager, "x") oneOrMore(.eager) { "x" }

I can't decide about this. On the one hand, it's nice to have them in a consistent order. On the other, I think it reads better to have the thing you're matching come first, and only shift it to the end when it's in the trailing closure position. IIUC, this is how SwiftUI handles it pretty much across the board — for example, when creating a Button with just a label, the string is the first parameter, otherwise, the builder closure comes at the end.

Let's punt on this for now to see how many different calls like this we have.

Good point. If there's a precedent in SwiftUI I agree we should follow that.

I wonder if the fact we keep using the term "kind" clues us in a bit. An eager oneOrMore is a different kind of operation than a reluctant or possessive one, and both spelling it as passing a ternary enum or as 3 separate overloads could make sense. Under that interpretation, I think the kind should come first.

We have a default (.eager for regex default semantics), so I would expect the Button(label) analogy for declaring the construction of a button with a label to be more analogous to oneOrMore(pattern) when you just want the default.

rxwei · 2022-02-09T06:13:50Z

Sources/_StringProcessing/RegexDSL/DSL.swift

@@ -62,6 +62,44 @@ extension CharacterClass: RegexProtocol {

 // Note: Quantifiers are currently gyb'd.

+/// Specifies how much to attempt to match when using a quantifier.
+public struct QuantificationKind {


"Kind" feels a bit too general. Kind could also refer to one-or-more/zero-or-more/zero-or-one. Is there a more specific word for this? Maybe QuantificationBehavior?

For the DSL, should quantifier kinds read like an adverb for fluency?
oneOrMore("x", .eagerly) oneOrMore("x", .reluctantly)

Agreed on both these counts — good call!

Would these instead be properties returning Self if we're going that route?

rxwei

Nice!

rxwei · 2022-02-09T20:26:08Z

Sources/VariadicsGenerator/VariadicsGenerator.swift

-        _ component: Component
+      public func \(kind.rawValue)<\(genericParams)>(
+        _ component: Component,
+        _ kind: QuantificationBehavior = .eagerly


Nit

Suggested change

_ kind: QuantificationBehavior = .eagerly

_ behavior: QuantificationBehavior = .eagerly

Ah, missed these!

rxwei · 2022-02-09T20:26:15Z

Sources/VariadicsGenerator/VariadicsGenerator.swift

      }

      \(arity == 0 ? "@_disfavoredOverload" : "")
-      public func \(kind.rawValue)<\(genericParameters(withConstraints: true))>(
+      public func \(kind.rawValue)<\(genericParams)>(
+        _ kind: QuantificationBehavior = .eagerly,


Suggested change

_ kind: QuantificationBehavior = .eagerly,

_ behavior: QuantificationBehavior = .eagerly,

natecook1000 · 2022-02-09T23:22:55Z

@swift-ci Please test

rxwei · 2022-02-10T06:35:51Z

Sorry for the merge conflicts!

milseman

LGTM

milseman · 2022-02-10T14:20:50Z

Sources/_StringProcessing/RegexDSL/DSL.swift

@@ -62,6 +62,44 @@ extension CharacterClass: RegexProtocol {

 // Note: Quantifiers are currently gyb'd.

+/// Specifies how much to attempt to match when using a quantifier.
+public struct QuantificationBehavior {


Thoughts on this being a "thing" in API vs a set of operations on quantifiable patterns? E.g. there could be a sub-protocol for quantifiables with a var eagerly: Self requirement.

(Not arguing one way or another)

I think it makes the most sense as a parameter to the quantification methods — making it an operation on a oneOrMore that uses a builder puts the adverb too far from what it describes. The .eagerly in this example doesn't read clearly as modifying oneOrMore to me:

let regex = Regex { oneOrMore { "a" optionally(.digit) .word }.eagerly }

milseman · 2022-02-10T14:22:05Z

Tests/RegexTests/RegexDSLTests.swift

+    {
+      oneOrMore {
+        oneOrMore(.word, .reluctantly)
+        CharacterClass.digit.capture()


Can we drop CharacterClass?

At some point I had to add a RegexWithComponent protocol and some overloads to make the static vars discoverable, but @rxwei was able to remove that and keep the lookup. I don't recall the issues there.

Is the compiler able to distinguish between .digit being a method of oneOrMore(...)'s return type and it being a static member call on its own? I haven't been able to get that kind of lookup to work for static protocol methods.

natecook1000 · 2022-02-10T16:12:06Z

@swift-ci Please test Linux platform

natecook1000 added 4 commits February 8, 2022 22:59

Consolidate RegexDSLTests testing logic

b1675a0

Remove unused parameter

5875fb3

Fix indentation in variadic generator

453505b

natecook1000 requested a review from rxwei February 9, 2022 05:58

rxwei reviewed Feb 9, 2022

View reviewed changes

Update naming (thanks to @rxwei)

51c4f1c

rxwei approved these changes Feb 9, 2022

View reviewed changes

rxwei reviewed Feb 9, 2022

View reviewed changes

Update parameter names

c1a6f0c

milseman approved these changes Feb 10, 2022

View reviewed changes

natecook1000 added 2 commits February 10, 2022 10:10

Merge branch 'main' into dsl_quanitification_kind

bfea965

Catch a few captures I missed

fd9438f

natecook1000 merged commit 12b4b4a into swiftlang:main Feb 10, 2022

natecook1000 deleted the dsl_quanitification_kind branch February 10, 2022 16:40

	_ kind: QuantificationBehavior = .eagerly
	_ behavior: QuantificationBehavior = .eagerly

Add support for quantification kinds to the DSL #150

Add support for quantification kinds to the DSL #150

Uh oh!

Conversation

natecook1000 commented Feb 9, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rxwei left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

natecook1000 commented Feb 9, 2022

Uh oh!

rxwei commented Feb 10, 2022

Uh oh!

milseman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

natecook1000 commented Feb 10, 2022

Uh oh!

Uh oh!