Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement OrderedDictionary<K, V> type #3913

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

flobernd
Copy link
Member

@flobernd flobernd commented Mar 7, 2025

Adds a new builtin dictionary type OrderedDictionary<K, V> that can be used instead of Dictionary<K, V> when the order of items is important.

Motivation

Some dictionaries require a fixed order (e.g. aggregations), but currently we do not model this fact at all.

This is quite special since the JSON spec does not make any guarantees about the order of properties in an object, but still Elasticsearch relies on that.

Closes: #3908
Related to: #3909

@l-trotta
Copy link
Contributor

l-trotta commented Mar 7, 2025

as reported in elastic/elasticsearch-java#932

@flobernd flobernd force-pushed the ordered_dictionary branch from 829202e to 94a2249 Compare March 7, 2025 12:07
@@ -773,6 +773,13 @@ export default async function validateModel (apiModel: model.Model, restSpec: Ma
}

context.push(`Property '${prop.name}'`)

if (prop.type.kind === 'dictionary_of' && prop.type.ordered) {
if (prop.name !== 'aggregations') {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check only works for the "top-level" type of properties and does not check the type-name, but I think this is sufficient for now. WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is to catch the many occurrences of aggregations as a property name?

It seems a bit brittle, but will ensure we can easily use OrderedDictionary for all aggregations property, which is where it should always be used.

... which makes me think that ideally we should have a validation that forbids Dictionary<string, Aggregation> to ensure we haven't forgotten anything.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems a bit brittle, but will ensure we can easily use OrderedDictionary for all aggregations property, which is where it should always be used.

Correct!

... which makes me think that ideally we should have a validation that forbids Dictionary<string, Aggregation> to ensure we haven't forgotten anything.

I don't think there is a good way to detect this using a pattern. We have also cases like this:

Dictionary<string, ApiKeyAggregationContainer>

where we define special containers to restrict the valid aggregations to a subset.

Copy link
Member

@swallez swallez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be more assertive in the usage warning. Also left a few comments.

@@ -88,6 +88,7 @@ export class DictionaryOf {
key: ValueOf
value: ValueOf
singleKey: boolean
ordered: boolean
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side comment: interesting case of a boolean property (singleKey) that evolves into an enum (singleKey, ordered, unordered) as ordered only makes sense when singleKey == false.

We can however keep it as two properties:

  • some languages have their default dictionary keeping order (e.g. PHP, Ruby, JS) and their generators can safely ignore ordered because it makes no difference to them,
  • some languages don't have ordered dictionaries (e.g. Go) and their generators will therefore also ignore ordered.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was thinking about an enum, but I as well didn't want to break all existing tooling 😅

@@ -500,7 +515,7 @@ export function modelEnumDeclaration (declaration: EnumDeclaration): model.Enum
}

if (typeof tags.es_quirk === 'string') {
type.esQuirk = tags.es_quirk
type.esQuirk = tags.es_quirk.replace(/\r/g, '')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Window-ism?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes 😵‍💫 I had applied this to most other places a while ago, but seems like I forgot about the quirks. This will ensure deterministic output regardless of being generated on *nix or Windows.

@@ -773,6 +773,13 @@ export default async function validateModel (apiModel: model.Model, restSpec: Ma
}

context.push(`Property '${prop.name}'`)

if (prop.type.kind === 'dictionary_of' && prop.type.ordered) {
if (prop.name !== 'aggregations') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is to catch the many occurrences of aggregations as a property name?

It seems a bit brittle, but will ensure we can easily use OrderedDictionary for all aggregations property, which is where it should always be used.

... which makes me think that ideally we should have a validation that forbids Dictionary<string, Aggregation> to ensure we haven't forgotten anything.

@@ -20,3 +20,5 @@
export class Dictionary<TKey, TValue> {}

export class SingleKeyDictionary<TKey, TValue> {}

export class OrderedDictionary<TKey, TValue> {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should restate the warning here, for people who will discover it in the API spec code.

Copy link
Member

@swallez swallez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also have another occurrence of this in elastic/elasticsearch-java#99 - Pivot.group_config should be ordered (and added in the validation):

group_by?: Dictionary<string, PivotGroupByContainer>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OrderedDictionary<T>
3 participants