Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NH-3488 - Strongly typed Inserts, Updates and Deletes #391

Merged
merged 7 commits into from
Aug 24, 2017

Conversation

gliljas
Copy link
Member

@gliljas gliljas commented Jan 1, 2015

https://nhibernate.jira.com/browse/NH-3488

The bulk of the code in the commit is the unit tests, which is a copy of the tests in Hql/Ast/BulkManipulation.cs.

It would have been nice to reuse the same code but almost all the mappings in the original have casing problems, i.e they refer to lower case properties, but the properties are not lower cased. The HQL tests work, since they refer to the mapped properties, not the real ones. One solution would have been to check against the mapping, but I found the existing behavior strange and other parts of the Linq implementation doesn't do this, so I'll leave it as is for now.

The goal was to provide the new functionality by deriving from the existing Linq system, but that was easier said than done. Lots of code in NHibernate is very hard to derive from, with static calls and massive amounts of logic in the constructors. I decided to create derived types of NHLinqExpression, and handle most of the initialization in the constructor call. I could have created a derived QueryModelVisitor and IntermediateHqlTree too, but opted not to.

The syntax for the update and insert is worth discussing, but I wanted to support both a member initialization version and an explicit one. They could not be implemented with the same method name, since the compiler then sometimes will not be able to distinguish between the overloads.

For inserts we have:

session.Query<Cat>().Insert().As(x=>new Dog{Name=x.Name});

or

session.Query<Cat>().Insert().Into<Dog>(x=>x.Set(y=>y.Name,y=>y.Name));

For updates we have:

session.Query<Cat>().Update().As(x=>new Cat{Name=x.Name + " the cat"});

or

session.Query<Cat>().Update().Assign(x=>x.Set(y=>y.Name,y=>y.Name + " the cat"));

A better name than "Assign" is welcome. Or maybe for "Set".

@hazzik hazzik added this to the 5.0.0 milestone Jan 1, 2015
@gliljas
Copy link
Member Author

gliljas commented Jan 2, 2015

A bit of clarification.

While the delete part was easy enough - it just throws away the select and replaces it with a delete - the update and insert required a bit more. The concept used was to embed the UPDATE SET clause and the INSERT INTO [..] SELECT [..] in a projected SELECT clause.

session.Query<Cat>().Update().As(x=>new Cat{Name=x.Name + " the cat"});

becomes

session.Query<Cat>().Select(x=>new Dictionary<string,object>{{"Name",x.Name + " the cat"}});

By doing this, it is ensured that anything inside the clauses, such as subqueries, parameters etc., is going through the same rewriting process as everything else. It maybe feels like bit of a hack, and a full refactoring of the query modeling could open up for a more explicit path, but right now I think this is OK.

Also, regarding the syntax. The reason the "fluent" stuff was needed was that without it...

session.Query<Cat>().Insert().Into<Dog>(x=>x.Set(y=>y.Name,y=>y.Name));

would have to be

session.Query<Cat>().Insert<Cat,Dog>(x=>x.Set(y=>y.Name,y=>y.Name));

since the compiler can't infer just Dog, and if the base query contains a projection to an anonymous type, type inference is a must.

Also, supporting both..

session.Query<Cat>().Update(x=>new Cat{Name="Missy"});

and

session.Query<Cat>().Update(x=>x.Set(y=>y.Name,"Missy"));

with the same method name is not possible, since the compiler can't distinguish between the two signatures when the lambda parameter is not used, unless the first one is qualified like this:

session.Query<Cat>().Update((Cat x)=>new Cat{Name="Missy"});

A minor nuisance maybe, but also something that could unnecessarily become a frequently asked question.

The reason to use a fluent syntax for the Update and not only the Insert, which required it, was simply consistency.

@jogibear9988
Copy link

Any News to this Pull Req???

@gliljas
Copy link
Member Author

gliljas commented Jun 21, 2016

No, but any input you might have would be welcome. Especially regarding the syntax. I feel quite confident that the implementation itself is solid.

@jogibear9988
Copy link

I think the Syntax is quite good.

We're using NHibernate quite long now in a Project. But not supporting Linq for Deletes and Updates, is a real show stopper.

What are the conflicts wich are not yet resolved?

@jogibear9988
Copy link

Is there any active discussion to merge this?

@gliljas
Copy link
Member Author

gliljas commented Jul 16, 2016

Sorry for leaving you hanging there. Not much discussion, since this was aimed for 5.0.

@hazzik
Copy link
Member

hazzik commented Feb 12, 2017

Hi @gliljas, can you please enable Allow edits from maintainers feature?

Copy link
Member

@fredericDelaporte fredericDelaporte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have locally rebased for having a look at this. It looks really great!

I have made some comments on the syntax as code review, along with some other suggestions.

Maybe adding Future version of executing methods would be nice, but it is not at all important I think.

There are some misses with white-spaces (spaces/tabs) I have fixed on my local branch.

Test cases sometime just do a "no op" for some tests, based on db support, maybe should they Assert.Ignore instead. A bunch of tests may have an additional teardown failure due to unclosed session if they fail, due to lack of using.

(Sorry for having messed up with single comments instead of "start review".)

/// <param name="assignments">The assignments.</param>
/// <param name="versioned">if set to <c>true</c> [versioned].</param>
/// <returns></returns>
public int Assign(Action<Assignments<T, T>> assignments, bool versioned = false)
Copy link
Member

@fredericDelaporte fredericDelaporte Mar 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be versioned by default for entity having versioning, if possible? (I guess there are some hurdles indeed.)
The xml doc should maybe be written : <c>true</c> for having the update increments the entities version, <c>false</c> otherwise.

Instead of Assign, maybe Values would do.

using NHibernate.Hql.Ast.ANTLR;
using NHibernate.Util;

namespace NHibernate.Test.NHSpecificTest.NH3438
Copy link
Member

@fredericDelaporte fredericDelaporte Mar 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo, it should be 3488. All files and the older of the test cases for NH3488 have this typo. (Already renamed on my local branch.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops!

public Assignments<TInput, TOutput> Set<TProp>(Expression<Func<TOutput, TProp>> property, Expression<Func<TInput, TProp>> expression)
{
ArgumentUtility.CheckNotNull("property", property);
ArgumentUtility.CheckNotNull("expression", expression);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This utility seems to be no more available after rebasing, I have re-coded it "old style" on my local branch.

/// <typeparam name="TOutput">The type of the output.</typeparam>
/// <param name="assignmentActions">The assignments.</param>
/// <returns></returns>
public int Into<TOutput>(Action<Assignments<TInput, TOutput>> assignmentActions)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe renaming everywhere TInput and TOutput as TSource and TTarget would be more explicit. I have also thought about TSelected and TInserted but well, I am not sure.
And xml doc for TTarget could be "The type of the entities to insert."
Returns: "Number of inserted rows/entities"? What about insert into entities dispatched into join tables?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points. I agree fully.

/// <typeparam name="T"></typeparam>
/// <param name="query">The query.</param>
/// <returns></returns>
public static int Delete<T>(this IQueryable<T> query)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contrary to Update and Insert, this one causes immediate execution. Maybe should we require an additional Execute call for aligning with Update and Insert.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly. I think that Delete().Execute() would look a bit strange, with a double verb.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, maybe would it be too "noisy" just for consistency.

@gliljas
Copy link
Member Author

gliljas commented Mar 31, 2017

"Allow edits from maintainers" has been enabled.

@fredericDelaporte
Copy link
Member

May I force push my rebase?

@gliljas
Copy link
Member Author

gliljas commented Apr 1, 2017

Yes!

@fredericDelaporte
Copy link
Member

The tests are in fact the old BulkManipulationFixture tests ported to Linq, thus some old pattern. A bunch of those tests are failing for some dialects.

Either we add them to the list of failings tests, or we fix them for being ignored on unsupported dialects.

Some attempts at that were present, such as a test on Dialect.SupportsSubqueryOnMutatingTable, but this property appears to be never set to false for any dialect. (In Hibernate, it is, see MySQLDialect by example.)

I am in the process of modernizing those Linq tests and fixing their conditions.

@@ -866,6 +910,11 @@ public void UpdateSetNullOnJoinedSubclass()
[Test]
public void DeleteWithSubquery()
{
if (Dialect is MsSqlCeDialect)
{
Assert.Ignore("Test failing on Ms SQL CE.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fails too in original BulkManipulationFixture, with a slightly different SQL query, but same error message. As this DB is deprecated since 2013, I do not intend to investigate more what is going on for this case.

@fredericDelaporte
Copy link
Member

Under Firebird InsertToComponent takes 54 minutes and gets timed-out.
Trying that locally, the test gets stuck at executing:

insert into SimpleClassWithComponent ( id, name_first )
	select gen_id(hibernate_sequence, 1 ), cast(@p0 as VARCHAR(255)) as col_0_0_
		from SimpleClassWithComponent simpleclas0_

And the db size endlessly grows... It looks like this kind of query under Firebird causes a recursive insert, as if newly inserted rows where taken into account in the select and re-inserted... I am going to ignore that test for Firebird.

Under SqlServerCe, there is a TeamCity build failure I do not understand. The log is huge due to the numerous known failing tests (700+), and at the end we can find:

[16:17:21][NHibernate.Test.VisualBasic.dll] NHibernate.Test.VisualBasic.Issues.NH3302.Fixture.LinqStringLikeQuestion
[16:17:21][NHibernate.Test.VisualBasic.Issues.NH3302.Fixture.LinqStringLikeQuestion] Test ignored: NHibernate.Test.VisualBasic.Issues.NH3302.Fixture.LinqStringLikeQuestion, ignore reason: OneTimeSetUp: Not fixed yet.
[16:17:21][call] common.remove-connection-settings-from-app-config
[16:17:21][common.remove-connection-settings-from-app-config] xmlpoke
[16:17:21][xmlpoke] No matching nodes were found with XPath expression '/configuration/nhibernate/add[@key='hibernate.connection.connection_string']/@value'.
[16:17:21][call] common.run-database-tests
[16:17:21][Step 1/1] verify-test-results (3s)
[16:17:21][verify-test-results] if (3s)
[16:17:21][if] copy
[16:17:21][copy] Copying 1 file to 'D:\BuildAgent\work\b0ecfa5dcda2234\build\NHibernate-5.0.0.Alpha1-debug\bin\net-4.0\test-results'.
[16:17:24]
[Step 1/1] D:\BuildAgent\work\b0ecfa5dcda2234\teamcity.build(30,5):
Function call failed.
Expression: ${testResult::CompareResults(teamcity.current.result, teamcity.last.result)}
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[16:17:24][Step 1/1] Process exited with code 1
[16:17:24][Step 1/1] NAnt output
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.EntityModeToTuplizerPerf.Fixture.VerifyEntityModeNotFound
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.NH2505.Fixture.WhenQueryComplexEqualToComplexThentUseTheCaseConstructorForBoth
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.NH2505.Fixture.WhenQueryConstantEqualToMemberThenDoesNotUseTheCaseConstructor
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.NH2505.Fixture.WhenQueryConstantEqualToNullableMemberThenUseTheCaseConstructorForMember
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.NH2505.Fixture.WhenQueryConstantEqualToNullableMemberValueThenDoesNotUseTheCaseConstructorForMember
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.NH2505.Fixture.WhenQueryConstantNotEqualToMemberThenDoesNotUseTheCaseConstructor
[16:17:24][NAnt output]     PASS    - NHibernate.Test.NHSpecificTest.NH2505.Fixture.WhenQueryConstantNotEqualToNullableMemberThenUseTheCaseConstructorForMember
[16:17:24][NAnt output]     
[16:17:24][NAnt output]     *** Tests ignored since last recorded results ***
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.DeleteOnJoinedSubclass
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.DeleteOnMappedJoin
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.DeleteRestrictedOnManyToOne
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.DeleteUnionSubclassAbstractRoot
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.DeleteUnionSubclassConcreteSubclass
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.SimpleDeleteOnAnimal
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateMultiplePropertyOnAnimal
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateOnAnimal
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateOnComponent
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateOnMammal
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateOnManyToOne
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateSetNullOnJoinedSubclass
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateSetNullUnionSubclass
[16:17:24][NAnt output]     IGNORED - NHibernate.Test.Hql.Ast.BulkManipulation.UpdateWithWhereExistsSubquery
[16:17:24][NAnt output]     
[16:17:24][NAnt output]     *** Tests broken since last recorded results ***
[16:17:24][NAnt output]     None
[16:17:24][NAnt output]     
[16:17:24][NAnt output] 
[16:17:24][NAnt output] Total time: 453.4 seconds.
[16:17:24][NAnt output] 
[16:17:35][Step 1/1] Process exited with code 1
[16:17:35][Step 1/1] Step NAnt failed

@fredericDelaporte
Copy link
Member

I have changed some xml comments according to earlier discussion, with some minor code cleanup. And made a third try at fixing the tests for accounting dialect unsupported features.

@fredericDelaporte
Copy link
Member

SqlServerCe TeamCity build still failing, but looks like a test tooling failure.

[19:14:09][Step 1/1] verify-test-results (2s)
[19:14:09][verify-test-results] if (2s)
[19:14:10][if] copy
[19:14:10][copy] Copying 1 file to 'D:\BuildAgent\work\b0ecfa5dcda2234\build\NHibernate-5.0.0.Alpha1-debug\bin\net-4.0\test-results'.
[19:14:12]
[Step 1/1] D:\BuildAgent\work\b0ecfa5dcda2234\teamcity.build(30,5):
Function call failed.
Expression: ${testResult::CompareResults(teamcity.current.result, teamcity.last.result)}
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[19:14:12][Step 1/1] Process exited with code 1

@hazzik
Copy link
Member

hazzik commented Apr 2, 2017 via email

@hazzik
Copy link
Member

hazzik commented Apr 2, 2017 via email

@hazzik
Copy link
Member

hazzik commented Apr 2, 2017

*** Tests new (failed) since last recorded results ***
FAIL    - NHibernate.Test.Hql.Ast.BulkManipulation.SimpleInsertFromAggregate

@hazzik
Copy link
Member

hazzik commented Apr 2, 2017

Otherwise we have to find among more than seven hundred usually failing tests which ones are "new".

The comparision.txt here to do that for us.

@fredericDelaporte
Copy link
Member

Ok, I was not knowing we had this file for finding out which test causes trouble.

@fredericDelaporte
Copy link
Member

I have updated the Linq reference documentation for documenting this new feature.
And fourth try at conditionally ignoring tests incompatible with some dialects.

@fredericDelaporte
Copy link
Member

I think we can hide DML operation into a specific namespace (NHibernate.Linq.Dml or NHibernate.Linq.Update/...Insert/...Delete).

Great for whose who do not wish to see that everywhere they use queryables and Nhibernate.Linq, but quite bad for discover-ability.

NHibernate.Linq IMO already illustrates the pain of poor discover-ability due to being hidden in a namespace, causing many people to not easily find how to perform a Linq query with NHibernate. (And with some trying to use queryover as a link provider instead, as frequently seen on Stack Overflow.) We all knows most developers first try to use the software, and eventually go seeking answers only if failing using it. Discover-ability is a very important aspect IMO, we should no more add features in a poorly discover-able way I think.

@fredericDelaporte
Copy link
Member

The linq2db alike insert syntax with individual setter was not working, I have updated my comparison comment.

We are back to initial issue with inserts indeed.

@fredericDelaporte
Copy link
Member

fredericDelaporte commented Jul 19, 2017

I have pushed an alternate syntax proposal. Commit to be dropped if preferring the previous one.

Insert

Old syntax

By single selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Insert()
    .As(c => new Pickup { Id = c.Id, Vin = c.Vin, Owner = c.Owner });

By single anonymous selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Insert()
    .As<Pickup>(c => new { Id = c.Id, Vin = c.Vin, Owner = c.Owner });

By individual setters

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Insert()
    .Into<Pickup>(
        a => a // assignments object
            .Set(p => p.Id, c => c.Id)
            .Set(p => p.Vin, c => c.Vin)
            .Set(p => p.Owner, "The owner"));

New linq2db alike

By single selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .InsertInto(c => new Pickup { Id = c.Id, Vin = c.Vin, Owner = c.Owner });

By single anonymous selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    // Sadly, have to re-specify the source type.
    .InsertInto<Car, Pickup>(c => new { Id = c.Id, Vin = c.Vin, Owner = c.Owner });

By individual setters

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    // Value are not yielding a queryable anyway, so better be explicit
    // with the fact we are toggling to a builder mode. Allows not re-specifying source type.
    .InsertBuilder().Into<Pickup>()
    .Value(p => p.Id, c => c.Id)
    .Value(p => p.Vin, c => c.Vin)
    .Value(p => p.Owner, "The owner")
    .Insert();

Update

Old syntax

By single selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Update() // .UpdateVersioned() available too
    .As(c => new Car { Vin = c.Vin + "b", Owner = c.Owner }); 

By single anonymous selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Update() // .UpdateVersioned() available too
    .As(c => new { Vin = c.Vin + "b", Owner = c.Owner }); 

By individual setters

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Update() // .UpdateVersioned() available too
    .Assign(
        a => a // assignments object
            .Set(c => c.Vin, c => c.Vin + "b")
            .Set(c => c.Owner, "The owner"));

New linq2db alike

By single selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Update(c => new Car { Vin = c.Vin + "b", Owner = c.Owner }); // Or UpdateVersioned

By single anonymous selector

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    .Update(c => new { Vin = c.Vin + "b", Owner = c.Owner }); // Or UpdateVersioned

By individual setters

session
    .Query<Car>()
    .Where(c => c.UnitsInStock == 0)
    // For consistency with insert case, and for telling explicitly we are switching away from
    // queryable to a builder mode.
    .UpdateBuilder()
    .Set(c => c.Vin, c => c.Vin + "b")
    .Set(c => c.Owner, "The owner")
    .Update(); // Or UpdateVersioned

Delete

Same with both.

@jogibear9988
Copy link

I think in linq2db you could use some set-functions, and after that you could add an additional where before you update (i've not tested, but i think it will work).
will thos work with your syntax also?

@fredericDelaporte
Copy link
Member

No, we no more have a queryable from the point we start using one of these new methods. And that looks to be the same with linq2b: calling Set yields an IUpdatable which is not a queryable either, and seems to only allow specifying additional Set and call a final Update.

Anyway, maybe this would look more natural from a SQL standpoint to be able to put the Where after, but well, Linq is not SQL and users are already used to this kind of clauses reversal, like putting Select last.

@fredericDelaporte
Copy link
Member

fredericDelaporte commented Aug 22, 2017

Rebased, bulk manipulation fixture (non linq) refactoring dropped.

@hazzik hazzik changed the title NH-3488 - Strongly Typed Updates and Deletes NH-3488 - Strongly typed Inserts, Updates and Deletes Aug 24, 2017
@hazzik hazzik merged commit 333eed6 into nhibernate:master Aug 24, 2017
@hazzik hazzik added r: Fixed and removed to review labels Aug 24, 2017
@dzmitry-lahoda
Copy link

This stuff will make NH live in enterprise much longer. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants