Could anyone more across the detail of this chime in on what this means for the 'average' .NET developer?
I rely heavily on LINQ calls in a .NET (Core) Web App, should I replace these with Zlinq calls?
Or is this only helpful in case you want to do LINQ operations on let's say 1000's of objects that would get garbage collected at the end of a game loop?
In theory .Net 10 should make this obsolete, the headline features[1] are basically all about this. In practice, well, it's heuristics, I'm adding this to a particularly performance sensitive project right now :)
Edit: what's also nice is that C# recognizes Linq as a contract. So long as this has the correct method names and signatures (it does), the Linq syntax will light up automatically. You can also use this trick for your own home-grown things (add Select, Join, Where, etc. overloads) if the Linq syntax is something you like.
Could you elaborate? I don’t see anything about improving the performance of enumerator. Zlinq appears to remove the penalty of allocating enumerators on the heap to be garbage collected. The link you sent mention improvements, but I don’t see how they lead to linq avoiding heap allocations.
Not just that but Zlinq also works across all C# environments it seems including versions embedded in game engines like Godot, Unity, .NET Standard, .NET 8 and 9.
I believe they're referring to the stack allocation improvements, which would ideally allow all the LINQ temporary objects to live on the stack. I'm not sure whether it does in practice though.
This is great. I've worked on production .NET services and we often had to avoid LINQ in hot paths because of the allocations. Reimplementing functions with for-loops and other structures was time-consuming and error-prone compared to LINQ method chaining. Chaining LINQ methods is extremely powerful; like filter, map, and reduce in JS but with a bunch of other operators and optimizations. I wish more languages had something like it.
What are the advantages of this over using higher order functions? In Ruby I can do list.map { }.select { } …. That feels more natural (doesn’t require special language support), has a very rich set of functions (group_by, chunk_while, etc.), and is something the user can extend with their own methods (if they don’t mind monkeypatching)
LINQ is higher-order functions - Ruby `map` is 'Enumerable.Select`, Ruby `select` is `Enumerable.Where` etc.
The special syntax is really just syntactic sugar on top of all this that makes things a little bit more readable for complex queries because e.g. you don't have to repeat computed variables every time after binding it once in the chain. Consider:
from x in xs
where x.IsFoo
let y = Frob(x)
where y.IsBar
let z = Frob(y)
where z.IsBaz
order by x, y descending, z
select z;
If you were to rewrite this with explicit method calls and lambdas, it becomes something like:
Note how it needs to weave `x` and `y` through all the Select/Where calls so that they can be used for ordering in the end here, whereas with syntactic sugar the scope of `let` extends to the remainder of the expression (although under the hood it still does roughly the same thing as the handwritten code).
Like mentioned, groupby,etc all operate on functions, (map/reduce/filter/etc) are just named differently (select/aggregate/where/etc).
What makes people love Linq is that it handles 2 different cases (identical syntax but different backing objects).
1: The in-memory variant does things lazily, select/aggregate/where produce enumeration objects so after the chain you add ToArray, ToList, ToDictionary,etc and the final object is built lazily with most of the chain executed on as few objects as possible (thus if you have an effective Where at the start, the rest of the pipeline will do very little work and very few allocations).
2: The compiler also helps the libraries by providing syntax-tree's, thus database Linq providers just translates the Linq to SQL and sends it off to the server, letting the server do the heavy lifting _with indexes_ so that we can query tables of arbitrary sizes with regular C# Linq syntax very quickly without most of it never going over the network.
Linq came out as part of a set of features that addressed the comforts of languages like Ruby. I don't know if they considered Ruby to be a threat at the time but they put a bunch of marketing power behind the release of linq. The way I understand it, as someone who jumped from C# to Ruby just around the time Linq came out is that it's a DSL for composing higher order functions as well as a library of higher order functions.
I always liked how the C# team took inspiration from other language ecosystems. Usually they do it with a lot more taste than the C++ committee. The suppose the declarative linq syntax gives the compiler some freedom of optimization, but I feel Ruby's do syntax makes higher order functions shine in a way that's only surpassed by functional languages like Haskell or Lisp.
LINQ methods are higher order functions ? LINQ syntax is just sugar and probably a design mistake (dead weight feature that I've only seen abused to implement monadic composition by insane people).
And Ruby doesn't even enter this conversation if we're talking about these kinds of optimizations - it's an order of magnitude away from what you're aiming from if you're unrolling sequence operations in C#.
This is cool - excited to try it - I would note that I've been a dotnet grunt for almost 15 years now. I am good at it, I know how to use the language, I know the ecosystem - this level of familiarity with the language is just not within my grasp. I can understand the code (mostly) reading it, but I never would have been able to conjure up, let alone implement this. Props to the author.
This is neat, but how does this get away with being zero allocation? It appears to use `Func<T,U>` for its predicates, and since `Func<T>` is a reference type this will allocate. The IL definitely generates definitely seems like it's forming a reference type from what I can tell.
The JIT can optimize this. I know for sure if there's no captures in the lambda it won't allocate. It's likely also smart enough to recognize when a function parameter doesn't have its lifetime extended past the caller's, which is a case where it would also be possible to not allocate.
It's 2 different syntactically identical API's under an umbrella.
1: IEnumerable<T> that works lazily in-memory (and similar to the authors improvement) can be done in any language with first class functions, see the authors linq.js or Java's streams library (it's not entirely the same as a chain of map/reduce/filter since it's lazy but that's mostly not a drawback since it improves performance by removing temporary storage objects).
2: IQueryable<T> is the really magical part though, by specifying certain wrapper types the compiler is informed that the library expects an bound syntax tree(AST) of the expression you write, the library can then translate the syntax tree to SQL queries and send the query to the server to be processed.
Thus huge tables can be efficiently queried by just writing regular C# and never touch SQL. In most ORM's it's annoying or have impedance mistmatches but with EF you can write code and be fairly certain that it'll produce a good SQL query since the entire Linq syntax is fairly close to SQL.
C# can turn lambdas into expression trees at runtime allowing libraries like EF to transform code like `db.Products.Where(p => p.Price < 100).Select(p => p.Name);` right to SQL by iterating the structure of that code. JavaScript ORMs would be revolutionized if they had this ability.
Not easily. There's no built-in way to access the abstract syntax tree (or equivalent) of a function at run time. The best thing you can do is to obtain the source code of a function using `.toString()` and then use a separate JS parser to process it, but that's not a very realistic option.
There is a limited form of such "expression rewriting" using tagged template strings introduced in ES2015. But it wouldn't be particularly useful for the ORM case.
Of course, if you want to run this against a query provider, you do need compiler support to instead give you an expression tree, and provider to process it and convert them to a language (often sql) that database can understand.
C# is definitely not the only possible language, but some things stand out:
1. You can extend other people's interfaces. If you care about method chaining, _something_ like that is required (alternative solutions include syntactic support for method chaining as a generic function-call syntax).
2. The language has support for "code as data." The mechanism is expression trees, but it's really powerful to be able to use method chaining to define a computation and then dispatch it to different backends, complete with optimization steps.
3. The language has a sub-language as a form of syntactic sugar, allowing certain blessed constructs to be written as basically inline SQL with full editor support.
Compare C# ORMs to JS/TS for example. In C#, it is possible to use expression trees to build queries. In TS, the only options are as strings or using structural representation of the trees.
A simple language can make written in it code complex. A complex language can make code simpler. It's not a perfect rule or anything but it's been my experience that attempts at making simpler programming languages just put more demands on the programmer. The lack of expressive power has to be paid for somewhere.
Basic LINQ on in-memory collections isn't really that different from what you have in other languages. Where things get special is the LINQ used by Entity Framework. It operates on expressions, which allow code to be compiled into the application and manipulated at runtime. For example, the lambda expression that you pass to Where() will be examined by an EF query provider that translates it into the where clause for a SQL query.
I feel like pretty much every language with generics has a LINQ, like functools/itertools in Python, lodash for javascript. It’s just a different expression of the same ideas.
Nope, very different. Depending on whether the expression is on an Enumerable or a Queryable, the compiler generates an anonymous function or an AST. That is, you can get "code as data" as in say Lisp; and allows expressions to be converted to say SQL based on the backend.
ZLinq relies on its own enumerable type called ValueEnumerable, which is a struct. While it would probably work when using this as a drop-in replacement and re-compiling, things will be more complicated in larger applications. There might be some code that depends on the exact signature of the Linq methods. This might not even be detectable in cases involving reflection and could break stuff silently.
Adding another enumerable type would be a very large change that could effectively double the API surface of the entire ecosystem. This could take some time. Some places still don't even support Span<T>. Also there were some design decisions related to Linq where the number of overloads were a consideration.
Adding this API to .NET could probably be done with that extension method that converts to ValueEnumerable. But without support for that enumerable, this would pretty much be a walled garden where you have to convert back and forth between different enumerable types. Not that great if you'd ask me, but possible I guess.
I can easily imagine the kind of person that goes out and builds something like this would have little patience with the bureaucracy of getting it integrated into .NET.
I'd say it's less about bureaucracy and more about what the .NET team has to consider when they make sweeping changes.
Backwards compatibility, security, edge cases, downstream effects on other libraries that are reliant on LINQ, etc.
One guy with an optional library can break things. If the .NET team breaks things in LINQ, it's going to be a bad, bad time for a lot of people.
I think Evan You's approach with Vue is really interesting. Effectively, they have set up a build pipeline that includes testing major downstream projects as well for compatibility. This means that when the Vue team build something like "Vapor Mode" for 3.6, they've already run it against a large body of community projects to check for breaking changes and edge cases. You can see some of the work they do in this video: https://www.youtube.com/watch?v=zvjOT7NHl4Q
1. Fedora in collaboration with GCC maintainers keep GCC on the bleeding edge so it can be used to compile the whole Fedora corpus. This validates the compiler against a set of packages which known to work with the previous GCC
2. I think the rust team also builds all crates on crates.io when working on `rustc`. It seems they created a tool to achieve that: https://github.com/rust-lang/crater
I would assume the .NET guys have something similar already but maybe there’s not enough open code to do that
Rust also has the advantage of having no ABI. Binary interface is a whole lot more difficult to maintain than code interface.
C# has multiple technologies built to deal with ABI (though it probably all goes unused these days with folder-based deployments, you really need the GAC for it to work).
You have to add an extra function call at the start of the Linq method chain in order to make it zero-allocation. So I don't think it would break backwards compatibility. But adding it does create an additional maintenance burden.
From some experience, the MS guys are actually really eager to get more outside help and many will help guide you through the process if you have something to offer.
Every release has a fairly decent amount of fixes and additions from outside contributors, and while I can see a lot of to/fro on the PRs to get them through, it's probably not quite as bad as you'd expect.
Using reference types are more idiomatic in C#. To some degree they are less bug prone as well (they can be passed around without issue). Most of the core library use them instead of starting with value types and boxing.
The Task library has successfully added ValueTask but it took some doing. LINQ on the other hand can be replaced with unrolled loops or libraries more easily so the pressure just hasn't been there.
I could see something happening in the future but it would take a lot of be work.
There are some minor breaking changes like the order of iteration is not always the same as the official Linq implementation, or Sum might give different values due to checked vs unchecked summing. Probably not an issue for most people, but a subtle breaking change nevertheless.
From looking at the blog post I suspect the explosion of generic instances could be a serious problem for code size and startup time, but that's probably solvable somehow. The performance certainly seems impressive.
The way LINQ currently works by default makes aggressive use of interfaces like IEnumerable to hide the actual types being iterated over. This has performance consequences (which is part of why ZLinq can beat it) but it has advantages - for example, the same implementation of Where<T>(seq) can be used for various T's instead of having to JIT or AOT-compile a unique body for every distinct class you iterate over.
From looking at ZLinq it seems like it would potentially have an explosion of unique generic struct types as your queries get more complex, since for it to work you potentially end up with types vaguely resembling Query3<Query2<Query1<T>>>>. But it might not actually be that bad in practice.
Internal iterators, expression trees, etc. Many LINQ variants (depends on the data source) also do not execute the chain step by step but building first an expression tree and then translating that into native query syntax (e.g. SQL)
LINQ methods build up a chain of IEnumerable/IQueryable objects, which then build up chains of IEnumerator objects each time you iterate.
These types are all .NET interfaces, which are reference types, so they're allocated on the heap. .NET's escape analysis can sometimes move reference types to the stack, but this feature is currently very limited and didn't even exist until .NET 9.
ZLinq uses generic structs to prevent these allocations at the expense of some really verbose intermediate types.
Could anyone more across the detail of this chime in on what this means for the 'average' .NET developer?
I rely heavily on LINQ calls in a .NET (Core) Web App, should I replace these with Zlinq calls?
Or is this only helpful in case you want to do LINQ operations on let's say 1000's of objects that would get garbage collected at the end of a game loop?
I don't use .NET, but always thought LINQ a really interesting part of it.
In theory .Net 10 should make this obsolete, the headline features[1] are basically all about this. In practice, well, it's heuristics, I'm adding this to a particularly performance sensitive project right now :)
Edit: what's also nice is that C# recognizes Linq as a contract. So long as this has the correct method names and signatures (it does), the Linq syntax will light up automatically. You can also use this trick for your own home-grown things (add Select, Join, Where, etc. overloads) if the Linq syntax is something you like.
[1]: https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotn...
Could you elaborate? I don’t see anything about improving the performance of enumerator. Zlinq appears to remove the penalty of allocating enumerators on the heap to be garbage collected. The link you sent mention improvements, but I don’t see how they lead to linq avoiding heap allocations.
Not just that but Zlinq also works across all C# environments it seems including versions embedded in game engines like Godot, Unity, .NET Standard, .NET 8 and 9.
Select doesn’t have to return IEnumerable. A struct that exposes the same methods will work. So allocate-free foreach is very possible.
But that's what ZLinq does, not what the upcoming changes to .NET do. What's your point?
I believe they're referring to the stack allocation improvements, which would ideally allow all the LINQ temporary objects to live on the stack. I'm not sure whether it does in practice though.
Unfortunately, those improvements don't work for Linq.
Some notes on why this is so here: https://github.com/dotnet/runtime/blob/main/docs/design/core...
Aw, I had no idea that it didn't work out. If they work that out I'd put good money on a colossal perf boost across the board.
This is great. I've worked on production .NET services and we often had to avoid LINQ in hot paths because of the allocations. Reimplementing functions with for-loops and other structures was time-consuming and error-prone compared to LINQ method chaining. Chaining LINQ methods is extremely powerful; like filter, map, and reduce in JS but with a bunch of other operators and optimizations. I wish more languages had something like it.
What are the advantages of this over using higher order functions? In Ruby I can do list.map { }.select { } …. That feels more natural (doesn’t require special language support), has a very rich set of functions (group_by, chunk_while, etc.), and is something the user can extend with their own methods (if they don’t mind monkeypatching)
LINQ is higher-order functions - Ruby `map` is 'Enumerable.Select`, Ruby `select` is `Enumerable.Where` etc.
The special syntax is really just syntactic sugar on top of all this that makes things a little bit more readable for complex queries because e.g. you don't have to repeat computed variables every time after binding it once in the chain. Consider:
If you were to rewrite this with explicit method calls and lambdas, it becomes something like: Note how it needs to weave `x` and `y` through all the Select/Where calls so that they can be used for ordering in the end here, whereas with syntactic sugar the scope of `let` extends to the remainder of the expression (although under the hood it still does roughly the same thing as the handwritten code).Like mentioned, groupby,etc all operate on functions, (map/reduce/filter/etc) are just named differently (select/aggregate/where/etc).
What makes people love Linq is that it handles 2 different cases (identical syntax but different backing objects).
1: The in-memory variant does things lazily, select/aggregate/where produce enumeration objects so after the chain you add ToArray, ToList, ToDictionary,etc and the final object is built lazily with most of the chain executed on as few objects as possible (thus if you have an effective Where at the start, the rest of the pipeline will do very little work and very few allocations).
2: The compiler also helps the libraries by providing syntax-tree's, thus database Linq providers just translates the Linq to SQL and sends it off to the server, letting the server do the heavy lifting _with indexes_ so that we can query tables of arbitrary sizes with regular C# Linq syntax very quickly without most of it never going over the network.
Linq came out as part of a set of features that addressed the comforts of languages like Ruby. I don't know if they considered Ruby to be a threat at the time but they put a bunch of marketing power behind the release of linq. The way I understand it, as someone who jumped from C# to Ruby just around the time Linq came out is that it's a DSL for composing higher order functions as well as a library of higher order functions.
I always liked how the C# team took inspiration from other language ecosystems. Usually they do it with a lot more taste than the C++ committee. The suppose the declarative linq syntax gives the compiler some freedom of optimization, but I feel Ruby's do syntax makes higher order functions shine in a way that's only surpassed by functional languages like Haskell or Lisp.
LINQ methods are higher order functions ? LINQ syntax is just sugar and probably a design mistake (dead weight feature that I've only seen abused to implement monadic composition by insane people).
And Ruby doesn't even enter this conversation if we're talking about these kinds of optimizations - it's an order of magnitude away from what you're aiming from if you're unrolling sequence operations in C#.
This is cool - excited to try it - I would note that I've been a dotnet grunt for almost 15 years now. I am good at it, I know how to use the language, I know the ecosystem - this level of familiarity with the language is just not within my grasp. I can understand the code (mostly) reading it, but I never would have been able to conjure up, let alone implement this. Props to the author.
This is neat, but how does this get away with being zero allocation? It appears to use `Func<T,U>` for its predicates, and since `Func<T>` is a reference type this will allocate. The IL definitely generates definitely seems like it's forming a reference type from what I can tell.
The JIT can optimize this. I know for sure if there's no captures in the lambda it won't allocate. It's likely also smart enough to recognize when a function parameter doesn't have its lifetime extended past the caller's, which is a case where it would also be possible to not allocate.
To add on that, you can define your lambdas as static to make sure you're not capturing anything by mistake.
Something like dates.Where(static x => x.Date > DateTime.Now)
What features does C# has that makes LINQ possible in it and not in other languages?
It's 2 different syntactically identical API's under an umbrella.
1: IEnumerable<T> that works lazily in-memory (and similar to the authors improvement) can be done in any language with first class functions, see the authors linq.js or Java's streams library (it's not entirely the same as a chain of map/reduce/filter since it's lazy but that's mostly not a drawback since it improves performance by removing temporary storage objects).
2: IQueryable<T> is the really magical part though, by specifying certain wrapper types the compiler is informed that the library expects an bound syntax tree(AST) of the expression you write, the library can then translate the syntax tree to SQL queries and send the query to the server to be processed.
Thus huge tables can be efficiently queried by just writing regular C# and never touch SQL. In most ORM's it's annoying or have impedance mistmatches but with EF you can write code and be fairly certain that it'll produce a good SQL query since the entire Linq syntax is fairly close to SQL.
C# can turn lambdas into expression trees at runtime allowing libraries like EF to transform code like `db.Products.Where(p => p.Price < 100).Select(p => p.Name);` right to SQL by iterating the structure of that code. JavaScript ORMs would be revolutionized if they had this ability.
An interesting thing about expression trees is that with JIT they can be compiled at runtime, but with AOT they are interpreted at runtime.
> JavaScript ORMs would be revolutionized if they had this ability.
Is this possible in JavaScript?
Not easily. There's no built-in way to access the abstract syntax tree (or equivalent) of a function at run time. The best thing you can do is to obtain the source code of a function using `.toString()` and then use a separate JS parser to process it, but that's not a very realistic option.
There is a limited form of such "expression rewriting" using tagged template strings introduced in ES2015. But it wouldn't be particularly useful for the ORM case.
It's part of the compiler - ast. Linq has two forms - one in the linq ordinary syntax
from x select x.name
And other is just lambda with anonymous types and so on.
For the lambda syntax, you can just do this: https://www.npmjs.com/package/linq
Of course, if you want to run this against a query provider, you do need compiler support to instead give you an expression tree, and provider to process it and convert them to a language (often sql) that database can understand.
There seems to be some transpilers, or things like that - but i don't know what the state of the art is on this: https://github.com/sinclairzx81/linqbox
C# is definitely not the only possible language, but some things stand out:
1. You can extend other people's interfaces. If you care about method chaining, _something_ like that is required (alternative solutions include syntactic support for method chaining as a generic function-call syntax).
2. The language has support for "code as data." The mechanism is expression trees, but it's really powerful to be able to use method chaining to define a computation and then dispatch it to different backends, complete with optimization steps.
3. The language has a sub-language as a form of syntactic sugar, allowing certain blessed constructs to be written as basically inline SQL with full editor support.
Expression trees are highly underrated.
Compare C# ORMs to JS/TS for example. In C#, it is possible to use expression trees to build queries. In TS, the only options are as strings or using structural representation of the trees.
Compare this:
To the equivalent in Prisma (structural representation of the tree): Yikes! Look how dicey that gets with even a basic query!I get that Go maintainers want to keep things simple, but this stuff is pretty useful.
A simple language can make written in it code complex. A complex language can make code simpler. It's not a perfect rule or anything but it's been my experience that attempts at making simpler programming languages just put more demands on the programmer. The lack of expressive power has to be paid for somewhere.
Basic LINQ on in-memory collections isn't really that different from what you have in other languages. Where things get special is the LINQ used by Entity Framework. It operates on expressions, which allow code to be compiled into the application and manipulated at runtime. For example, the lambda expression that you pass to Where() will be examined by an EF query provider that translates it into the where clause for a SQL query.
I feel like pretty much every language with generics has a LINQ, like functools/itertools in Python, lodash for javascript. It’s just a different expression of the same ideas.
Nope, very different. Depending on whether the expression is on an Enumerable or a Queryable, the compiler generates an anonymous function or an AST. That is, you can get "code as data" as in say Lisp; and allows expressions to be converted to say SQL based on the backend.
Is there a reason these sort of improvements cannot be contributed back into .NET itself?
ZLinq relies on its own enumerable type called ValueEnumerable, which is a struct. While it would probably work when using this as a drop-in replacement and re-compiling, things will be more complicated in larger applications. There might be some code that depends on the exact signature of the Linq methods. This might not even be detectable in cases involving reflection and could break stuff silently.
Adding another enumerable type would be a very large change that could effectively double the API surface of the entire ecosystem. This could take some time. Some places still don't even support Span<T>. Also there were some design decisions related to Linq where the number of overloads were a consideration.
Adding this API to .NET could probably be done with that extension method that converts to ValueEnumerable. But without support for that enumerable, this would pretty much be a walled garden where you have to convert back and forth between different enumerable types. Not that great if you'd ask me, but possible I guess.
I can easily imagine the kind of person that goes out and builds something like this would have little patience with the bureaucracy of getting it integrated into .NET.
I'd say it's less about bureaucracy and more about what the .NET team has to consider when they make sweeping changes.
Backwards compatibility, security, edge cases, downstream effects on other libraries that are reliant on LINQ, etc.
One guy with an optional library can break things. If the .NET team breaks things in LINQ, it's going to be a bad, bad time for a lot of people.
I think Evan You's approach with Vue is really interesting. Effectively, they have set up a build pipeline that includes testing major downstream projects as well for compatibility. This means that when the Vue team build something like "Vapor Mode" for 3.6, they've already run it against a large body of community projects to check for breaking changes and edge cases. You can see some of the work they do in this video: https://www.youtube.com/watch?v=zvjOT7NHl4Q
I think this approach predates Vue.
I know of two examples:
1. Fedora in collaboration with GCC maintainers keep GCC on the bleeding edge so it can be used to compile the whole Fedora corpus. This validates the compiler against a set of packages which known to work with the previous GCC
2. I think the rust team also builds all crates on crates.io when working on `rustc`. It seems they created a tool to achieve that: https://github.com/rust-lang/crater
I would assume the .NET guys have something similar already but maybe there’s not enough open code to do that
Rust also has the advantage of having no ABI. Binary interface is a whole lot more difficult to maintain than code interface.
C# has multiple technologies built to deal with ABI (though it probably all goes unused these days with folder-based deployments, you really need the GAC for it to work).
IIRC perl tested new releases by running all the unit tests in the CPAN library, waaaaay back when.
They still do and investigate each failure. If the end result is that the library is “wrong” tickets and patches get sent to the library maintainers.
You have to add an extra function call at the start of the Linq method chain in order to make it zero-allocation. So I don't think it would break backwards compatibility. But adding it does create an additional maintenance burden.
From some experience, the MS guys are actually really eager to get more outside help and many will help guide you through the process if you have something to offer.
Every release has a fairly decent amount of fixes and additions from outside contributors, and while I can see a lot of to/fro on the PRs to get them through, it's probably not quite as bad as you'd expect.
Using reference types are more idiomatic in C#. To some degree they are less bug prone as well (they can be passed around without issue). Most of the core library use them instead of starting with value types and boxing.
The Task library has successfully added ValueTask but it took some doing. LINQ on the other hand can be replaced with unrolled loops or libraries more easily so the pressure just hasn't been there.
I could see something happening in the future but it would take a lot of be work.
There are some minor breaking changes like the order of iteration is not always the same as the official Linq implementation, or Sum might give different values due to checked vs unchecked summing. Probably not an issue for most people, but a subtle breaking change nevertheless.
See here for more info: https://github.com/Cysharp/ZLinq?tab=readme-ov-file#differen...
I don't see why not: https://github.com/dotnet/runtime/pulls
There's an official process for API change requests: https://github.com/dotnet/runtime/blob/main/docs/project/api...
From looking at the blog post I suspect the explosion of generic instances could be a serious problem for code size and startup time, but that's probably solvable somehow. The performance certainly seems impressive.
The way LINQ currently works by default makes aggressive use of interfaces like IEnumerable to hide the actual types being iterated over. This has performance consequences (which is part of why ZLinq can beat it) but it has advantages - for example, the same implementation of Where<T>(seq) can be used for various T's instead of having to JIT or AOT-compile a unique body for every distinct class you iterate over.
From looking at ZLinq it seems like it would potentially have an explosion of unique generic struct types as your queries get more complex, since for it to work you potentially end up with types vaguely resembling Query3<Query2<Query1<T>>>>. But it might not actually be that bad in practice.
Why is LINQ allocating memory in the first place?
Internal iterators, expression trees, etc. Many LINQ variants (depends on the data source) also do not execute the chain step by step but building first an expression tree and then translating that into native query syntax (e.g. SQL)
LINQ methods build up a chain of IEnumerable/IQueryable objects, which then build up chains of IEnumerator objects each time you iterate.
These types are all .NET interfaces, which are reference types, so they're allocated on the heap. .NET's escape analysis can sometimes move reference types to the stack, but this feature is currently very limited and didn't even exist until .NET 9.
ZLinq uses generic structs to prevent these allocations at the expense of some really verbose intermediate types.
Thanks.
[dead]
[dead]
[flagged]