Tech Off Thread

57 posts

Forum Read Only

This forum has been made read only by the site admins. No new threads or comments can be added.

How about some syntax sugar for IEnumerable<>?

Back to Forum: Tech Off
  • User profile image
    wkempf

    wkempf said:
    evildictaitor said:
    *snip*
    I always wonder about posts like this.  It's obvious from the post your replying to that I fully understand why IEnumerable<T> doesn't have a Count property or provide random access.  That kinda was the entire point of my post, don't ya think?
    Oh, and after reading the entire post, you've missed several things here in any event.

    The only one who talked about casting the IEnumerable<T> was myself.  Go read what I said carefully.  I said that LINQ was doing the cast as an optimization.  LINQ casts to an ICollection<T>, not to a List<T>.  This cast is safe and not dodgy.  What's dodgy is relying on the computational complexity of an IEnumerable<T> exposed by someone like the original poster, when calling such LINQ methods as Count() and ElementAt().  When using LINQ you must expect the worst case scenario, because you're dealing with an IEnumerable<T>.  It's just a happy bonus when it performs better.  That's why you shouldn't return an IEnumerable<T> just because you want a "read only" collection.

    No one was suggesting you should wrap the collection in a ReadOnlyCollection that was returned to the user as an IEnumerable<T>.  Where you got that nonsense is beyond me.

  • User profile image
    evildictait​or

    wkempf said:
    wkempf said:
    *snip*
    Oh, and after reading the entire post, you've missed several things here in any event.

    The only one who talked about casting the IEnumerable<T> was myself.  Go read what I said carefully.  I said that LINQ was doing the cast as an optimization.  LINQ casts to an ICollection<T>, not to a List<T>.  This cast is safe and not dodgy.  What's dodgy is relying on the computational complexity of an IEnumerable<T> exposed by someone like the original poster, when calling such LINQ methods as Count() and ElementAt().  When using LINQ you must expect the worst case scenario, because you're dealing with an IEnumerable<T>.  It's just a happy bonus when it performs better.  That's why you shouldn't return an IEnumerable<T> just because you want a "read only" collection.

    No one was suggesting you should wrap the collection in a ReadOnlyCollection that was returned to the user as an IEnumerable<T>.  Where you got that nonsense is beyond me.
    You're reading too much into the "reply to" box from C9. I was just commenting on the whole thread, not just replying to you.

    My point about why Count isn't there was a response to TommyCarlier who explictly lamented the fact that it's not there by default on an IEnumerable<T>.

    The optimisation you mention of casting to an ICollection<T> is fine, because it first checks that the cast is valid, and ensures that if the cast is valid that it doesn't mutate the collection - my point was more general and related to CannotResolveSymbol's post who noted that exposing IEnumerable<T> isn't a guarrantee that the inner property won't be mutated by the caller.

    You say that noone was suggesting you wrap the collection in a ReadOnlyCollection, but W3bbo's code example does exactly that, so your point is invalid.

    Please don't take the C9 "reply to" as nessisarrilly a refutation of what you said. I just clicked on the reply closest to the bottom of the screen, and I meant nothing by it.


    Edit: I note actually that the order complexity of ReadOnlyCollection isn't O(_val.Count), but actually a minor overhead, since the ReadOnlyCollection defers all of the accesses of the list to the contained list and throws an exception on all mutation attempts, so it's not quite as bad as I made it sound.

  • User profile image
    wkempf

    evildictaitor said:
    wkempf said:
    *snip*
    You're reading too much into the "reply to" box from C9. I was just commenting on the whole thread, not just replying to you.

    My point about why Count isn't there was a response to TommyCarlier who explictly lamented the fact that it's not there by default on an IEnumerable<T>.

    The optimisation you mention of casting to an ICollection<T> is fine, because it first checks that the cast is valid, and ensures that if the cast is valid that it doesn't mutate the collection - my point was more general and related to CannotResolveSymbol's post who noted that exposing IEnumerable<T> isn't a guarrantee that the inner property won't be mutated by the caller.

    You say that noone was suggesting you wrap the collection in a ReadOnlyCollection, but W3bbo's code example does exactly that, so your point is invalid.

    Please don't take the C9 "reply to" as nessisarrilly a refutation of what you said. I just clicked on the reply closest to the bottom of the screen, and I meant nothing by it.


    Edit: I note actually that the order complexity of ReadOnlyCollection isn't O(_val.Count), but actually a minor overhead, since the ReadOnlyCollection defers all of the accesses of the list to the contained list and throws an exception on all mutation attempts, so it's not quite as bad as I made it sound.
    I wouldn't have made the connection to TommyCarlier's post, because he only talks about Count, while you talk about both Count and random access.  It's also interesting to note, that thanks to LINQ, IEnumerable<T> now has both of the features you say were deliberately left out.  Yes, I understand the difference between the two designs, but you're glossing over the reasoning here.

    W3bbo's code example didn't do what you did.  He didn't wrap the collection in a ROC just to expose it to the user as an IEnumerable<T>.  What he did was the most appropriate way to expose a collection that's supposed to be read-only.  He wrapped it in a ROC and exposed a ROC.  There are reasons to expose an IEnumerable<T>, as you pointed out with several examples like the file enumerator.  However, exposing a read-only collection isn't a reason for making it an IEnumerable<T>.

  • User profile image
    dpratt71

    wkempf said:
    TommyCarlier said:
    *snip*
    There's lots you lose with IEnumerable<T>.  Count and random access are the most obvious.  LINQ sort of gives this back to you.  The Count() method will give you the count, and is optimized to try and cast the IEnumerable<T> to an ICollection<T> in order to get the count without having to iterate over the collection.  However, I believe that is bad design.  If the underlying collection is changed to no longer be an ICollection<T> the complexity of Count() has radically changed.  Since the public interface only exposed IEnumerable<T>, I have to expect that sort of behavior.

    The ReadOnlyCollection types aren't ideal.  I'd much prefer it if the collection designs had started with IReadOnlyCollection and ICollection had derived from there.  IOW, I share your consternation about the existence of non-readonly methods that throw exceptions.  That said, however, we have what we have, and in practice there's seldom if ever an issue with the IsReadOnly/exception design of ReadOnlyCollection types.
    Interesting that you should bring up the concept of an IReadOnlyCollection from which ICollection would derive. This idea was really the starting point of the thought process of my original post in this thread. Basically, I asked myself what it was that I wanted IReadOnlyCollection to give me that I didn't get from IEnumerable+LINQ.

    Your point about "complexity" is not without merit and is something that I had considered. On the other hand, unless you are referencing a sealed class, I don't think there are ever any guarantees about such things. To put it another way, I think it is the responsibility of the language to help us validate the correctness of our program, but it is entirely our responsibility to determine the fitness of tool "X" for task "Y". I haven't met a language yet that doesn't make it trivially easy to do something really stupid Wink

    Hey, maybe we can get the Spec# folks to incorporate algorithmic complexity as a contract Wink

  • User profile image
    wkempf

    dpratt71 said:
    wkempf said:
    *snip*
    Interesting that you should bring up the concept of an IReadOnlyCollection from which ICollection would derive. This idea was really the starting point of the thought process of my original post in this thread. Basically, I asked myself what it was that I wanted IReadOnlyCollection to give me that I didn't get from IEnumerable+LINQ.

    Your point about "complexity" is not without merit and is something that I had considered. On the other hand, unless you are referencing a sealed class, I don't think there are ever any guarantees about such things. To put it another way, I think it is the responsibility of the language to help us validate the correctness of our program, but it is entirely our responsibility to determine the fitness of tool "X" for task "Y". I haven't met a language yet that doesn't make it trivially easy to do something really stupid Wink

    Hey, maybe we can get the Spec# folks to incorporate algorithmic complexity as a contract Wink
    There are complexity guarantees documented on the various interfaces here.  Exposing an IEnumerable<T> means that a large number of operations consumers are going to want to perform are going to have O(N) complexity.  LINQ may "cheat" and give you better performance, but the complexity that consumers MUST assume is O(N).  Now, expose a ReadOnlyCollection<T>, and most of those same operations are now O(1).  A hypothetical IReadOnlyCollection would provide the same complexity guarantees you have with ReadOnlyCollection<T>, but without the existense of mutating methods, making for a "cleaner" design.  Again, though, in practice the current design is "good enough", and certainly preferable to IEnumerable<T>.

  • User profile image
    TommyCarlier

    evildictaitor said:
    wkempf said:
    *snip*
    You're reading too much into the "reply to" box from C9. I was just commenting on the whole thread, not just replying to you.

    My point about why Count isn't there was a response to TommyCarlier who explictly lamented the fact that it's not there by default on an IEnumerable<T>.

    The optimisation you mention of casting to an ICollection<T> is fine, because it first checks that the cast is valid, and ensures that if the cast is valid that it doesn't mutate the collection - my point was more general and related to CannotResolveSymbol's post who noted that exposing IEnumerable<T> isn't a guarrantee that the inner property won't be mutated by the caller.

    You say that noone was suggesting you wrap the collection in a ReadOnlyCollection, but W3bbo's code example does exactly that, so your point is invalid.

    Please don't take the C9 "reply to" as nessisarrilly a refutation of what you said. I just clicked on the reply closest to the bottom of the screen, and I meant nothing by it.


    Edit: I note actually that the order complexity of ReadOnlyCollection isn't O(_val.Count), but actually a minor overhead, since the ReadOnlyCollection defers all of the accesses of the list to the contained list and throws an exception on all mutation attempts, so it's not quite as bad as I made it sound.

    Actually, I didn't say (or mean) that I wanted IEnumerable<T> to have a Count-property: IEnumerable<T> is perfectly fine the way it is. I myself have recently writen a tokenizer that has a method ReadAll which returns an IEnumerable<T> that lazily reads from a TextReader and produces tokens. In this case, a Count-property would mean that all the tokens would have to be read before anything can be done.

    What I want is that there was an interface IReadOnlyCollection<T> or ICountEnumerable<T> the my classes can expose in cases where my classes know the number of elements. I never indicated index-based random access. In fact, read-only index-based random access could also be exposed as an interface IRandomAccessEnumerable<T>. Adding these 2 additional interfaces would not break existing code (because the existing code would not know about them). The interface hierarchy would then look like this:

    public
     interface IEnumerable<T> : IEnumerable { IEnumerator<T> GetEnumerator(); } public interface ICountEnumerable<T> : IEnumerable<T> { int Count { get; } } public interface IRandomAccessEnumerable<T> : ICountEnumerable<T> { T this[int index] { get; } } public
     interface ICollection<T> : ICountEnumerable<T> { void Add(T item); void Clear(); bool Contains(T item); void CopyTo(T[] array, int arrayIndex); bool Remove(T item); bool IsReadOnly { get; } } public interface IList<T> : ICollection<T>, IRandomAccessEnumerable<T>
     { int IndexOf(T item); void Insert(int index, T item); void RemoveAt(int index); T this[int index] { get; set; } }

  • User profile image
    TommyCarlier

    TommyCarlier said:
    evildictaitor said:
    *snip*

    Actually, I didn't say (or mean) that I wanted IEnumerable<T> to have a Count-property: IEnumerable<T> is perfectly fine the way it is. I myself have recently writen a tokenizer that has a method ReadAll which returns an IEnumerable<T> that lazily reads from a TextReader and produces tokens. In this case, a Count-property would mean that all the tokens would have to be read before anything can be done.

    What I want is that there was an interface IReadOnlyCollection<T> or ICountEnumerable<T> the my classes can expose in cases where my classes know the number of elements. I never indicated index-based random access. In fact, read-only index-based random access could also be exposed as an interface IRandomAccessEnumerable<T>. Adding these 2 additional interfaces would not break existing code (because the existing code would not know about them). The interface hierarchy would then look like this:

    public
     interface IEnumerable<T> : IEnumerable { IEnumerator<T> GetEnumerator(); } public interface ICountEnumerable<T> : IEnumerable<T> { int Count { get; } } public interface IRandomAccessEnumerable<T> : ICountEnumerable<T> { T this[int index] { get; } } public
     interface ICollection<T> : ICountEnumerable<T> { void Add(T item); void Clear(); bool Contains(T item); void CopyTo(T[] array, int arrayIndex); bool Remove(T item); bool IsReadOnly { get; } } public interface IList<T> : ICollection<T>, IRandomAccessEnumerable<T>
     { int IndexOf(T item); void Insert(int index, T item); void RemoveAt(int index); T this[int index] { get; set; } }

    Actually, now that I think about it further: it could be a good idea to move all the members of ICollection<T> and IList<T> that can be performed on read-only collections to interfaces IReadOnlyCollection<T> and IReadOnlyList<T>. It would allow for richer API's without additional overhead. The current collection classes would not have to be changed. The only thing that changes is the addition of these interfaces and the modification of ICollection<T> and IList<T> (so they implement IReadOnlyCollection<T> and IReadOnlyList<T>).

    The interface hierarchy would look like this:
     public interface IEnumerable<T> : IEnumerable {   IEnumerator<T> GetEnumerator(); } public interface IReadOnlyCollection<T> : IEnumerable<T> {   int Count { get; }   bool Contains(T item);
       void CopyTo(T[] array, int arrayIndex); } public interface IReadOnlyList<T> : IReadOnlyCollection<T> {   T this[int index] { get; }   int IndexOf(T item); } public interface ICollection<T> : IReadOnlyCollection<T> {   void Add(T item);   void Clear();   bool
     Remove(T item);   bool IsReadOnly { get; } } public interface IList<T> : ICollection<T>, IReadOnlyList<T> {   void Insert(int index, T item);   void RemoveAt(int index);   T this[int index] { get; set; } }

  • User profile image
    dpratt71

    wkempf said:
    dpratt71 said:
    *snip*
    There are complexity guarantees documented on the various interfaces here.  Exposing an IEnumerable<T> means that a large number of operations consumers are going to want to perform are going to have O(N) complexity.  LINQ may "cheat" and give you better performance, but the complexity that consumers MUST assume is O(N).  Now, expose a ReadOnlyCollection<T>, and most of those same operations are now O(1).  A hypothetical IReadOnlyCollection would provide the same complexity guarantees you have with ReadOnlyCollection<T>, but without the existense of mutating methods, making for a "cleaner" design.  Again, though, in practice the current design is "good enough", and certainly preferable to IEnumerable<T>.

    Up 'til now, you were making some sense Wink

    "There are complexity guarantees documented on the various interfaces here. "

    Where is this documentation of which you speak and how does it even make sense for an interface to guarantee something that is wholly dependent on the implementation.

    "Exposing an IEnumerable<T> means that a large number of operations consumers are going to want to perform are going to have O(N) complexity.  LINQ may "cheat" and give you better performance, but the complexity that consumers MUST assume is O(N)."

    I think this is a bogus argument. To make an analogy, if you came from the world of functional programming, it might shock you to learn that something as innocuous as this:

        int DoStuff(int value);

    may very well "launch the missiles", so to speak. This does not mean that the imperative programmer MUST assume that he or she may very well start WWIII with the next method call.

    "Now, expose a ReadOnlyCollection<T>, and most of those same operations are now O(1)."

    As has been pointed out at least a couple times on this thread, ReadOnlyCollection delegates such operations to the wrapped IList. Given that I can wrap *anything* that implements IList, how is ReadOnlyCollection going to guarantee the complexity?

  • User profile image
    TommyCarlier

    TommyCarlier said:
    TommyCarlier said:
    *snip*

    Actually, now that I think about it further: it could be a good idea to move all the members of ICollection<T> and IList<T> that can be performed on read-only collections to interfaces IReadOnlyCollection<T> and IReadOnlyList<T>. It would allow for richer API's without additional overhead. The current collection classes would not have to be changed. The only thing that changes is the addition of these interfaces and the modification of ICollection<T> and IList<T> (so they implement IReadOnlyCollection<T> and IReadOnlyList<T>).

    The interface hierarchy would look like this:
     public interface IEnumerable<T> : IEnumerable {   IEnumerator<T> GetEnumerator(); } public interface IReadOnlyCollection<T> : IEnumerable<T> {   int Count { get; }   bool Contains(T item);
       void CopyTo(T[] array, int arrayIndex); } public interface IReadOnlyList<T> : IReadOnlyCollection<T> {   T this[int index] { get; }   int IndexOf(T item); } public interface ICollection<T> : IReadOnlyCollection<T> {   void Add(T item);   void Clear();   bool
     Remove(T item);   bool IsReadOnly { get; } } public interface IList<T> : ICollection<T>, IReadOnlyList<T> {   void Insert(int index, T item);   void RemoveAt(int index);   T this[int index] { get; set; } }
    I've extended the idea to read-only dictionaries (IReadOnlyDictionary<TKey, TValue>) and just wrote a blog post about it: .NET BCL proposition: read-only collection interfaces. Because I think this idea is solid and would be a good addition to the BCL, I sent an e-mail to the .NET BCL Team. Let's see if they respond.

  • User profile image
    dpratt71

    TommyCarlier said:
    TommyCarlier said:
    *snip*

    Actually, now that I think about it further: it could be a good idea to move all the members of ICollection<T> and IList<T> that can be performed on read-only collections to interfaces IReadOnlyCollection<T> and IReadOnlyList<T>. It would allow for richer API's without additional overhead. The current collection classes would not have to be changed. The only thing that changes is the addition of these interfaces and the modification of ICollection<T> and IList<T> (so they implement IReadOnlyCollection<T> and IReadOnlyList<T>).

    The interface hierarchy would look like this:
     public interface IEnumerable<T> : IEnumerable {   IEnumerator<T> GetEnumerator(); } public interface IReadOnlyCollection<T> : IEnumerable<T> {   int Count { get; }   bool Contains(T item);
       void CopyTo(T[] array, int arrayIndex); } public interface IReadOnlyList<T> : IReadOnlyCollection<T> {   T this[int index] { get; }   int IndexOf(T item); } public interface ICollection<T> : IReadOnlyCollection<T> {   void Add(T item);   void Clear();   bool
     Remove(T item);   bool IsReadOnly { get; } } public interface IList<T> : ICollection<T>, IReadOnlyList<T> {   void Insert(int index, T item);   void RemoveAt(int index);   T this[int index] { get; set; } }

    (Edit: I was tired and I screwed up the example a bit. All fixed now)

    Given the work they are doing around covariance and contravariance in 4.0, I think this idea makes a lot of sense. It makes so much sense that I think the only reason they might not do it is for reasons of backwards compatibility (I have no idea if that would be an issue or not). To take this to an insane level, what about splitting the interfaces between IReadOnlyBlah and IMutableBlah and an IBlah that derives and unifies both? Why IMutableBlah? Suppose:

    class Foo : Bar
    {
        ...
    }

    void PopulateFoos(IMutableCollection<Foo> foos)
    {
        ...
    }

    with contravariance I can do this:

    var bars = new List<Bar>();
    ...
    PopulateFoos(bars); // Look, Ma! No casting or converting!

  • User profile image
    TommyCarlier

    dpratt71 said:
    TommyCarlier said:
    *snip*

    (Edit: I was tired and I screwed up the example a bit. All fixed now)

    Given the work they are doing around covariance and contravariance in 4.0, I think this idea makes a lot of sense. It makes so much sense that I think the only reason they might not do it is for reasons of backwards compatibility (I have no idea if that would be an issue or not). To take this to an insane level, what about splitting the interfaces between IReadOnlyBlah and IMutableBlah and an IBlah that derives and unifies both? Why IMutableBlah? Suppose:

    class Foo : Bar
    {
        ...
    }

    void PopulateFoos(IMutableCollection<Foo> foos)
    {
        ...
    }

    with contravariance I can do this:

    var bars = new List<Bar>();
    ...
    PopulateFoos(bars); // Look, Ma! No casting or converting!

    I never expected this, but I received a reply very quickly from Kim Hamilton of the BCL team. Here's what Kim wrote:

    Hi Tommy,
    Thanks for the excellent feedback! We're considering similar collections interface refactorings (such as you described) -- not only to allow more appropriate read-only interfaces, but also to expand support for covariance and contravariance. Note that IEnumerable<T> and IEnumerable<T> are the only collections interfaces that work with CLI variance support currently. That includes a restriction on T being exclusively in the "in" or "out" position (see Eric Lippert's blogs on variance support on C# for more details). The interface hierarchy you described _almost_ allows a covariant IReadOnlyCollection<T> but read-only properties like Contains are a problem (T in the input position). I'd like to solve both problems at the same time. There are possibilities but they're slightly less elegant than the nice hierarchy to solve just the read-only problem.

    In any case, we're definitely interested in such an API enhancement and I'm glad to hear that you'd find this useful.
    Thanks!
    Kim

  • User profile image
    longnightmo​on

    And all this time I thought IEnumerable<T> was syntax candy.

  • User profile image
    Sven Groot

    longnightmoon said:
    And all this time I thought IEnumerable<T> was syntax candy.
    Well, everything above 1s and 0s is syntactical sugar, if you want to get technical. Tongue Out

  • User profile image
    stevo_

    Tell me about it, I've wanted this for ages - but I'd also like some immutable structures as well, perhaps the f# ones will be available in .net 4, even if I have to reference an f# assembly thats gac'd.

  • User profile image
    Yggdrasil

    To get back to the original topic - I'm all for the proposed syntax change. When you start using LINQ a lot, passing IEnumerable<T>s back and forth, you start hating how long it is - not to write, since we have autocomplete intellisense for that, but to read. It fills up the code with useless letters. 


    I second the * syntax, though that would cause problems with unsafe code. A coworker of mine wants to just tack the plural 's' onto it, so an IEnumerable<string> would just be strings. Of course, that would mean that an IEnumerable<Bus>, for instance, would have to be correctly parsed by the compiler from Buses. And that's before we even make the compiler understand the shorthand for IEnumerable<Mouse>...

  • User profile image
    evildictait​or

    stevo_ said:
    Tell me about it, I've wanted this for ages - but I'd also like some immutable structures as well, perhaps the f# ones will be available in .net 4, even if I have to reference an f# assembly thats gac'd.
    IEnumerable<T> has no direct methods for mutating a collection, so unless you use reflection or do an upwards cast then an IEnumerable<T> is immutable.

    If you want to be really sure, then do this:

    class {
      List<foo> myMutableCollection = new List<foo>();

      public IEnumerable<foo> EnumerableFoo {
        get {
           return new ReadOnlyCollection(myMutableCollection);
        }
      } 
    }

    and now the mutablecollection can only be altered by the class or by reflection (and there's not much you can do to prevent reflection doing bad things)

  • User profile image
    evildictait​or

    Yggdrasil said:
    To get back to the original topic - I'm all for the proposed syntax change. When you start using LINQ a lot, passing IEnumerable<T>s back and forth, you start hating how long it is - not to write, since we have autocomplete intellisense for that, but to read. It fills up the code with useless letters. 

    I second the * syntax, though that would cause problems with unsafe code. A coworker of mine wants to just tack the plural 's' onto it, so an IEnumerable<string> would just be strings. Of course, that would mean that an IEnumerable<Bus>, for instance, would have to be correctly parsed by the compiler from Buses. And that's before we even make the compiler understand the shorthand for IEnumerable<Mouse>...

    I would prefer the IDE to auto-expand out my "var" keywords to what they really are when I give it a shortcut key - this would allow you to just write

    var myFoo = select ... <Tab> <Tab> <ShortCutKey>

    IEnumerable<ICollection<Structure<MyFoo<Dictionary<Gets<Quite<Long>, Doesnt> It>>...>> myFoo = select ...

    Actually adding keywords dilutes the language and makes it harder to read.

  • User profile image
    stevo_

    Yes we all know this, but clearly what everyone is talking about here is having immutable interfaces as a step in the middle of IEnumerable and IWhateverMutableDataStructure...

    I'm still trying to understand why thats so hard to grasp.. nobody is wanting to corrupt what IEnumerable is.. we just want interfaces that describe the readonly actions of the common data structures that .NET has.. this is something I'm sure most people here have thought about even years ago.. I was thinking this time last year that I would love an immutable collection / dictionary interface.. and then wondering why they don't exist.. the great thing about the interface over the concrete type say.. ReadOnlyCollection.. is:

    - Allows complete variance on implementation
    - Allows you to expose the instance as either readonly or mutable in a cleaner way (vs keeping a reference to the underlying Collection that ReadOnlyCollection wraps)..

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.