Random LINQ | Tech Off

SecretSoftware
Code to live, but Live to code.

How to select random records from a query?

So, Something like TakeRandom (10), or Select Random new {}

Is there an elegant way of doing something like this using LINQ (C#).

SecretSoftware
Code to live, but Live to code.

Here is an impelemntation of an extension to LINQ that uses random sampling. But Maybe the LINQ team can give us this with in the .NET framework Linq classes.

Code Proj

littleguru
<3 Seattle

SecretSoftware wrote:
Here is an impelemntation of an extension to LINQ that uses random sampling. But Maybe the LINQ team can give us this with in the .NET framework Linq classes.

Code Proj

It looks very similar to what Microsoft has implemented with their own iterators for linq.

SecretSoftware
Code to live, but Live to code.

littleguru wrote:

SecretSoftware wrote:Here is an impelemntation of an extension to LINQ that uses random sampling. But Maybe the LINQ team can give us this with in the .NET framework Linq classes.

Code Proj

It looks very similar to what Microsoft has implemented with their own iterators for linq.

how so? Show example? Do they have random sampling built in?

SecretSoftware
Code to live, but Live to code.

odujosh wrote:

Its already baked in. Make your own extension method.

public T NextRandom<T>(IEnumerable<T> this source)

{

Random gen = new Random((int)DateTime.UtcNow.Ticks);

return source.Skip(gen.Next(0, source.Count() -1) -1).Take(1);

}
My question is do you really need a standard query operator for something you can write a three line extension method for:)

Not sure I would condone random sorting from the outset lazy loading could save perf time. If randomizing the whole list may be your end goal then you should keep an IEnumerable of already provided values around and use a Where clause to check that whatever you return has not been recieved on a previous call.

but that code does not account for the amount of rows that gets returned. You might get index out of range exception here.

I see your point. I can also do it in the sprocs in SQL, but I just wanted it to be built in, more syntactic sugar, and more elegant solutions.

Select Random, or TakeRandom (10) .etc.., Off course we can do with LINQ al together and just use what we used to. so its all about comforts.

odujosh
Need Microsoft SUX now!

Its already baked in. Make your own extension method.

public T NextRandom<T>(IEnumerable<T> this source)

{

Random gen = new Random((int)DateTime.UtcNow.Ticks);

return source.Skip(gen.Next(0, source.Count() -1) -1).Take(1);

}

My question is do you really need a standard query operator for something you can write a three line extension method for:)

Not sure I would condone random sorting from the outset lazy loading could save perf time. If randomizing the whole list may be your possible end goal then you should keep an IEnumerable (any collection) of already provided values around and use a Where clause to check that whatever you return has not been recieved on a previous call.

public IEnumerator<T> Radomized<T>(IEnumerable<T> this source)

{

List<T> Remaining = new List<T>(source);

while (Remaining.Count >= 1)

{

T temp = NextRandom<T>(Remaining);

Remaining.Remove(temp);

yield return temp;

}

}

odujosh
Need Microsoft SUX now!

SecretSoftware wrote:

odujosh wrote:
Its already baked in. Make your own extension method.

public T NextRandom<T>(IEnumerable<T> this source)

{

Random gen = new Random((int)DateTime.UtcNow.Ticks);

return source.Skip(gen.Next(0, source.Count() -1) -1).Take(1);

}
My question is do you really need a standard query operator for something you can write a three line extension method for:)

Not sure I would condone random sorting from the outset lazy loading could save perf time. If randomizing the whole list may be your end goal then you should keep an IEnumerable of already provided values around and use a Where clause to check that whatever you return has not been recieved on a previous call.

but that code does not account for the amount of rows that gets returned. You might get index out of range exception here.

I see your point. I can also do it in the sprocs in SQL, but I just wanted it to be built in, more syntactic sugar, and more elegant solutions.

Select Random, or TakeRandom (10) .etc.., Off course we can do with LINQ al together and just use what we used to. so its all about comforts.

I set Min and Max on the Random number Generator so the value will always always be in range.

Why would you need to TakeRandom(10) when you could Take off of the enumerator extension method I wrote above? Why would you want to composite the two concepts together. And also if you not testing for the size being greater than 10 before taking isn't that bad programming ? Smiley

I think Take(int count) accounts for running into the end of an IEnumerator because how a Enumerator is built it will never let you run off the end.

SecretSoftware
Code to live, but Live to code.

odujosh wrote:

SecretSoftware wrote:

odujosh wrote:
Its already baked in. Make your own extension method.

public T NextRandom<T>(IEnumerable<T> this source)

{

Random gen = new Random((int)DateTime.UtcNow.Ticks);

return source.Skip(gen.Next(0, source.Count() -1) -1).Take(1);

}
My question is do you really need a standard query operator for something you can write a three line extension method for:)

Not sure I would condone random sorting from the outset lazy loading could save perf time. If randomizing the whole list may be your end goal then you should keep an IEnumerable of already provided values around and use a Where clause to check that whatever you return has not been recieved on a previous call.

but that code does not account for the amount of rows that gets returned. You might get index out of range exception here.

I see your point. I can also do it in the sprocs in SQL, but I just wanted it to be built in, more syntactic sugar, and more elegant solutions.

Select Random, or TakeRandom (10) .etc.., Off course we can do with LINQ al together and just use what we used to. so its all about comforts.

I set Min and Max on the Random number Generator so the value will always always be in range.

Why would you need to TakeRandom(10) when you could Take off of the enumerator extension method I wrote above? Why would you want to composite the two concepts together. And also if you not testing for the size being greater than 10 before taking isn't that bad programming ?

All of your points are valid. I just thought it would make a more elegant solution and less work, if you let SQL do the randomization.

Select Random, would generate an expression tree, that would execute at SQL Server, and sql server would get me the top 10 random results.

But we can do it other ways too as you suggested.

odujosh
Need Microsoft SUX now!

And the random query Smiley

var Q = from something in mycollection.Randomized()

select something;

odujosh
Need Microsoft SUX now!

Its a wonder what syntax errors creep in when you don't have a compiler handy. The below code has all been tested! Smiley Here is the corrections and a sample project:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using LinqRandom;

namespace Program

{

class Program

{

static void Main(string[] args)

{

var x = new[] { 23, 303, -2332, 338, 8000, 100123, 254, 2234, 980, -90 };

Console.WriteLine("Count of Items:{0}", x.Count());

int count = 0;

foreach (var y in x.Radomized())

{

Console.WriteLine("Item #{0}:{1}", count++, y);

}

//End Sample project begin extension methods class

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

namespace LinqRandom

{

public static class Class1

{

public static T NextRandom<T>(this IEnumerable<T> source)

{

Random gen = new Random((int)DateTime.Now.Ticks);

return source.Skip(gen.Next(0, source.Count() - 1) - 1).Take(1).ToArray()[0];

}

public static IEnumerable<T> Radomized<T>(this IEnumerable<T> source)

{

List<T> Remaining = new List<T>(source);

while (Remaining.Count >= 1)

{

T temp = NextRandom<T>(Remaining);

Remaining.Remove(temp);

yield return temp;

}

odujosh
Need Microsoft SUX now!

Sent feedback:

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=277155

SecretSoftware
Code to live, but Live to code.

Great!

odujosh
Need Microsoft SUX now!

Bad news:

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=277155

Given the status of resolved by design. Either way this is how you get your voiced heard. I recommend people provide feedback this way and on their blogs. This way you are sure to be heard. I will follow up.. Wink

SecretSoftware
Code to live, but Live to code.

odujosh wrote:

Bad news:

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=277155

Given the status of resolved by design. Either way this is how you get your voiced heard. I recommend people provide feedback this way and on their blogs. This way you are sure to be heard. I will follow up..

I think Randomization built into LINQ is important for a wide array of things. They should implement it. We want to be able to query an engine and let it return out random rows.

odujosh
Need Microsoft SUX now!

Write a comment on my issue. Give specific examples.

littleguru
<3 Seattle

SecretSoftware wrote:

littleguru wrote:

SecretSoftware wrote: Here is an impelemntation of an extension to LINQ that uses random sampling. But Maybe the LINQ team can give us this with in the .NET framework Linq classes.

Code Proj

It looks very similar to what Microsoft has implemented with their own iterators for linq.

how so? Show example? Do they have random sampling built in?

It's not built in... But have a look with reflector at the Enumerable class. You will see how the linq magic happens for IEnumerables. What Code Proj did, looks really like how Microsoft has implemented the other stuff in Enumerable.

odujosh
Need Microsoft SUX now!

I saw that after my first post the code project. It is my thinking that SS and me are not the first ones to submit this feedback. Thats probably why they were not overly exploratory with their response.

Its a rather basic requirement. I do question the specificity of the requirement. But could see the undue burden it could place on simpler domains where the mechanic may not make sense. Like XPATH does not implement it in its default implementation. You have to use a third party namespace and processor. TSQL, Oracle, MySql, PLSQL it is a first class citizen because it makes a lot of sense because of the perf issue being addressed by the platform. With XML you are limited by the size of the file sometimes.

Fabrice

Here is a solution.

stevo_
Human after all

Personally I don't think random set of results is a core feature of a queryable system.. its something that would, can, and has ^ be / been implemented on an abstraction.