A couple of times now I've needed to lazy-enumerate a large collection, but in ‘chunks’ – or to put it in another way, I've needed to bring data back from the collection in bite-sized batches.
There currently isn't a Linq extension to support this (afaik), so I wrote one. It preserves the lazy-enumerating characteristics of most other Linq extensions, and seems to work very well.
public static IEnumerable<IEnumerable<TResult>> SelectChunk<TSource, TResult>(
this IEnumerable<TSource> source, Func<TSource, TResult> selector, int chunkSize)
{
IEnumerator<TSource> enumerator = source.GetEnumerator();
while(true)
{
if (!enumerator.MoveNext())
break;
var resultArray = new TResult[chunkSize];
for (int i = 0; i < chunkSize; i++)
{
resultArray[i] = selector(enumerator.Current);
if (i == chunkSize-1 || !enumerator.MoveNext())
break;
}
yield return resultArray;
}
}
And here is an example of its usage:
IEnumerable<string> data = new[] {"John", "Doe", "Male", "Jane", "Smith", "Female"};
var people =
data.SelectChunk(s => s, 3)
.Select(s => new
{
FirstName = s.ElementAt(0),
LastName = s.ElementAt(1),
Sex = s.ElementAt(2)
});