LINQ - Dynamically Leverage IList<T> Functionality With VirtualList<T>

There are a lot of extension methods in the LINQ space that allow you to manipulate lists quickly and efficiently, but we may not always want to use an actual list to leverage the power of this functionality.

confused - well, consider this code:

Enumerable.Range(0, int.MaxValue).Aggregate(........)

the line above is to run an aggregation function a very large number of times.
This could be a candidate for task parallelism via the TPL, but doing the following actually has no effect

Enumerable.Range(0, int.MaxValue).AsParallel().Aggregate(.........)

This is because you cannot partition collections via an enumerator, and the 'AsParallel' method will not do anything. You need a collection that implements IList. So how about this you say

Enumerable.Range(0, int.MaxValue).ToList().AsParallel().Aggregate(.........)

Well, you'll get an 'OutOfMemoryException' because you are trying to create an enormous list, which should not be necessary as you could easily calculate the values for each item in the list.

The solution is my new VirtualList<T> class (implementation below), which allows the following to be parallelised

var result = new VirtualList<double>(int.MaxValue, i => i).AsParallel().Aggregate(........)

The underlying concept is that for a give indexer value, you run a function to retrieve your value. There is no actual list, but a class that implements the necessary interfaces and masquerades as a list.

Still not sure ? Well with list we can easily parallelise a computation that calculates the value of PI

var pi = new VirtualList<double>(int.MaxValue, i => Math.Pow(-1d, i) / (2 * i + 1) * 4).AsParallel().Aggregate(() => 0d, (tot, next) => tot + next, (maint,localt) => maint + localt, final => final);

There you go, parallelising a computation using Linq extension methods without actually having the requisite list

And here's the class (its very simple)

public class VirtualList<T> : IList<T>, IList
{
    private readonly int count;
    private readonly Func<int, T> getValueForIndex;

    public VirtualList(int count, Func<int, T> getValueForIndex)
    {
        this.getValueForIndex = getValueForIndex;
        this.count = count;
    }
    IEnumerator<T> IEnumerable<T>.GetEnumerator()
    {
        for (int i = 0; i < count; i++)
            yield return getValueForIndex(i);
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return ((IEnumerable<T>)this).GetEnumerator();
    }

    void ICollection<T>.Add(T item)
    {
        throw new NotSupportedException();
    }

    int IList.Add(object value)
    {
        throw new NotSupportedException();
    }

    bool IList.Contains(object value)
    {
        throw new NotSupportedException();
    }

    void IList.Clear()
    {
        throw new NotSupportedException();
    }

    int IList.IndexOf(object value)
    {
        throw new NotSupportedException();
    }

    void IList.Insert(int index, object value)
    {
        throw new NotSupportedException();
    }

    void IList.Remove(object value)
    {
        throw new NotSupportedException();
    }

    void IList.RemoveAt(int index)
    {
        throw new NotSupportedException();
    }

    object IList.this[int index]
    {
        get { return getValueForIndex(index); }
        set { throw new NotSupportedException(); }
    }

    bool IList.IsReadOnly
    {
        get { return true; }
    }

    bool IList.IsFixedSize
    {
        get { return true; }
    }

    void ICollection<T>.Clear()
    {
        throw new NotSupportedException();
    }

    bool ICollection<T>.Contains(T item)
    {
        throw new NotSupportedException();
    }

    void ICollection<T>.CopyTo(T[] array, int arrayIndex)
    {
        throw new NotSupportedException();
    }

    bool ICollection<T>.Remove(T item)
    {
        throw new NotSupportedException();
    }

    void ICollection.CopyTo(Array array, int index)
    {
        throw new NotSupportedException();
    }

    int ICollection.Count
    {
        get { return count; }
    }

    object ICollection.SyncRoot
    {
        get { return this; }
    }

    bool ICollection.IsSynchronized
    {
        get { return false; }
    }

    int ICollection<T>.Count
    {
        get { return count; }
    }

    bool ICollection<T>.IsReadOnly
    {
        get { return true; }
    }

    int IList<T>.IndexOf(T item)
    {
        throw new NotSupportedException();
    }

    void IList<T>.Insert(int index, T item)
    {
        throw new NotSupportedException();
    }

    void IList<T>.RemoveAt(int index)
    {
        throw new NotSupportedException();
    }

    T IList<T>.this[int index]
    {
        get { return getValueForIndex(index); }
        set { throw new NotSupportedException(); }
    }
}

Of course, not all functionality relating to lists is supported, but its a neat trick if you want to parallelise large iterative computations

Dean