Package extra166y

Class ParallelArray<T>

  • All Implemented Interfaces:
    java.lang.Iterable<T>

    public class ParallelArray<T>
    extends ParallelArrayWithBounds<T>
    implements java.lang.Iterable<T>
    An array supporting parallel operations.

    A ParallelArray maintains a ForkJoinPool and an array in order to provide parallel aggregate operations. The main operations are to apply some procedure to each element, to map each element to a new element, to replace each element, to select a subset of elements based on matching a predicate or ranges of indices, and to reduce all elements into a single value such as a sum.

    A ParallelArray is constructed by allocating, using, or copying an array, using one of the static factory methods create(int, java.lang.Class<? super T>, jsr166y.ForkJoinPool), createEmpty(int, java.lang.Class<? super T>, jsr166y.ForkJoinPool), createUsingHandoff(T[], jsr166y.ForkJoinPool) and createFromCopy(T[], jsr166y.ForkJoinPool). Upon construction, the encapsulated array managed by the ParallelArray must not be shared between threads without external synchronization. In particular, as is the case with any array, access by another thread of an element of a ParallelArray while another operation is in progress has undefined effects.

    The ForkJoinPool used to construct a ParallelArray can be shared safely by other threads (and used in other ParallelArrays). To avoid the overhead associated with creating multiple executors, it is often a good idea to use the defaultExecutor() across all ParallelArrays. However, you might choose to use different ones for the sake of controlling processor usage, isolating faults, and/or ensuring progress.

    A ParallelArray is not a List. It relies on random access across array elements to support efficient parallel operations. However, a ParallelArray can be viewed and manipulated as a List, via method asList(). The asList view allows incremental insertion and modification of elements while setting up a ParallelArray, generally before using it for parallel operations. Similarly, the list view may be useful when accessing the results of computations in sequential contexts. A ParallelArray may also be created using the elements of any other Collection, by constructing from the array returned by the Collection's toArray method. The effects of mutative asList operations may also be achieved directly using method setLimit(int) along with element-by-element access methods get(int) and set(int, T).

    While ParallelArrays can be based on any kind of an object array, including "boxed" types such as Long, parallel operations on scalar "unboxed" type are likely to be substantially more efficient. For this reason, classes ParallelLongArray and ParallelDoubleArray are also supplied, and designed to smoothly interoperate with ParallelArrays. You should also use a ParallelLongArray for processing other integral scalar data (int, short, etc). And similarly use a ParallelDoubleArray for float data. (Further specializations for these other types would add clutter without significantly improving performance beyond that of the Long and Double versions.)

    Most usages of ParallelArray involve sets of operations prefixed with range bounds, filters, and mappings (including mappings that combine elements from other ParallelArrays), using withBounds, withFilter, and withMapping, respectively. For example, aParallelArray.withFilter(aPredicate).all() creates a new ParallelArray containing only those elements matching the predicate. And for ParallelLongArrays a, b, and c, a.withMapping(CommonOps.longAdder(),b).withMapping(CommonOps.longAdder(),c).min() returns the minimum value of a[i]+b[i]+c[i] for all i. As illustrated below, a mapping often represents accessing some field or invoking some method of an element. These versions are typically more efficient than performing selections, then mappings, then other operations in multiple (parallel) steps. The basic ideas and usages of filtering and mapping are similar to those in database query systems such as SQL, but take a more restrictive form. Series of filter and mapping prefixes may each be cascaded, but all filter prefixes must precede all mapping prefixes, to ensure efficient execution in a single parallel step. In cases of combined mapping expressions, this rule is only dynamically enforced. For example, pa.withMapping(op, pb.withFilter(f)) will compile but throw an exception upon execution because the filter precedes the mapping.

    While series of filters and mappings are allowed, it is usually more efficient to combine them into single filters or mappings when possible. For example pa.withMapping(addOne).withMapping(addOne) is generally less efficient than pa.withMapping(addTwo). Methods withIndexedFilter and withIndexedMapping may be useful when combining such expressions.

    This class includes some reductions, such as min, that are commonly useful for most element types, as well as a combined version, summary, that computes all of them in a single parallel step, which is normally more efficient than computing each in turn.

    The methods in this class are designed to perform efficiently with both large and small pools, even with single-thread pools on uniprocessors. However, there is some overhead in parallelizing operations, so short computations on small arrays might not execute faster than sequential versions, and might even be slower.

    Sample usages. The main difference between programming with plain arrays and programming with aggregates is that you must separately define each of the component functions on elements. For example, the following returns the maximum Grade Point Average across all senior students, given a (fictional) Student class:

     import static Ops.*;
     class StudentStatistics {
       ParallelArray<Student> students = ...
       // ...
       public double getMaxSeniorGpa() {
         return students.withFilter(isSenior).withMapping(gpaField).max();
       }
    
       // helpers:
       static final class IsSenior implements Predicate<Student> {
         public boolean op(Student s) { return s.credits > 90; }
       }
       static final IsSenior isSenior = new IsSenior();
       static final class GpaField implements ObjectToDouble<Student> {
         public double op(Student s) { return s.gpa; }
       }
       static final GpaField gpaField = new GpaField();
     }
     
    • Method Detail

      • defaultExecutor

        public static jsr166y.ForkJoinPool defaultExecutor()
        Returns a common default executor for use in ParallelArrays. This executor arranges enough parallelism to use most, but not necessarily all, of the available processors on this system.
        Returns:
        the executor
      • create

        public static <T> ParallelArray<T> create​(int size,
                                                  java.lang.Class<? super T> elementType,
                                                  jsr166y.ForkJoinPool executor)
        Creates a new ParallelArray using the given executor and an array of the given size constructed using the indicated base element type.
        Parameters:
        size - the array size
        elementType - the type of the elements
        executor - the executor
      • createUsingHandoff

        public static <T> ParallelArray<T> createUsingHandoff​(T[] handoff,
                                                              jsr166y.ForkJoinPool executor)
        Creates a new ParallelArray initially using the given array and executor. In general, the handed off array should not be used for other purposes once constructing this ParallelArray. The given array may be internally replaced by another array in the course of methods that add or remove elements.
        Parameters:
        handoff - the array
        executor - the executor
      • createFromCopy

        public static <T> ParallelArray<T> createFromCopy​(T[] source,
                                                          jsr166y.ForkJoinPool executor)
        Creates a new ParallelArray using the given executor and initially holding copies of the given source elements.
        Parameters:
        source - the source of initial elements
        executor - the executor
      • createFromCopy

        public static <T> ParallelArray<T> createFromCopy​(int size,
                                                          T[] source,
                                                          jsr166y.ForkJoinPool executor)
        Creates a new ParallelArray using an array of the given size, initially holding copies of the given source truncated or padded with nulls to obtain the specified length.
        Parameters:
        source - the source of initial elements
        size - the array size
        executor - the executor
      • createEmpty

        public static <T> ParallelArray<T> createEmpty​(int size,
                                                       java.lang.Class<? super T> elementType,
                                                       jsr166y.ForkJoinPool executor)
        Creates a new ParallelArray using the given executor and an array of the given size constructed using the indicated base element type, but with an initial effective size of zero, enabling incremental insertion via asList() operations.
        Parameters:
        size - the array size
        elementType - the type of the elements
        executor - the executor
      • getExecutor

        public jsr166y.ForkJoinPool getExecutor()
        Returns the executor used for computations
        Returns:
        the executor
      • reduce

        public T reduce​(Ops.Reducer<T> reducer,
                        T base)
        Returns reduction of elements
        Overrides:
        reduce in class ParallelArrayWithMapping<T,​T>
        Parameters:
        reducer - the reducer
        base - the result for an empty array
        Returns:
        reduction
      • all

        public ParallelArray<T> all​(java.lang.Class<? super T> elementType)
        Returns a new ParallelArray with the given element type holding all elements
        Overrides:
        all in class ParallelArrayWithMapping<T,​T>
        Parameters:
        elementType - the type of the elements
        Returns:
        a new ParallelArray holding all elements
      • replaceWithMapping

        public ParallelArray<T> replaceWithMapping​(Ops.Op<? super T,​? extends T> op)
        Replaces elements with the results of applying the given transform to their current values.
        Overrides:
        replaceWithMapping in class ParallelArrayWithFilter<T>
        Parameters:
        op - the op
        Returns:
        this (to simplify use in expressions)
      • replaceWithMapping

        public ParallelArray<T> replaceWithMapping​(Ops.BinaryOp<T,​T,​T> combiner,
                                                   T[] other)
        Replaces elements with results of applying op(thisElement, otherElement)
        Overrides:
        replaceWithMapping in class ParallelArrayWithFilter<T>
        Parameters:
        other - the other array
        combiner - the combiner
        Returns:
        this (to simplify use in expressions)
      • indexOf

        public int indexOf​(T target)
        Returns the index of some element equal to given target, or -1 if not present
        Parameters:
        target - the element to search for
        Returns:
        the index or -1 if not present
      • binarySearch

        public int binarySearch​(T target)
        Assuming this array is sorted, returns the index of an element equal to given target, or -1 if not present. If the array is not sorted, the results are undefined.
        Parameters:
        target - the element to search for
        Returns:
        the index or -1 if not present
      • binarySearch

        public int binarySearch​(T target,
                                java.util.Comparator<? super T> comparator)
        Assuming this array is sorted with respect to the given comparator, returns the index of an element equal to given target, or -1 if not present. If the array is not sorted, the results are undefined.
        Parameters:
        target - the element to search for
        comparator - the comparator
        Returns:
        the index or -1 if not present
      • summary

        public ParallelArray.SummaryStatistics<T> summary​(java.util.Comparator<? super T> comparator)
        Returns summary statistics, using the given comparator to locate minimum and maximum elements.
        Overrides:
        summary in class ParallelArrayWithMapping<T,​T>
        Parameters:
        comparator - the comparator to use for locating minimum and maximum elements
        Returns:
        the summary.
      • min

        public T min​(java.util.Comparator<? super T> comparator)
        Returns the minimum element, or null if empty
        Overrides:
        min in class ParallelArrayWithMapping<T,​T>
        Parameters:
        comparator - the comparator
        Returns:
        minimum element, or null if empty
      • min

        public T min()
        Returns the minimum element, or null if empty, assuming that all elements are Comparables
        Overrides:
        min in class ParallelArrayWithMapping<T,​T>
        Returns:
        minimum element, or null if empty
        Throws:
        java.lang.ClassCastException - if any element is not Comparable.
      • max

        public T max​(java.util.Comparator<? super T> comparator)
        Returns the maximum element, or null if empty
        Overrides:
        max in class ParallelArrayWithMapping<T,​T>
        Parameters:
        comparator - the comparator
        Returns:
        maximum element, or null if empty
      • max

        public T max()
        Returns the maximum element, or null if empty assuming that all elements are Comparables
        Overrides:
        max in class ParallelArrayWithMapping<T,​T>
        Returns:
        maximum element, or null if empty
        Throws:
        java.lang.ClassCastException - if any element is not Comparable.
      • cumulate

        public ParallelArray<T> cumulate​(Ops.Reducer<T> reducer,
                                         T base)
        Replaces each element with the running cumulation of applying the given reducer. For example, if the contents are the numbers 1, 2, 3, and the reducer operation adds numbers, then after invocation of this method, the contents would be 1, 3, 6 (that is, 1, 1+2, 1+2+3);
        Parameters:
        reducer - the reducer
        base - the result for an empty array
        Returns:
        this (to simplify use in expressions)
      • precumulate

        public T precumulate​(Ops.Reducer<T> reducer,
                             T base)
        Replaces each element with the cumulation of applying the given reducer to all previous values, and returns the total reduction. For example, if the contents are the numbers 1, 2, 3, and the reducer operation adds numbers, then after invocation of this method, the contents would be 0, 1, 3 (that is, 0, 0+1, 0+1+2, and the return value would be 6 (that is, 1+2+3);
        Parameters:
        reducer - the reducer
        base - the result for an empty array
        Returns:
        the total reduction
      • sort

        public ParallelArray<T> sort​(java.util.Comparator<? super T> comparator)
        Sorts the array. Unlike Arrays.sort, this sort does not guarantee that elements with equal keys maintain their relative position in the array.
        Parameters:
        comparator - the comparator to use
        Returns:
        this (to simplify use in expressions)
      • sort

        public ParallelArray<T> sort()
        Sorts the array, assuming all elements are Comparable. Unlike Arrays.sort, this sort does not guarantee that elements with equal keys maintain their relative position in the array.
        Returns:
        this (to simplify use in expressions)
        Throws:
        java.lang.ClassCastException - if any element is not Comparable.
      • allUniqueElements

        public ParallelArray<T> allUniqueElements()
        Returns a new ParallelArray containing only the non-null unique elements of this array (that is, without any duplicates), using each element's equals method to test for duplication.
        Overrides:
        allUniqueElements in class ParallelArrayWithFilter<T>
        Returns:
        the new ParallelArray
      • allNonidenticalElements

        public ParallelArray<T> allNonidenticalElements()
        Returns a new ParallelArray containing only the non-null unique elements of this array (that is, without any duplicates), using reference identity to test for duplication.
        Overrides:
        allNonidenticalElements in class ParallelArrayWithFilter<T>
        Returns:
        the new ParallelArray
      • removeAll

        public ParallelArray<T> removeAll​(Ops.Predicate<? super T> selector)
        Removes from the array all elements for which the given selector holds.
        Parameters:
        selector - the selector
        Returns:
        this (to simplify use in expressions)
      • hasAllEqualElements

        public <U,​V> boolean hasAllEqualElements​(ParallelArrayWithMapping<U,​V> other)
        Returns true if all elements at the same relative positions of this and other array are equal.
        Overrides:
        hasAllEqualElements in class ParallelArrayWithFilter<T>
        Parameters:
        other - the other array
        Returns:
        true if equal
      • removeConsecutiveDuplicates

        public ParallelArray<T> removeConsecutiveDuplicates()
        Removes consecutive elements that are equal (or null), shifting others leftward, and possibly decreasing size. This method may be used after sorting to ensure that this ParallelArray contains a set of unique elements.
        Returns:
        this (to simplify use in expressions)
      • removeNulls

        public ParallelArray<T> removeNulls()
        Removes null elements, shifting others leftward, and possibly decreasing size.
        Returns:
        this (to simplify use in expressions)
      • addAll

        public ParallelArray<T> addAll​(T[] other)
        Equivalent to asList().addAll but specialized for array arguments and likely to be more efficient.
        Parameters:
        other - the elements to add
        Returns:
        this (to simplify use in expressions)
      • addAll

        public <V> ParallelArray<T> addAll​(ParallelArrayWithMapping<V,​T> other)
        Appends all (possibly bounded, filtered, or mapped) elements of the given ParallelArray, resizing and/or reallocating this array if necessary.
        Parameters:
        other - the elements to add
        Returns:
        this (to simplify use in expressions)
      • withBounds

        public ParallelArrayWithBounds<T> withBounds​(int firstIndex,
                                                     int upperBound)
        Returns an operation prefix that causes a method to operate only on the elements of the array between firstIndex (inclusive) and upperBound (exclusive).
        Parameters:
        firstIndex - the lower bound (inclusive)
        upperBound - the upper bound (exclusive)
        Returns:
        operation prefix
      • withFilter

        public ParallelArrayWithFilter<T> withFilter​(Ops.Predicate<? super T> selector)
        Returns an operation prefix that causes a method to operate only on the elements of the array for which the given selector returns true
        Parameters:
        selector - the selector
        Returns:
        operation prefix
      • withIndexedFilter

        public ParallelArrayWithFilter<T> withIndexedFilter​(Ops.IntAndObjectPredicate<? super T> selector)
        Returns an operation prefix that causes a method to operate only on elements for which the given indexed selector returns true
        Parameters:
        selector - the selector
        Returns:
        operation prefix
      • withMapping

        public <U> ParallelArrayWithMapping<T,​U> withMapping​(Ops.Op<? super T,​? extends U> op)
        Returns an operation prefix that causes a method to operate on mapped elements of the array using the given op.
        Parameters:
        op - the op
        Returns:
        operation prefix
      • withMapping

        public ParallelArrayWithDoubleMapping<T> withMapping​(Ops.ObjectToDouble<? super T> op)
        Returns an operation prefix that causes a method to operate on mapped elements of the array using the given op.
        Parameters:
        op - the op
        Returns:
        operation prefix
      • withMapping

        public ParallelArrayWithLongMapping<T> withMapping​(Ops.ObjectToLong<? super T> op)
        Returns an operation prefix that causes a method to operate on mapped elements of the array using the given op.
        Parameters:
        op - the op
        Returns:
        operation prefix
      • withMapping

        public <U,​V,​W> ParallelArrayWithMapping<T,​V> withMapping​(Ops.BinaryOp<? super T,​? super U,​? extends V> combiner,
                                                                                   ParallelArrayWithMapping<W,​U> other)
        Returns an operation prefix that causes a method to operate on binary mappings of this array and the other array.
        Overrides:
        withMapping in class ParallelArrayWithMapping<T,​T>
        Parameters:
        combiner - the combiner
        other - the other array
        Returns:
        operation prefix
        Throws:
        java.lang.IllegalArgumentException - if other array is a filtered view (all filters must precede all mappings).
      • withIndexedMapping

        public <U> ParallelArrayWithMapping<T,​U> withIndexedMapping​(Ops.IntAndObjectToObject<? super T,​? extends U> mapper)
        Returns an operation prefix that causes a method to operate on mappings of this array using the given mapper that accepts as arguments an element's current index and value, and produces a new value. Index-based mappings allow parallel computation of many common array operations. For example, you could create function to average the values at the same index of multiple arrays and apply it using this method.
        Parameters:
        mapper - the mapper
        Returns:
        operation prefix
      • withIndexedMapping

        public ParallelArrayWithDoubleMapping<T> withIndexedMapping​(Ops.IntAndObjectToDouble<? super T> mapper)
        Returns an operation prefix that causes a method to operate on mappings of this array using the given mapper that accepts as arguments an element's current index and value, and produces a new value.
        Parameters:
        mapper - the mapper
        Returns:
        operation prefix
      • withIndexedMapping

        public ParallelArrayWithLongMapping<T> withIndexedMapping​(Ops.IntAndObjectToLong<? super T> mapper)
        Returns an operation prefix that causes a method to operate on mappings of this array using the given mapper that accepts as arguments an element's current index and value, and produces a new value.
        Parameters:
        mapper - the mapper
        Returns:
        operation prefix
      • iterator

        public java.util.Iterator<T> iterator()
        Returns an iterator stepping through each element of the array up to the current limit. This iterator does not support the remove operation. However, a full ListIterator supporting add, remove, and set operations is available via asList().
        Specified by:
        iterator in interface java.lang.Iterable<T>
        Returns:
        an iterator stepping through each element.
      • asList

        public java.util.List<T> asList()
        Returns a view of this ParallelArray as a List. This List has the same structural and performance characteristics as ArrayList, and may be used to modify, replace or extend the bounds of the array underlying this ParallelArray. The methods supported by this list view are not in general implemented as parallel operations. This list is also not itself thread-safe. In particular, performing list updates while other parallel operations are in progress has undefined (and surely undesired) effects.
        Returns:
        a list view
      • size

        public int size()
        Returns the effective size of the underlying array. The effective size is the current limit, if used (see setLimit(int)), or the length of the array otherwise.
        Overrides:
        size in class AbstractParallelAnyArray
        Returns:
        the effective size of array
      • get

        public T get​(int i)
        Returns the element of the array at the given index
        Parameters:
        i - the index
        Returns:
        the element of the array at the given index
      • set

        public void set​(int i,
                        T x)
        Sets the element of the array at the given index to the given value
        Parameters:
        i - the index
        x - the value
      • getArray

        public T[] getArray()
        Returns the underlying array used for computations
        Returns:
        the array
      • toString

        public java.lang.String toString()
        Equivalent to asList().toString()
        Overrides:
        toString in class java.lang.Object
        Returns:
        a string representation
      • setLimit

        public final void setLimit​(int newLimit)
        Ensures that the underlying array can be accessed up to the given upper bound, reallocating and copying the underlying array to expand if necessary. Or, if the given limit is less than the length of the underlying array, causes computations to ignore elements past the given limit.
        Parameters:
        newLimit - the new upper bound
        Throws:
        java.lang.IllegalArgumentException - if newLimit less than zero.