com.chemaxon.overlap.bruteforce

Class UnguardedPagedOverlap<T extends Serializable>

java.lang.Object

com.chemaxon.overlap.bruteforce.UnguardedPagedOverlap<T>

Type Parameters:

T - Unguarded form of the descriptors

All Implemented Interfaces:

Serializable

public final class UnguardedPagedOverlap<T extends Serializable>
extends Object
implements Serializable

Brute force paged similarity search.

This class is an immutable similarity query engine implementation. The query works on unguarded descriptors for storage and computational efficiency.

Licensing: this class can be used with valid LicenseGlobals.MADFAST license.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAX_QUEUE_SIZE Max queue size to avoid filling up the completion queue.

Constructor Summary

Constructors

Constructor and Description
UnguardedPagedOverlap(Function<D,T> extractor, UnguardedDissimilarityCalculator<T> comparator, List<ImmutableList<D>> pages, List<D> page) Construct new immutable reference from pages of Descriptor instances.
UnguardedPagedOverlap(UnguardedDissimilarityCalculator<T> comparator, ImmutableList<ImmutableList<T>> pages) Construct new immutable reference from prepared storage.
UnguardedPagedOverlap(UnguardedDissimilarityCalculator<T> comparator, ImmutableList<ImmutableList<T>> pages, int size) Deprecated. Use UnguardedPagedOverlap(com.chemaxon.descriptors.common.unguarded.UnguardedDissimilarityCalculator, com.google.common.collect.ImmutableList)
UnguardedPagedOverlap(UnguardedDissimilarityCalculator<T> comparator, UnguardedPagedStorage<T> storage) Construct new immutable reference from prepared storage.

Method Summary

Modifier and Type	Method and Description
LinearHistogram	calculateDissimilarityDistribution(T query, double l, double h, int bincount, SubProgressObserver po, PartialResultConsumer<LinearHistogram> partialResultConsumer, ExecutorService ex) Calculate dissimilarity distribution.
ImmutableList<double[]>	calculateFullMatrix(List<T> queries, SubProgressObserver po, ExecutorService ex, boolean managed) Calculate full dissimilarity matrix.
KnnResults	calculateSelfKnn(int k, int queriesGroup, SubProgressObserver po, ExecutorService ex) Brute force find kNN among the contents of the represented descriptors.
ImmutableList<SimilarityResult>	findMostSimilar(List<T> queries, SubProgressObserver po, ExecutorService ex) Brute force find most similar structure for a set of structures.
KnnResults	findMostSimilar(UnguardedPagedStorage<T> queries, int queriesGroup, SubProgressObserver po, ExecutorService ex) Brute force find most similar structure for a set of structures.
KnnResults	findMostSimilarOnSingleThread(List<T> queries, SubProgressObserver po) Single threaded reference for multi query most similar lookup.
SimilarityResult	findMostSimilarOnSingleThread(T query, SubProgressObserver po) Find most similar for a given query structure.
KnnResults	findMostSimilars(int count, UnguardedPagedStorage<T> queries, SubProgressObserver po, ExecutorService ex) Brute force find k-most similar structures for a set of queries.
ImmutableList<SimilarityResult>	findMostSimilars(T query, int count, ProgressObserver progressObserver, PartialResultConsumer<List<SimilarityResult>> partialResultConsumer, ExecutorService executorService) Find most similar structures for a single query.
ImmutableList<ImmutableList<SimilarityResult>>	findMostSimilarsForMultipleQueries(List<T> queries, int count, SubProgressObserver po, ExecutorService ex) Find most multiple similar structures for multiple queries.
ImmutableList<SimilarityResult>	findMostSimilarsOnSingleThread(T query, SubProgressObserver po, int maxCount) Find most similars for a given query structure.
UnguardedDissimilarityCalculator<T>	getDissimilarityComparator() Get represented dissimilarity comparison.
UnguardedPagedStorage<T>	getStorage() Get represented unguarded storage.
UnguardedPagedOverlap<T>	withComparator(UnguardedDissimilarityCalculator<T> comparator) Construct another BruteForcePagedSimilarity instance representing a different comparator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_QUEUE_SIZE

public static final int MAX_QUEUE_SIZE

Max queue size to avoid filling up the completion queue.

See Also:

Constant Field Values

Constructor Detail

UnguardedPagedOverlap

public UnguardedPagedOverlap(UnguardedDissimilarityCalculator<T> comparator,
                             ImmutableList<ImmutableList<T>> pages)

Construct new immutable reference from prepared storage.

Parameters:

comparator - Comparator to be used

pages - List of pages

Throws:

LicenseException - when appropriate license is not available

UnguardedPagedOverlap

@Deprecated
public UnguardedPagedOverlap(UnguardedDissimilarityCalculator<T> comparator,
                                         ImmutableList<ImmutableList<T>> pages,
                                         int size)

Deprecated. Use UnguardedPagedOverlap(com.chemaxon.descriptors.common.unguarded.UnguardedDissimilarityCalculator, com.google.common.collect.ImmutableList)

Construct new immutable reference from prepared storage.

Parameters:

comparator - Comparator to be used

pages - List of pages

size - Total number of descriptors

Throws:

LicenseException - when appropriate license is not available

UnguardedPagedOverlap

public UnguardedPagedOverlap(UnguardedDissimilarityCalculator<T> comparator,
                             UnguardedPagedStorage<T> storage)

Construct new immutable reference from prepared storage.

Parameters:

comparator - Comparator to be used

storage - New storage

Throws:

LicenseException - when appropriate license is not available

UnguardedPagedOverlap

public UnguardedPagedOverlap(Function<D,T> extractor,
                             UnguardedDissimilarityCalculator<T> comparator,
                             List<ImmutableList<D>> pages,
                             List<D> page)

Construct new immutable reference from pages of Descriptor instances.

Type Parameters:

D - Type of descriptors to transform

Parameters:

extractor - Function to extract unguarded descriptor content for storage

comparator - Unguarded comparator to be represented by the constructed instance

pages - List of descriptor pages. This parameter can not be null.

page - Additional descriptor page, considered as last page. Ignored if null given.

Throws:

LicenseException - when appropriate license is not available

Method Detail

withComparator

public UnguardedPagedOverlap<T> withComparator(UnguardedDissimilarityCalculator<T> comparator)

Construct another BruteForcePagedSimilarity instance representing a different comparator.

This operation is memory efficient, since page storage is immutable thus reused.

Parameters:

comparator - Comparator to be used

Returns:

Instance representing comparison with the given comparator

getStorage

public UnguardedPagedStorage<T> getStorage()

Get represented unguarded storage.

Returns:

Unguarded storage

getDissimilarityComparator

public UnguardedDissimilarityCalculator<T> getDissimilarityComparator()

Get represented dissimilarity comparison.

Returns:

Comparator

findMostSimilars

public ImmutableList<SimilarityResult> findMostSimilars(T query,
                                                        int count,
                                                        ProgressObserver progressObserver,
                                                        PartialResultConsumer<List<SimilarityResult>> partialResultConsumer,
                                                        ExecutorService executorService)

Find most similar structures for a single query.

This method utilizes concurrent execution and blocks until completion. ProgressObserver callback is invoked from the calling thread only.

\

Parameters:

query - Query descriptor

count - Number of expected most similar structures

partialResultConsumer -

executionMonitoring - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

executorService - ExecutorService to run workers

Returns:

List of most similar structures, ordered by increasing dissimilarity order

Throws:

CancellationException - when cancelled through supplied progress observer

findMostSimilarsForMultipleQueries

public ImmutableList<ImmutableList<SimilarityResult>> findMostSimilarsForMultipleQueries(List<T> queries,
                                                                                         int count,
                                                                                         SubProgressObserver po,
                                                                                         ExecutorService ex)

Find most multiple similar structures for multiple queries.

This method utilizes concurrent execution and blocks until completion. ProgressObserver callback is invoked from the calling thread only.

Note that ProgressObserver.subTask(java.lang.String, long) is invoked on the supplied ProgressObserver.

Parameters:

queries - Query descriptors

count - Number of expected most similar structures

po - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

ex - ExecutorService to run workers

Returns:

List of most similar structures (ordered by increasing dissimilarity order) for every query structures.

Throws:

CancellationException - when cancelled through supplied progress observer

calculateFullMatrix

public ImmutableList<double[]> calculateFullMatrix(List<T> queries,
                                                   SubProgressObserver po,
                                                   ExecutorService ex,
                                                   boolean managed)

Calculate full dissimilarity matrix.

Calculate full dissimilarity matrix between the represented targets and the given queries. Note that the resulting matrix might have be excessively large size.

The resulting structure is allocated upon startup.

Parameters:

queries - Query descriptors

po - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

ex - ExecutorService to run workers

managed - if true, use JMX for monitoring

Returns:

List of dissimilarity values for each query. Element i will contain the dissimilarity vector for query i. The dissimilarity vector contains the dissimilarities of targets. *

Throws:

CancellationException - when cancelled through supplied progress observer

IllegalStateException - possible exceptions due to JMX connection are wrapped to IllegalStateException

calculateDissimilarityDistribution

public LinearHistogram calculateDissimilarityDistribution(T query,
                                                          double l,
                                                          double h,
                                                          int bincount,
                                                          SubProgressObserver po,
                                                          PartialResultConsumer<LinearHistogram> partialResultConsumer,
                                                          ExecutorService ex)

Calculate dissimilarity distribution.

Parameters:

query - Query descriptor

l - Lowest bound

h - Highest bound

bincount - Bun count

po - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

partialResultConsumer -

ex - ExecutorService to run workers

Returns:

Dissimilarity distribution histogram

calculateSelfKnn

public KnnResults calculateSelfKnn(int k,
                                   int queriesGroup,
                                   SubProgressObserver po,
                                   ExecutorService ex)
                            throws ExecutionException

Brute force find kNN among the contents of the represented descriptors.

Parameters:

k - Number of nearest neighbors to find

queriesGroup - Number of queries to group

po - ProgressObserver to track progress.

ex - ExecutorService to run workers. Upon completion ProgressObserver.done() will be invoked.

Returns:

Storage of knn found. Note that self ids are not part of the knn lists,

Throws:

ExecutionException - Re-theown

findMostSimilars

public KnnResults findMostSimilars(int count,
                                   UnguardedPagedStorage<T> queries,
                                   SubProgressObserver po,
                                   ExecutorService ex)

Brute force find k-most similar structures for a set of queries.

Parameters:

count - Most similars count to find

queries - Query descriptors

po - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

ex - ExecutorService to run workers

Returns:

Storage of most similars

findMostSimilar

public KnnResults findMostSimilar(UnguardedPagedStorage<T> queries,
                                  int queriesGroup,
                                  SubProgressObserver po,
                                  ExecutorService ex)

Brute force find most similar structure for a set of structures. For each query index the most similar target index is recorded.

Parameters:

queries - Query descriptors

queriesGroup - Number of queries to group

po - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

ex - ExecutorService to run workers

Returns:

Storage of most similars

findMostSimilar

public ImmutableList<SimilarityResult> findMostSimilar(List<T> queries,
                                                       SubProgressObserver po,
                                                       ExecutorService ex)

Brute force find most similar structure for a set of structures.

This method utilizes concurrent execution and blocks until completion. ProgressObserver callback is invoked from the calling thread only.

Note that when the best dissimilarity score is associated for multiple queries one of them is picked. The selection in this case is non deterministic.

Parameters:

queries - Query descriptors (in the unguarded form)

po - ProgressObserver to track progress. Upon completion ProgressObserver.done() will be invoked.

ex - ExecutorService to run workers

Returns:

List of most similar nodes associated to the query structures

Throws:

CancellationException - when cancelled through supplied progress observer

findMostSimilarOnSingleThread

public KnnResults findMostSimilarOnSingleThread(List<T> queries,
                                                SubProgressObserver po)
                                         throws CancellationException

Single threaded reference for multi query most similar lookup.

Parameters:

queries - Queries to search

po - Observer; will be closed upon finish/abort/error. One work unit is assigned to one comparison.

Returns:

Results

Throws:

CancellationException - When cancelled through supplied progress observer

findMostSimilarOnSingleThread

public SimilarityResult findMostSimilarOnSingleThread(T query,
                                                      SubProgressObserver po)
                                               throws CancellationException

Find most similar for a given query structure.

This method is intended only for test/diagnostincs. Users of this API usually need to invoke findMostSimilar(java.util.List, com.chemaxon.calculations.common.SubProgressObserver, java.util.concurrent.ExecutorService) instead.

This method blocks until ready and uses a single (the calling) thread to do the calculation.

Parameters:

query - Query descriptor

po - Progress observer to track progress. Completion is reported by invoking ProgressObserver.done() upon completion, cancellation or error

Returns:

The most similar structure found

Throws:

CancellationException - upon cancellation through the given progress observer

findMostSimilarsOnSingleThread

public ImmutableList<SimilarityResult> findMostSimilarsOnSingleThread(T query,
                                                                      SubProgressObserver po,
                                                                      int maxCount)
                                                               throws CancellationException

Find most similars for a given query structure.

This method is intended only for test/diagnostincs. Users of this API usually need to invoke findMostSimilar(java.util.List, com.chemaxon.calculations.common.SubProgressObserver, java.util.concurrent.ExecutorService) instead.

This method blocks until ready and uses a single (the calling) thread to do the calculation.

Parameters:

query - Query descriptor

po - Progress observer to track progress. Completion is reported by invoking ProgressObserver.done() upon completion, cancellation or error

maxCount - Max results count

Returns:

List of at most the given number of most similars, in increasing dissimilarity order

Throws:

CancellationException - upon cancellation through the given progress observer