Περίληψη:
Large scale duplicate detection, clustering and mining of documents or
images has been conventionally treated with seed detection via hashing,
followed by seed growing heuristics using fast search. Principled
clustering methods, especially kernelized and spectral ones, have higher
complexity and are difficult to scale above millions. Under the
assumption of documents or images embedded in Euclidean space, we
revisit recent advances in approximate k-means variants, and borrow
their best ingredients to introduce a new one, inverted-quantized
k-means (IQ-means). Key underlying concepts are quantization of data
points and multi-index based inverted search from centroids to cells.
Its quantization is a form of hashing and analogous to seed detection,
while its updates are analogous to seed growing, yet principled in the
sense of distortion minimization. We further design a dynamic variant
that is able to determine the number of clusters k in a single run at
nearly zero additional cost. Combined with powerful deep learned
representations, we achieve clustering of a 100 million image collection
on a single machine in less than one hour.
Συγγραφείς:
Avrithis, Yannis
Kalantidis, Yannis
Anagnostopoulos, Evangelos
and Emiris, Ioannis Z.