Large-scale independent component analysis by speeding up Lie group techniques
- We are interested in computing a mini-batch-capable end-to-end algorithm to identify statistically independent components (ICA) in large scale and high-dimensional datasets. Current algorithms typically rely on pre-whitened data and do not integrate the two procedures of whitening and ICA estimation. Our online approach estimates a whitening and a rotation matrix with stochastic gradient descent on centered or uncentered data. We show that this can be done efficiently by combining Batch Karhunen-Löwe-Transformation [1] with Lie group techniques. Our algorithm is recursion-free and can be organized as feed-forward neural network which makes the use of GPU acceleration straight-forward. Because of the very fast convergence of Batch KLT, the gradient descent in the Lie group of orthogonal matrices stabilizes quickly. The optimization is further enhanced by integrating ADAM [2], an improved stochastic gradient descent (SGD) technique from the field of deep learning. We test the scaling capabilities by computing the independent components of the well-known ImageNet challenge (144 GB). Due to its robustness with respect to batch and step size, our approach can be used as a drop-in replacement for standard ICA algorithms where memory is a limiting factor.
Author: | Matthias Hermann, Georg UmlaufORCiDGND, Matthias O. FranzORCiDGND |
---|---|
DOI: | https://doi.org/10.1109/ICASSP43922.2022.9746444 |
ISBN: | 978-1-6654-0540-9 |
ISBN: | 978-1-6654-0541-6 |
Parent Title (English): | International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022, 22 - 27 May, Singapore |
Publisher: | IEEE |
Document Type: | Conference Proceeding |
Language: | English |
Year of Publication: | 2022 |
Release Date: | 2022/12/07 |
Tag: | ICA; Lie group; ADAM |
Page Number: | 5 Seiten |
First Page: | 4388 |
Last Page: | 4392 |
Note: | Volltext im Campusnetz via Datenbank IEEE Xplore abrufbar. |
Relevance: | Peer reviewed nach anderen Listungen (mit Nachweis zum Peer Review Verfahren) |
Open Access?: | Nein |
Licence (German): | Urheberrechtlich geschützt |