Learning in Compressed Space
Alexander Fabisch, Yohannes Kassahun, Hendrik Wöhrle, Frank Kirchner
In Neural Networks, Elsevier, volume 42, pages 83-93, 2013.
Abstract
:
We examine two methods which are used to deal with complex machine learning problems: compressed sensing and model compression. We discuss both methods in the context of feed-forward artificial neural networks andmdevelop the ackpropagation method in compressed parameter space. We further show that compressing the weights of a layer of a multilayer perceptron is equivalent to compressing the input of the layer. Based on this theoretical framework, we will use orthogonal functions and especially random projections for compression and perform experiments in supervised and reinforcement learning to demonstrate that the presented methods reduce training time significantly.
Keywords
:
model compression, compressed sensing, artificial neural networks, supervised learning, reinforcement learning
Files:
140611_Learning_in_Compressed_Space_Journal_Fabisch.pdf
Links:
https://www.sciencedirect.com/science/article/pii/S089360801300035X