From iterative algorithms to deep learning

Ami Wiesel

Hebrew University of Jerusalem, Israel

Email: ami.wiesel@huji.ac.il

Abstract 

Deep neural network and machine learning have recently revolutionized all fields of engineering. The most significant progress is probably in image processing and speech, but these trends are penetrating other applications as wireless communication, radar and tomography. The switch from classical signal processing to modern deep learning involves many challenges. The first is technical and simple, yet daunting for experienced researchers. This is the transition to modern numerical toolboxes which are more suitable for learning tasks, e.g., the migration from Matlab to Python. A more fundamental change is that the applications mentioned above all have a well understood physical model that should be exploited in the design. This leads to a switch from model based design to data driven design, and, more importantly the shift to architecture based design. Another significant change is that standard machine learning methods are not suitable for parametric signal processing, and must be re-learned from scratch each time the parameters change. These challenges led researchers to consider hybrid approaches that exploit the benefits of both worlds: the network’s architectures are based on unfolding iterative signal processing algorithms, while deep learning allows more degrees of freedom and more expression power. Iterations are transformed into layers, and algorithms are succeeded by networks (see also the related concept of Recurrent Neural Networks). The resulting architectures allow a single training for multiple parametric models and achieve state of the art accuracy vs complexity tradeoffs. The goal of this tutorial is to introduce these ideas to engineers with a strong background in classical signal processing and zero experience in deep learning.

Tutorial Duration

2 hours

Biography

Ami Wiesel received the B.Sc. and M.Sc. degrees in electrical engineering from Tel-Aviv University, Tel-Aviv, Israel, in 2000 and 2002, respectively, and the Ph.D. degree in electrical engineering from the Technion - Israel Institute of Technology, Haifa, Israel, in 2007. He was a postdoctoral fellow with the Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA, during 2007–2009. He is currently an Associate Professor in the Rachel and Selim Benin School of Computer Science and Engineering, Hebrew University of Jerusalem, Israel.

Website: http://www.cs.huji.ac.il/~amiw/