Embed Size (px)
Citation preview
DATA
Edouard Oyallon
On the road to Affine Scattering Transform
1
with Stéphane Mallatfollowing the work of Laurent Sifre, Joan Bruna, …
DATA
• Caltech 101, etc
22
Training set to predict labels Rhino
Not a rhinoRhino class from Caltech 101
High Dimensional classification(xi, yi) 2 R5122 ⇥ {1, ..., 100}, i = 1...104 �! y(x)?
DATA
High dimensionality issues
3
x y kx� yk2 = 2
Averaging is the key to get invariants
x y
Translation
Rotation
DATAFighting the curse of
dimensionality4
Geometric variability Class variabilityGroups acting on images:
translation, rotation, scalingIntraclass variability
Extraclass variabilityOther sources : luminosity, occlusion, small deformations
Can be carefully handled Needs to be learned
�x L�xx
Classifier
Not informative
x⌧ (u) = x(u� ⌧(u)), ⌧ 2 C1
L reduces class variability
I � ⌧
DATA
Existing approach
• Unsupervised learning: Bag of Words, Fisher Vector,…
• Supervised learning: Deep Learning,…
• Non learned: HMax, Scattering Transform.
5
{x1, ..., xN} �! �
{G1, ...} �! �
{(x1, y1), ..., (xN , yN )} �! L�
Ref.: Discovering objects and their location in images . Sivic et al.
High-dimensional Signature Compression for Large-Scale Image Classification, Perronnin et al.
Ref.: Robust Object Recognition with Cortex-Like Mechanisms. Serre et al.
DATADeepNet?
• It is a cascade based on a lot of linear operators followed by non linearities.
• Each operator is supervisedly learned
• Sort of “Super SIFT”
• State-of-the arts on ImageNet and most of the benchmarks.
6
Ref.: Rich feature hierarchies for accurate object detection and semantic segmentation. Girshick et al.
Convolutional network and applications in vision. Y. LeCun et al.
% a
ccur
acy
60
70
80
90
100
2010 2011 2012 2013 2014 2015
SIFT+FV DeepNet
Ref.: image-net.org
human
DATAComplexity of the
architecture• Requires a huge amount of data
• Need many engineering to select the hyper parameters and to optimise it
• Interpreting the learned operators is hard when the network is deep(i.e. more than 3 layers)
• Few theoretical results, yet outstanding numerical results.
7
Ref.: Intriguing properties of neural networks, C. Szegedy et al.
DATAOperators of a Deep
architecture• Linear operators are often convolutional whose
kernels are small filters.
• Deep.
• is often a ReLu:
• Sometimes “pooling” which leads to a down sampling.
8
xj(u,�2) = f(X
�1
xj�1(.,�1) ? hj,�1(u)) ! xj = f(Fjxj�1)
x ! max(0, x)
f
Contains a phase information
DATA 9
x0
x1x2
xJ
…f(F1) f(F2) f(FJ)
DeepNetwork
Classifier
Ref.: ImageNet Classification with Deep Convolutional Neural Networks.
A Krizhevsky et al.
DATADoes everything need to be
learned?• A Scattering Network is a deep architecture, where all the filters are predefined.
• We challenge the necessity to learn the weights of every filters of a deep architecture.
• Scattering gets state-of-the-art results on unsupervised learning for some complex datasets.
• We highlight the similarity of the Scattering Network with the DeepNet architectures.
10
Ref.: Invariant Convolutional Scattering Network, J. Bruna et Mallat S
Ref.: Deep Rototranslation Scattering Network for complex Image Recognition, EO, S Mallat
DATADesirable properties of a representation
11
• Invariance to group of transformation
• Stability to noise
• Reconstruction properties
• Linear separation of the different classes
8x, 8g 2 G,�(g.x) = �(x)<latexit sha1_base64="nEwMldBod1KkAndkku6+pt7TZD8=">AAACE3icbZBNS0JBFIbPtS+zr1st2wxKoChyb4tqE0gtammQKXhF5o6jDs6de5mZG4r4I9q07l+0aVHRtk27/k3jR1DaCwMP7zmHOef1I86UdpwvK7G0vLK6llxPbWxube/Yu3u3KowloRUS8lDWfKwoZ4JWNNOc1iJJceBzWvV7F+N69Y5KxUJxowcRbQS4I1ibEayN1bTzXjuUmHPUL/xQx2MCXRaQV+6ybKfYz51NqJ9r2hmn6EyEFsGdQaaU9vKPAFBu2p9eKyRxQIUmHCtVd51IN4ZYakY4HaW8WNEIkx7u0LpBgQOqGsPJUSN0aJwWMjuZJzSauL8nhjhQahD4pjPAuqvma2Pzv1o91u3TxpCJKNZUkOlH7ZgjHaJxQqjFJCWaDwxgIpnZFZEulphok2PKhODOn7wIlaPicdG5djOlc5gqCQeQhiy4cAIluIIyVIDAPTzBC7xaD9az9Wa9T1sT1mxmH/7I+vgGECKd9g==</latexit><latexit sha1_base64="SRDvLA/owkcV7KmoCnqMJpZnjn0=">AAACE3icbZDLSsNAFIYnXmu9RV26GVqElpaSuFA3QtCFLisYW2hCmUyn7dDJJMxMpKH0IQTxVdy4UHHrxl3fxulF0NYfBj7+cw5zzh/EjEplWSNjaXlldW09s5Hd3Nre2TX39u9klAhMXByxSNQDJAmjnLiKKkbqsSAoDBipBb3Lcb12T4SkEb9VaUz8EHU4bVOMlLaaZslrRwIxBvvlH+p4lMOrMvSqXVroVPrF8wn1i00zb1WsieAi2DPIOzmv9Dhy0mrT/PJaEU5CwhVmSMqGbcXKHyChKGZkmPUSSWKEe6hDGho5Con0B5OjhvBIOy2od9KPKzhxf08MUChlGga6M0SqK+drY/O/WiNR7TN/QHmcKMLx9KN2wqCK4Dgh2KKCYMVSDQgLqneFuIsEwkrnmNUh2PMnL4J7XDmpWDd23rkAU2XAIciBArDBKXDANagCF2DwAJ7BK3gznowX4934mLYuGbOZA/BHxuc3GAaffA==</latexit>
8x, y, k�(x)� �(y)k2 kx� yk2<latexit sha1_base64="5AfMzQTj9eto92B9ZxcFXNdIZTg=">AAACJXicbVDLSgNBEOyN7/iKevQyJAiKMezmoF6EoBePEYwJZEOYnXTMkNmHM7OSJfgvghd/xYsHFcGTv+Jkk4MmFgxTVHXT3eVFgitt219WZm5+YXFpeSW7ura+sZnb2r5RYSwZ1lgoQtnwqELBA6xprgU2IonU9wTWvf7FyK/fo1Q8DK51EmHLp7cB73JGtZHauTO3G0oqBBkUkyIh7g1K7VZ7fH9wcJT+yUGqtcuuwLuUksFRMtHauYJdslOQWeJMSKGSdw8fAaDazr25nZDFPgaaCapU07Ej3RpSqTkT+JB1Y4URZX16i01DA+qjag3TMx/InlE6xOxrXqBJqv7uGFJfqcT3TKVPdU9NeyPxP68Z6+5pa8iDKNYYsPGgbiyIDskoM9LhEpkWiSGUSW52JaxHJWXaJJs1ITjTJ8+SWrl0XLKvnELlHMZYhl3Iwz44cAIVuIQq1IDBE7zAG7xbz9ar9WF9jksz1qRnB/7A+v4BXDqllA==</latexit><latexit sha1_base64="6GQoM5yxWlHT4qVbK4XRAWsGLro=">AAACJXicbVDLTgIxFO34RHyhLt00EBOIQGZYqBsTohuXmMgjYQjplAs0dB62HcOE8Bf+gRt/xY0LNCas/BXLwELBkzQ9Oefe3HuPE3AmlWlOjbX1jc2t7cROcndv/+AwdXRck34oKFSpz33RcIgEzjyoKqY4NAIBxHU41J3B7cyvP4GQzPceVBRAyyU9j3UZJUpL7dS13fUF4RwP81EeY7sGQtmVPssOc4X4j3Kx1i7ZHB5jioeFaKG1UxmzaMbAq8RakEw5bZ8/T8tRpZ2a2B2fhi54inIiZdMyA9UaEaEY5TBO2qGEgNAB6UFTU4+4IFuj+MwxPtNKB+t99fMUjtXfHSPiShm5jq50ierLZW8m/uc1Q9W9ao2YF4QKPDof1A05Vj6eZYY7TABVPNKEUMH0rpj2iSBU6WSTOgRr+eRVUi0VL4rmvZUp36A5EugUpVEWWegSldEdqqAqougFvaEJ+jBejXfj0/ial64Zi54T9AfG9w9kHqca</latexit>
y = �(x) () x = ��1(y)<latexit sha1_base64="ZBot7JBA2TU1Iy/+RhD2jwIrZVo=">AAACFHicbVA9bxNBEJ0zSTAmHwZKmlWsSLaSWHcpgAbJgoYihSPFiSWfsfbWe/bKe7un3Tnw6eQ/QUPLz6ChIIiWgi7/JuuPIrHzpJGe3pvRzLwolcKi7996pSdb2ztPy88qz3f39g+qL15eWZ0ZxjtMS226EbVcCsU7KFDybmo4TSLJr6PJx7l//YUbK7S6xDzl/YSOlIgFo+ikQfUkfx+2x6I+bZDwXKuR5DEaMRojNUZ/JdOF+7k4DWb1vDGo1vymvwDZJMGK1FqH4fEPAGgPqv/DoWZZwhUySa3tBX6K/YIaFEzyWSXMLE8pm9AR7zmqaMJtv1h8NSNHThmSWBtXCslCvT9R0MTaPIlcZ0JxbNe9ufiY18swftcvhEoz5IotF8WZJKjJPCIyFIYzlLkjlBnhbiVsTA1l6IKsuBCC9Zc3Sees+abpXwS11gdYogyv4RDqEMBbaMEnaEMHGHyDn/Abbrzv3i/vj/d32VryVjOv4AG8f3ebxZ+F</latexit><latexit sha1_base64="yAq/o5/ileD4o5cpInGjcKTVrWY=">AAACFHicbVA9SwNBEN3z2/gVtbRZEoSIGu4s1EYI2lhYRDAq5GLY2+wlS/Z2j9059TjyJ0Twr9hYqNha2OXfuEks/How8Hhvhpl5QSy4AdftO2PjE5NT0zOzubn5hcWl/PLKuVGJpqxGlVD6MiCGCS5ZDTgIdhlrRqJAsIugezTwL66ZNlzJM0hj1ohIW/KQUwJWaua30gO/2uGl2w3snyjZFiwEzdsdIFqrG3w7dK+yba9XSjea+aJbdofAf4n3RYqVgr9536+k1Wb+w28pmkRMAhXEmLrnxtDIiAZOBevl/MSwmNAuabO6pZJEzDSy4Vc9vG6VFg6VtiUBD9XvExmJjEmjwHZGBDrmtzcQ//PqCYT7jYzLOAEm6WhRmAgMCg8iwi2uGQWRWkKo5vZWTDtEEwo2yJwNwfv98l9S2ynvlt1Tr1g5RCPMoDVUQCXkoT1UQceoimqIojv0iJ7Ri/PgPDmvztuodcz5mllFP+C8fwKjqaEL</latexit>
x<latexit sha1_base64="hoBhb2jLP8IXA4nPqY1HvRHBiuw=">AAAB53icbZC7SgNBFIbPeo3xFrW0GQyCVdi1ULExYGOZgGsCyRJmJ2eTMbOzy8ysGEKewMZCxVZ8EZ/BRnwbJ5dCE38Y+Pj/c5hzTpgKro3rfjsLi0vLK6u5tfz6xubWdmFn90YnmWLos0Qkqh5SjYJL9A03AuupQhqHAmth73KU1+5QaZ7Ia9NPMYhpR/KIM2qsVb1vFYpuyR2LzIM3heLFx/sXWFVahc9mO2FZjNIwQbVueG5qggFVhjOBw3wz05hS1qMdbFiUNEYdDMaDDsmhddokSpR90pCx+7tjQGOt+3FoK2Nquno2G5n/ZY3MRGfBgMs0MyjZ5KMoE8QkZLQ1aXOFzIi+BcoUt7MS1qWKMmNvk7dH8GZXngf/uHRScqtesXwOE+VgHw7gCDw4hTJcQQV8YIDwAE/w7Nw6j86L8zopXXCmPXvwR87bD6Wejzo=</latexit><latexit sha1_base64="FtGdyuSf7GZrM9SUbmoIozrVEXw=">AAAB53icbZDLSgNBEEVrfMbxFXXppjEIrsKMCxVBDLhxmYBjAskQejo1SZueB909Ygj5AjcuVNyKfojf4Eb8GzuPhSZeaDjcW0VXVZAKrrTjfFtz8wuLS8u5FXt1bX1jM7+1fa2STDL0WCISWQuoQsFj9DTXAmupRBoFAqtB92KYV29RKp7EV7qXoh/RdsxDzqg2VuWumS84RWckMgvuBArnH29f9ln6Xm7mPxuthGURxpoJqlTddVLt96nUnAkc2I1MYUpZl7axbjCmESq/Pxp0QPaN0yJhIs2LNRm5vzv6NFKqFwWmMqK6o6azoflfVs90eOL3eZxmGmM2/ijMBNEJGW5NWlwi06JngDLJzayEdaikTJvb2OYI7vTKs+AdFo+KTsUtlE5hrBzswh4cgAvHUIJLKIMHDBDu4RGerBvrwXq2Xsalc9akZwf+yHr9AXOgkJU=</latexit>
�(x)<latexit sha1_base64="Gm84J/6MA8kbnhEr0bKEWWdUghc=">AAAB7XicbZA9SwNBEIbn4leMX1E7bRajEJtwZ6FiFbCxjOCZQHKEvc0mWbK3d+zOieFI6Q+wsVCx9f/Y+W/cfBSa+MLCw/vOsDMTJlIYdN1vJ7e0vLK6ll8vbGxube8Ud/fuTZxqxn0Wy1g3Qmq4FIr7KFDyRqI5jULJ6+HgepzXH7g2IlZ3OEx4ENGeEl3BKFqr3qr1RfnxtF0suRV3IrII3gxK1YOnY7CqtYtfrU7M0ogrZJIa0/TcBIOMahRM8lGhlRqeUDagPd60qGjETZBNxh2RE+t0SDfW9ikkE/d3R0YjY4ZRaCsjin0zn43N/7Jmit3LIBMqSZErNv2om0qCMRnvTjpCc4ZyaIEyLeyshPWppgzthQr2CN78yovgn1XOK+6tV6pewVR5OIQjKIMHF1CFG6iBDwwG8Ayv8OYkzovz7nxMS3POrGcf/sj5/AFM+o/O</latexit><latexit sha1_base64="iG/W51Dvf9Qc2VZ4EWQmcWTiwZk=">AAAB7XicbZC7SgNBFIbPxluMt6idNoOLEJuwm0LFKmBjI0RwTSBZwuxkNhkyO7vMzIphCVY+gI2Fiq1v4UPY+SLWTi6FJv4w8PH/5zDnnCDhTGnH+bJyC4tLyyv51cLa+sbmVnF750bFqSTUIzGPZSPAinImqKeZ5rSRSIqjgNN60D8f5fVbKhWLxbUeJNSPcFewkBGsjVVv1XqsdHfULtpO2RkLzYM7Bbu692B/319+1NrFz1YnJmlEhSYcK9V0nUT7GZaaEU6HhVaqaIJJH3dp06DAEVV+Nh53iA6N00FhLM0TGo3d3x0ZjpQaRIGpjLDuqdlsZP6XNVMdnvoZE0mqqSCTj8KUIx2j0e6owyQlmg8MYCKZmRWRHpaYaHOhgjmCO7vyPHiV8nHZuXLt6hlMlId9OIASuHACVbiAGnhAoA+P8AwvVmI9Wa/W26Q0Z017duGPrPcfofySTQ==</latexit>
8i,�(�(Xi)) ⌧ 1<latexit sha1_base64="1XfMdUJGtThL4UpbiZCwN8IO4KM=">AAACCnicbVDLSsNAFL3xWesr6tLN0CK0KCVxoS6LblxWMLbQhDCZTtuhk0mYmQgldO/Gf/AL3LhQcesXuPNvnD4W2nrgwuGce7n3nijlTGnH+baWlldW19YLG8XNre2dXXtv/04lmSTUIwlPZCvCinImqKeZ5rSVSorjiNNmNLga+817KhVLxK0epjSIcU+wLiNYGym0S343kZhzxE6Qr1gvxhW/0WeVVsiqVeQbww3tslNzJkCLxJ2Rcr3kHz8BQCO0v/xOQrKYCk04VqrtOqkOciw1I5yOin6maIrJAPdo21CBY6qCfPLLCB0ZpYPMUaaERhP190SOY6WGcWQ6Y6z7at4bi/957Ux3L4KciTTTVJDpom7GkU7QOBjUYZISzYeGYCKZuRWRPpaYaBNf0YTgzr+8SLzT2lnNuXHL9UuYogCHUIIKuHAOdbiGBnhA4AGe4RXerEfrxXq3PqatS9Zs5gD+wPr8AVSLmlc=</latexit><latexit sha1_base64="hbbAurJdhcIxnua1KCMZnk8/EbI=">AAACCnicbVDLSsNAFJ3UV62vqEs3Q4vQopTEhboMunFZwdhCE8JkOmmHTiZhZiKE0L0bd36HGxcqbv0Cd/0bp4+Fth64cDjnXu69J0wZlcqyxkZpZXVtfaO8Wdna3tndM/cP7mWSCUxcnLBEdEIkCaOcuIoqRjqpICgOGWmHw+uJ334gQtKE36k8JX6M+pxGFCOlpcCselEiEGOQnkJP0n6M6l5rQOudgDYa0NOGHZg1q2lNAZeJPSc1p+qdPI+dvBWY314vwVlMuMIMSdm1rVT5BRKKYkZGFS+TJEV4iPqkqylHMZF+Mf1lBI+10oP6KF1cwan6e6JAsZR5HOrOGKmBXPQm4n9eN1PRpV9QnmaKcDxbFGUMqgROgoE9KghWLNcEYUH1rRAPkEBY6fgqOgR78eVl4p41z5vWrV1zrsAMZXAEqqAObHABHHADWsAFGDyCF/AG3o0n49X4MD5nrSVjPnMI/sD4+gFcb5vd</latexit>
8i 6= j, kE(�(Xi))� E(�(Xj))k2 � 1<latexit sha1_base64="mmW7b12YiT5pPp6c5E2nQA1B1iU=">AAACJXicbVDLSgMxFM3UV62vqks3wSK0oGWmiLoRiiK4rGAf0ClDJr1t02YyY5IRSunXuPFX3LioIrjyV0wfiLYeCJyccy/33uNHnClt259WYml5ZXUtuZ7a2Nza3knv7lVUGEsKZRryUNZ8ooAzAWXNNIdaJIEEPoeq37se+9VHkIqF4l73I2gEpC1Yi1GijeSlL91WKAnnmLkCHnD3GLsVkBrfZN1Sh2VrHsvlTn4+3VxuYnsF7Lbb2PHSGTtvT4AXiTMjGTRDyUuP3GZI4wCEppwoVXfsSDcGRGpGOQxTbqwgIrRH2lA3VJAAVGMwOXOIj4zSxGZf84TGE/V3x4AESvUD31QGRHfUvDcW//PqsW5dNAZMRLEGQaeDWjHHOsTjzHCTSaCa9w0hVDKzK6YdIgnVJtmUCcGZP3mRlAv5s7x9d5opXs3SSKIDdIiyyEHnqIhuUQmVEUVP6AWN0Jv1bL1a79bHtDRhzXr20R9YX99o7qI0</latexit><latexit sha1_base64="mmW7b12YiT5pPp6c5E2nQA1B1iU=">AAACJXicbVDLSgMxFM3UV62vqks3wSK0oGWmiLoRiiK4rGAf0ClDJr1t02YyY5IRSunXuPFX3LioIrjyV0wfiLYeCJyccy/33uNHnClt259WYml5ZXUtuZ7a2Nza3knv7lVUGEsKZRryUNZ8ooAzAWXNNIdaJIEEPoeq37se+9VHkIqF4l73I2gEpC1Yi1GijeSlL91WKAnnmLkCHnD3GLsVkBrfZN1Sh2VrHsvlTn4+3VxuYnsF7Lbb2PHSGTtvT4AXiTMjGTRDyUuP3GZI4wCEppwoVXfsSDcGRGpGOQxTbqwgIrRH2lA3VJAAVGMwOXOIj4zSxGZf84TGE/V3x4AESvUD31QGRHfUvDcW//PqsW5dNAZMRLEGQaeDWjHHOsTjzHCTSaCa9w0hVDKzK6YdIgnVJtmUCcGZP3mRlAv5s7x9d5opXs3SSKIDdIiyyEHnqIhuUQmVEUVP6AWN0Jv1bL1a79bHtDRhzXr20R9YX99o7qI0</latexit>
G
DATASuccess story
Scattering for Textures&Digits• Non-learned representation have been
successively used on:
• Digits (patterns):
• Textures (stationary process):
• However all the variabilities(groups) here are perfectly understood.
12
Small deformations +Translation
Ref.: Invariant Convolutional Scattering Network, J. Bruna and S Mallat
Ref.: Rotation, Scaling and Deformation Invariant Scattering for texture discrimination, Sifre L and Mallat S.
Rotation+Scale
DATAWavelets
• Wavelets help to describe signal structures. is a wavelet iff
• Learned in the first layers of a DeepNet.
• Wavelets can be dilated in order to be a multi-scale representation of signals, rotated to describe rotations.
• Design wavelets selective to an informative variability.
13
<latexit sha1_base64="vZM3p0oDK8G//ZjqteTWKP152iI=">AAAB6nicbZDLSgMxFIbPtF5qvY2XnZtgEVyVGRcqrgpuXFZwbKEdSibNtKFJZkgyQhkKPoEbFypufQ4fwp1vY6btQlt/CHz8/znknBOlnGnjed9OqbyyurZe2ahubm3v7Lp7+/c6yRShAUl4otoR1pQzSQPDDKftVFEsIk5b0ei6yFsPVGmWyDszTmko8ECymBFsCqubatZza17dmwotgz+HWuPws/wIAM2e+9XtJyQTVBrCsdYd30tNmGNlGOF0Uu1mmqaYjPCAdixKLKgO8+msE3RinT6KE2WfNGjq/u7IsdB6LCJbKbAZ6sWsMP/LOpmJL8OcyTQzVJLZR3HGkUlQsTjqM0WJ4WMLmChmZ0VkiBUmxp6nao/gL668DMFZ/bzu3fq1xhXMVIEjOIZT8OECGnADTQiAwBCe4AVeHeE8O2/O+6y05Mx7DuCPnI8frjmPpA==</latexit><latexit sha1_base64="MKReSqAJM21mHrPcc9EWVyc1loM=">AAAB6nicbZC7SgNBFIbPJl5ivMVLZzMYBKuwa6FiFbCxjOAaIVnC7GQ2GTIzO8zMCiHkFWwsVGx9DmsbGzvfxtkkhSb+MPDx/+cw55xYcWas7397heLS8spqaa28vrG5tV3Z2b01aaYJDUnKU30XY0M5kzS0zHJ6pzTFIua0GQ8u87x5T7VhqbyxQ0UjgXuSJYxgm1ttZVinUvVr/kRoEYIZVOv770X1+ZE1OpWvdjclmaDSEo6NaQW+stEIa8sIp+NyOzNUYTLAPdpyKLGgJhpNZh2jI+d0UZJq96RFE/d3xwgLY4YidpUC276Zz3Lzv6yV2eQ8GjGpMkslmX6UZBzZFOWLoy7TlFg+dICJZm5WRPpYY2LdecruCMH8yosQntROa/51UK1fwFQlOIBDOIYAzqAOV9CAEAj04QGe4NkT3qP34r1OSwverGcP/sh7+wFQoJGe</latexit>
<latexit sha1_base64="mduu63BoHMqGld23Vba2fG/RufU=">AAAB6nicbZC9TsMwFIVvyl8pfwFGGCwqJKYqYQDGChbGViK0UhtVjuu0Vm0nsh2kKuorsDAAYuUheA42Nh4Fp+0ALUey9Omce+V7b5Rypo3nfTmlldW19Y3yZmVre2d3z90/uNdJpggNSMIT1Y6wppxJGhhmOG2nimIRcdqKRjdF3nqgSrNE3plxSkOBB5LFjGBTWN1Us55b9WreVGgZ/DlU68cfzW8AaPTcz24/IZmg0hCOte74XmrCHCvDCKeTSjfTNMVkhAe0Y1FiQXWYT2edoFPr9FGcKPukQVP3d0eOhdZjEdlKgc1QL2aF+V/WyUx8FeZMppmhksw+ijOOTIKKxVGfKUoMH1vARDE7KyJDrDAx9jwVewR/ceVlCM5rFzWv6Vfr1zBTGY7gBM7Ah0uowy00IAACQ3iEZ3hxhPPkvDpvs9KSM+85hD9y3n8AnNqQXA==</latexit><latexit sha1_base64="KoiyL79C8JDW2tvOqjdwVaE3Hps=">AAAB6nicbZDNSgMxFIXv+FvrX9WlIsEiuCozLtRl0Y3LFhxbaIeSSTNtaJIZkoxQhi7dunGh4taH6HO48xl8CTNtF9p6IPBxzr3k3hsmnGnjul/O0vLK6tp6YaO4ubW9s1va27/XcaoI9UnMY9UMsaacSeobZjhtJopiEXLaCAc3ed54oEqzWN6ZYUIDgXuSRYxgk1vtRLNOqexW3InQIngzKFePxvXvx+NxrVP6bHdjkgoqDeFY65bnJibIsDKMcDoqtlNNE0wGuEdbFiUWVAfZZNYROrVOF0Wxsk8aNHF/d2RYaD0Uoa0U2PT1fJab/2Wt1ERXQcZkkhoqyfSjKOXIxChfHHWZosTwoQVMFLOzItLHChNjz1O0R/DmV14E/7xyUXHrXrl6DVMV4BBO4Aw8uIQq3EINfCDQhyd4gVdHOM/Om/M+LV1yZj0H8EfOxw95uZHC</latexit>
2 L2(R2,C) andRR2 (u)du = 0
<latexit sha1_base64="mrzq5xaVNdN46w+JQS8GOBQT3ZU=">AAACQnicbVBBaxNBFH7baptGW9N69DI0CAmUsJtD7aUQmosHD1GaJpiNYXZ2kgydnV1m3paGZaE/zYt/wJt49+KhivTmwdkkQkx8MPB933vfzJsvSKQw6LpfnK3tR493dkt75SdP9w+eVQ6Prkycasa7LJax7gfUcCkU76JAyfuJ5jQKJO8F1+2i37vh2ohYXeIs4cOITpQYC0bRSqPKez8xgvhCET+iOGVUZm/yD83anAVB9s6Sk7+kndd95LeYEapCklsXjrLVyZwU19XSepieu6NK1W248yKbwFuCaqv+8PUOADqjymc/jFkacYVMUmMGnpvgMKMaBZM8L/up4Qll13TCBxYqGnEzzOYZ5OSlVUIyjrU9CslcXXVkNDJmFgV2sljYrPcK8X+9QYrjs2EmVJIiV2zx0DiVBGNSBEpCoTlDObOAMi3sroRNqaYMbexlG4K3/uVN0G02ThvuW6/auoBFleAFHEMNPHgFLXgNHegCg4/wDe7hh/PJ+e78dH4tRrecpec5/FPO7z/pLLQC</latexit><latexit sha1_base64="hbb0Ca2ujpjCwTZDix9084r07Wg=">AAACQnicbVC7TsMwFHV4U14FRhaLCqmVUJUwAAtSBQsDQ0GUIppSOY4LVh0nsm8QVZSNP2JkYeEH2BA7CwMgxMaA04LE60qWzjn3Hvv6eJHgGmz7zhoYHBoeGR0bz01MTk3P5GfnDnQYK8pqNBShOvSIZoJLVgMOgh1GipHAE6zudbayfv2MKc1DuQ/diDUDciJ5m1MCRmrlj9xIc+xyid2AwCklItlJj1eKPeZ5yZ4hy19kKy25wM4hwUT6ODUuaCXfJ1OcXVeMS368YbfyBbts9wr/Bc4nKFRKb/dnl1cX1Vb+1vVDGgdMAhVE64ZjR9BMiAJOBUtzbqxZRGiHnLCGgZIETDeTXgYpXjKKj9uhMkcC7qnfHQkJtO4GnpnMFta/e5n4X68RQ3u9mXAZxcAk7T/UjgWGEGeBYp8rRkF0DSBUcbMrpqdEEQom9pwJwfn95b+gtlJeLdu7TqGyifo1hhbQIioiB62hCtpGVVRDFF2jB/SEnq0b69F6sV77owPWp2ce/Sjr/QNzfLXq</latexit>
j,✓ =1
22j (
r✓(u)
2j)
<latexit sha1_base64="4yWyX8g1ftSjdJSEzIePbvZr7Ik=">AAACJXicbVDLSsNAFJ34rPVVdekmWIQWpCRF1E2h6MZlBWsLTQ2T6aSddvJg5kYoIV/jxl9x46KK4MpfcdJkoa0HBg7nnMude5yQMwmG8aWtrK6tb2wWtorbO7t7+6WDwwcZRILQNgl4ILoOlpQzn7aBAafdUFDsOZx2nMlN6neeqJAs8O9hGtK+h4c+cxnBoCS71LBCyex4fGbBiAJOGpYrMInNJK4/xvVxkqR+JROFnYUqUTW1x0nVLpWNmjGHvkzMnJRRjpZdmlmDgEQe9YFwLGXPNELox1gAI5wmRSuSNMRkgoe0p6iPPSr78fzMRD9VykB3A6GeD/pc/T0RY0/KqeeopIdhJBe9VPzP60XgXvVj5ocRUJ9ki9yI6xDoaWf6gAlKgE8VwUQw9VedjLCqBFSzRVWCuXjyMmnXaxc14+683LzO2yigY3SCKshEl6iJblELtRFBz+gVzdC79qK9aR/aZxZd0fKZI/QH2vcP94GmQg==</latexit><latexit sha1_base64="4yWyX8g1ftSjdJSEzIePbvZr7Ik=">AAACJXicbVDLSsNAFJ34rPVVdekmWIQWpCRF1E2h6MZlBWsLTQ2T6aSddvJg5kYoIV/jxl9x46KK4MpfcdJkoa0HBg7nnMude5yQMwmG8aWtrK6tb2wWtorbO7t7+6WDwwcZRILQNgl4ILoOlpQzn7aBAafdUFDsOZx2nMlN6neeqJAs8O9hGtK+h4c+cxnBoCS71LBCyex4fGbBiAJOGpYrMInNJK4/xvVxkqR+JROFnYUqUTW1x0nVLpWNmjGHvkzMnJRRjpZdmlmDgEQe9YFwLGXPNELox1gAI5wmRSuSNMRkgoe0p6iPPSr78fzMRD9VykB3A6GeD/pc/T0RY0/KqeeopIdhJBe9VPzP60XgXvVj5ocRUJ9ki9yI6xDoaWf6gAlKgE8VwUQw9VedjLCqBFSzRVWCuXjyMmnXaxc14+683LzO2yigY3SCKshEl6iJblELtRFBz+gVzdC79qK9aR/aZxZd0fKZI/QH2vcP94GmQg==</latexit>
j,✓<latexit sha1_base64="h0os0qPCCwGOdy3Iax5X9vow0TE=">AAAB93icbVBNS8NAEN34WetHox69LBbBg5REinosePFYwdhCE8Jmu2nXbjZhdyLU0F/ixYOKV/+KN/+N2zYHbX0w8Hhvhpl5USa4Bsf5tlZW19Y3Nitb1e2d3b2avX9wr9NcUebRVKSqGxHNBJfMAw6CdTPFSBIJ1olG11O/88iU5qm8g3HGgoQMJI85JWCk0K75meZh8XDmw5ABmYR23Wk4M+Bl4pakjkq0Q/vL76c0T5gEKojWPdfJICiIAk4Fm1T9XLOM0BEZsJ6hkiRMB8Xs8Ak+MUofx6kyJQHP1N8TBUm0HieR6UwIDPWiNxX/83o5xFdBwWWWA5N0vijOBYYUT1PAfa4YBTE2hFDFza2YDokiFExWVROCu/jyMvHOGxcN57ZZbzXLNCroCB2jU+SiS9RCN6iNPERRjp7RK3qznqwX6936mLeuWOXMIfoD6/MHXVOTCg==</latexit><latexit sha1_base64="h0os0qPCCwGOdy3Iax5X9vow0TE=">AAAB93icbVBNS8NAEN34WetHox69LBbBg5REinosePFYwdhCE8Jmu2nXbjZhdyLU0F/ixYOKV/+KN/+N2zYHbX0w8Hhvhpl5USa4Bsf5tlZW19Y3Nitb1e2d3b2avX9wr9NcUebRVKSqGxHNBJfMAw6CdTPFSBIJ1olG11O/88iU5qm8g3HGgoQMJI85JWCk0K75meZh8XDmw5ABmYR23Wk4M+Bl4pakjkq0Q/vL76c0T5gEKojWPdfJICiIAk4Fm1T9XLOM0BEZsJ6hkiRMB8Xs8Ak+MUofx6kyJQHP1N8TBUm0HieR6UwIDPWiNxX/83o5xFdBwWWWA5N0vijOBYYUT1PAfa4YBTE2hFDFza2YDokiFExWVROCu/jyMvHOGxcN57ZZbzXLNCroCB2jU+SiS9RCN6iNPERRjp7RK3qznqwX6936mLeuWOXMIfoD6/MHXVOTCg==</latexit>
Isotropic| |
Non-Isotropic| |VS
Ref.: ImageNet Classification with Deep Convolutional Neural Networks.
A Krizhevsky et al.
DATA
The Gabor wavelet
14
�(u) =1
2⇡�e�
kuk22� (u) =
1
2⇡�e�
kuk22� (ei⇠.u � )
(for sake of simplicity, formula are given in the isotropic case)
DATA
Wavelet Transform• Wavelet transform :
• Isometric and linear operator, with
• Covariant with translation
• Nearly commutes with the action of diffeomorphism
• Why wavelets are not enough? Invariance…
15
Wx = {x ? j,✓, x ? �J}✓,jJ
kWxk2 =X
✓,jJ
Z|x ? j,✓|2 +
Zx ? �
2J
k[W, .⌧ ]k Ckr⌧k
!1
!2
W (x⌧=c) = (Wx)⌧=c
Ref.: Group Invariant Scattering, Mallat S
DATAFilter bank implementation
of a Fast WT• Assume it is possible to find and such that
• Set:
• The WT is then given by
• A WT can be interpreted as a deep cascade of linear operator, which is approximatively verified for the Gabor Wavelets.
16
�(!) =1p2h(
!
2)�(
!
2) ✓(!) =
1p2g✓(
!
2)�(
!
2)
Ref.: Fast WT, Mallat S, 89
h g
and
xj(u, 0) = x ? �j(u) = h ? (x ? �j�1)(2u)
xj(u, ✓) = x ? j,✓(u) = g✓ ? (x ? �j�1)(2u)
and
Wx = {xj(., ✓), xJ(., 0)}jJ,✓
DATA
Deep implementation of a WT
17
h
g✓ g✓
h
g✓
h
There is an oversampling
x0
x1
x2
x3
h � 0
�j =1p2h(
.
2)�j�1
j,✓ =1p2g✓(
.
2)�j�1
DATA
Scattering Transform18
• Scattering transform at scale J is the cascading of complex WT with modulus non-linearity, followed by a low pass-filtering:
• Mathematically well defined for a large class of wavelets.
with �i = {ji, ✓i}, ji J<latexit sha1_base64="UmL5ObII+VZe+lGikFghFEB1KH4=">AAACDXicbVA9SwNBEJ3z2/gVtdNmMQoWEu4sVAQhYCNWEYwRcuHY20zM6t6Hu3NCOFJa2fhXbCxUbO3t/DduEgu/Hiy8eW+G2XlhqqQh1/1wRkbHxicmp6YLM7Nz8wvFxaUzk2RaYE0kKtHnITeoZIw1kqTwPNXIo1BhPbw67Pv1G9RGJvEpdVNsRvwilm0pOFkpKG74yja3eCAP/PwykFs+dZBs6fe2mK19hdfsOCiW3LI7APtLvC9SqqzcroNFNSi++61EZBHGJBQ3puG5KTVzrkkKhb2CnxlMubjiF9iwNOYRmmY+OKfHNqzSYu1E2xcTG6jfJ3IeGdONQtsZceqY315f/M9rZNTea+YyTjPCWAwXtTPFKGH9bFhLahSkupZwoaX9KxMdrrkgm2DBhuD9PvkvqW2Xd8ruiVeq7MMQU7AKa7AJHuxCBY6gCjUQcAcP8ATPzr3z6Lw4r8PWEedrZhl+wHn7BCTZnHU=</latexit><latexit sha1_base64="XZvxeIR+93C6VoTUkYgk/eMNrAA=">AAACDXicbVC7SgNBFJ31bXxF7bQZXAQLCbsWKoIQsBFBiGCMkF2W2cmNGZ19OHNXCEuwsrLxV2wsVNLa2/kj1k4eha8DA+eecy937glTKTQ6zoc1Mjo2PjE5NV2YmZ2bXyguLp3pJFMcqjyRiToPmQYpYqiiQAnnqQIWhRJq4dVBz6/dgNIiiU+xnYIfsYtYNAVnaKSguO5J09xggdj38stAbHrYAjSl19mkpvYkXNOjoGg7JacP+pe4Q2KXV+7sz9vjbiUovnuNhGcRxMgl07ruOin6OVMouIROwcs0pIxfsQuoGxqzCLSf98/p0HWjNGgzUebFSPvq94mcRVq3o9B0Rgxb+rfXE//z6hk2d/1cxGmGEPPBomYmKSa0lw1tCAUcZdsQxpUwf6W8xRTjaBIsmBDc3yf/JdWt0nbJOXHt8h4ZYIqskjWyQVyyQ8rkkFRIlXByTx7JM3mxHqwn69XqDlpHrOHMMvkB6+0Ledue9A==</latexit><latexit sha1_base64="XZvxeIR+93C6VoTUkYgk/eMNrAA=">AAACDXicbVC7SgNBFJ31bXxF7bQZXAQLCbsWKoIQsBFBiGCMkF2W2cmNGZ19OHNXCEuwsrLxV2wsVNLa2/kj1k4eha8DA+eecy937glTKTQ6zoc1Mjo2PjE5NV2YmZ2bXyguLp3pJFMcqjyRiToPmQYpYqiiQAnnqQIWhRJq4dVBz6/dgNIiiU+xnYIfsYtYNAVnaKSguO5J09xggdj38stAbHrYAjSl19mkpvYkXNOjoGg7JacP+pe4Q2KXV+7sz9vjbiUovnuNhGcRxMgl07ruOin6OVMouIROwcs0pIxfsQuoGxqzCLSf98/p0HWjNGgzUebFSPvq94mcRVq3o9B0Rgxb+rfXE//z6hk2d/1cxGmGEPPBomYmKSa0lw1tCAUcZdsQxpUwf6W8xRTjaBIsmBDc3yf/JdWt0nbJOXHt8h4ZYIqskjWyQVyyQ8rkkFRIlXByTx7JM3mxHqwn69XqDlpHrOHMMvkB6+0Ledue9A==</latexit><latexit sha1_base64="KBet0gCnjjeHoNqWcqCFkvXlYrc=">AAAB43icbVBNS8NAEJ34WeNX9eplsQieSuJBPQpePFYwttCGstls2qWb3bA7EUroH/DgRfHqf/Lmv3H7AWrrg4HHezPMzEsKKSwGwZe3tr6xubVd2/F39/z9g8P60aPVpWE8Ylpq00mo5VIoHqFAyTuF4TRPJG8no9up337ixgqtHnBc8DinAyUywSg6qdWvN4JmMANZJeGCNGCBfv2zl2pW5lwhk9TabhgUGFfUoGCST/xeaXlB2YgOeNdRRXNu42p25oScOSUlmTauFJKZ+nuiorm14zxxnTnFoV32puJ/XrfE7DquhCpK5IrNF2WlJKjJ9GeSCsMZyrEjlBnhbiVsSA1l6JLxXQbh8serJLpoXjaD+58soAYncArnEMIV3MAdtCACBik8w6s39F68N+993rjmLSaO4Q+8j2+GborS</latexit><latexit sha1_base64="gWxPYxqNM5VZOkDkIViilIi+kJI=">AAACDXicbVA9SwNBEJ2LXzF+RS1tFiVgEcLFQm0EwUasFIwKuXDs7c2ZNXsf7s4J4cgvsPGv2Fio2Nrb+W/cxICfDxbevDfD7LwgU9KQ6747pYnJqemZ8mxlbn5hcam6vHJm0lwLbIlUpfoi4AaVTLBFkhReZBp5HCg8D3oHQ//8BrWRaXJK/Qw7Mb9MZCQFJyv51ZqnbHPIfbnnFVe+rHvURbKlN6gzW3sKr9mRX91wG+4I7C9pjskGjHHsV9+8MBV5jAkJxY1pN92MOgXXJIXCQcXLDWZc9Pglti1NeIymU4zOGbCaVUIWpdq+hNhI/T5R8NiYfhzYzphT1/z2huJ/XjunaLdTyCTLCRPxuSjKFaOUDbNhodQoSPUt4UJL+1cmulxzQTbBig2h+fvkv6S11dhuuCdfYUAZ1mAdNqEJO7APh3AMLRBwC/fwCE/OnfPgPDsvn60lZzyzCj/gvH4Aofia9w==</latexit><latexit sha1_base64="gWxPYxqNM5VZOkDkIViilIi+kJI=">AAACDXicbVA9SwNBEJ2LXzF+RS1tFiVgEcLFQm0EwUasFIwKuXDs7c2ZNXsf7s4J4cgvsPGv2Fio2Nrb+W/cxICfDxbevDfD7LwgU9KQ6747pYnJqemZ8mxlbn5hcam6vHJm0lwLbIlUpfoi4AaVTLBFkhReZBp5HCg8D3oHQ//8BrWRaXJK/Qw7Mb9MZCQFJyv51ZqnbHPIfbnnFVe+rHvURbKlN6gzW3sKr9mRX91wG+4I7C9pjskGjHHsV9+8MBV5jAkJxY1pN92MOgXXJIXCQcXLDWZc9Pglti1NeIymU4zOGbCaVUIWpdq+hNhI/T5R8NiYfhzYzphT1/z2huJ/XjunaLdTyCTLCRPxuSjKFaOUDbNhodQoSPUt4UJL+1cmulxzQTbBig2h+fvkv6S11dhuuCdfYUAZ1mAdNqEJO7APh3AMLRBwC/fwCE/OnfPgPDsvn60lZzyzCj/gvH4Aofia9w==</latexit><latexit sha1_base64="jGrKMMN/j+MNpd5niMDb/Px4PNI=">AAACDXicbVC7SgNBFJ31GeMramkzGAIWIWwsVAQhYCNWEVwTyC7L7OQmGTP7cOauEJZ8gY2/YmOhYmtv5984eRSaeGDg3HPu5c49QSKFRtv+thYWl5ZXVnNr+fWNza3tws7urY5TxcHhsYxVM2AapIjAQYESmokCFgYSGkH/YuQ3HkBpEUc3OEjAC1k3Eh3BGRrJL5RcaZrbzBfnbnbni7KLPUBTusMyNbUr4Z5e+YWiXbHHoPOkOiVFMkXdL3y57ZinIUTIJdO6VbUT9DKmUHAJw7ybakgY77MutAyNWAjay8bnDGnJKG3aiZV5EdKx+nsiY6HWgzAwnSHDnp71RuJ/XivFzqmXiShJESI+WdRJJcWYjrKhbaGAoxwYwrgS5q+U95hiHE2CeRNCdfbkeeIcVY4r9rVdrJ1N08iRfXJADkmVnJAauSR14hBOHskzeSVv1pP1Yr1bH5PWBWs6s0f+wPr8Acbqm3E=</latexit><latexit sha1_base64="BSW+/Z7XFttMFpRW/beCc1nU/m8=">AAACDXicbVC7SgNBFJ31GeNr1dJmMAQsQti1UBGEgI1YRTAmkA3L7OQmGTP7cOauEJZ8gY2/YmOhYmtv5984eRSaeGDg3HPu5c49QSKFRsf5thYWl5ZXVnNr+fWNza1te2f3Vsep4lDjsYxVI2AapIighgIlNBIFLAwk1IP+xcivP4DSIo5ucJBAK2TdSHQEZ2gk3y560jS3mS/OvezOFyUPe4Cm9IYlampPwj298u2CU3bGoPPEnZICmaLq219eO+ZpCBFyybRuuk6CrYwpFFzCMO+lGhLG+6wLTUMjFoJuZeNzhrRolDbtxMq8COlY/T2RsVDrQRiYzpBhT896I/E/r5li57SViShJESI+WdRJJcWYjrKhbaGAoxwYwrgS5q+U95hiHE2CeROCO3vyPKkdlY/LzrVbqJxN08iRfXJADolLTkiFXJIqqRFOHskzeSVv1pP1Yr1bH5PWBWs6s0f+wPr8Acc6m3I=</latexit><latexit sha1_base64="XZvxeIR+93C6VoTUkYgk/eMNrAA=">AAACDXicbVC7SgNBFJ31bXxF7bQZXAQLCbsWKoIQsBFBiGCMkF2W2cmNGZ19OHNXCEuwsrLxV2wsVNLa2/kj1k4eha8DA+eecy937glTKTQ6zoc1Mjo2PjE5NV2YmZ2bXyguLp3pJFMcqjyRiToPmQYpYqiiQAnnqQIWhRJq4dVBz6/dgNIiiU+xnYIfsYtYNAVnaKSguO5J09xggdj38stAbHrYAjSl19mkpvYkXNOjoGg7JacP+pe4Q2KXV+7sz9vjbiUovnuNhGcRxMgl07ruOin6OVMouIROwcs0pIxfsQuoGxqzCLSf98/p0HWjNGgzUebFSPvq94mcRVq3o9B0Rgxb+rfXE//z6hk2d/1cxGmGEPPBomYmKSa0lw1tCAUcZdsQxpUwf6W8xRTjaBIsmBDc3yf/JdWt0nbJOXHt8h4ZYIqskjWyQVyyQ8rkkFRIlXByTx7JM3mxHqwn69XqDlpHrOHMMvkB6+0Ledue9A==</latexit><latexit sha1_base64="XZvxeIR+93C6VoTUkYgk/eMNrAA=">AAACDXicbVC7SgNBFJ31bXxF7bQZXAQLCbsWKoIQsBFBiGCMkF2W2cmNGZ19OHNXCEuwsrLxV2wsVNLa2/kj1k4eha8DA+eecy937glTKTQ6zoc1Mjo2PjE5NV2YmZ2bXyguLp3pJFMcqjyRiToPmQYpYqiiQAnnqQIWhRJq4dVBz6/dgNIiiU+xnYIfsYtYNAVnaKSguO5J09xggdj38stAbHrYAjSl19mkpvYkXNOjoGg7JacP+pe4Q2KXV+7sz9vjbiUovnuNhGcRxMgl07ruOin6OVMouIROwcs0pIxfsQuoGxqzCLSf98/p0HWjNGgzUebFSPvq94mcRVq3o9B0Rgxb+rfXE//z6hk2d/1cxGmGEPPBomYmKSa0lw1tCAUcZdsQxpUwf6W8xRTjaBIsmBDc3yf/JdWt0nbJOXHt8h4ZYIqskjWyQVyyQ8rkkFRIlXByTx7JM3mxHqwn69XqDlpHrOHMMvkB6+0Ledue9A==</latexit><latexit sha1_base64="XZvxeIR+93C6VoTUkYgk/eMNrAA=">AAACDXicbVC7SgNBFJ31bXxF7bQZXAQLCbsWKoIQsBFBiGCMkF2W2cmNGZ19OHNXCEuwsrLxV2wsVNLa2/kj1k4eha8DA+eecy937glTKTQ6zoc1Mjo2PjE5NV2YmZ2bXyguLp3pJFMcqjyRiToPmQYpYqiiQAnnqQIWhRJq4dVBz6/dgNIiiU+xnYIfsYtYNAVnaKSguO5J09xggdj38stAbHrYAjSl19mkpvYkXNOjoGg7JacP+pe4Q2KXV+7sz9vjbiUovnuNhGcRxMgl07ruOin6OVMouIROwcs0pIxfsQuoGxqzCLSf98/p0HWjNGgzUebFSPvq94mcRVq3o9B0Rgxb+rfXE//z6hk2d/1cxGmGEPPBomYmKSa0lw1tCAUcZdsQxpUwf6W8xRTjaBIsmBDc3yf/JdWt0nbJOXHt8h4ZYIqskjWyQVyyQ8rkkFRIlXByTx7JM3mxHqwn69XqDlpHrOHMMvkB6+0Ledue9A==</latexit>
Ref.: Group Invariant Scattering, Mallat S
SJx = {x ? �J ,|x ? �1 | ? �J ,||x ? �1 | ? �2 | ? �J}
x�J �
|.|�J �
|.|�J
Depth
DATA
Relations with SIFT
19
SIFT performs a histogram of gradient
h(✓) =X
\g2[✓,✓+⌘]
kgk
=X
g
k1\g2[✓,✓+⌘]gk
⇡|x ? ✓| ? �
The averaging leads to a loss of information…
Quantification…
Gradient
For people into computer vision:
DATAFeature map
2020
x
?
�
<latexit
sha1_ba
se64="oP
27VGR2X2
TA/4MKMQ
73CW5eNz
I=">AAAB
8XicbZC7
SgNBFIbP
xluMt6il
FoNBsAq7
FmoZtLFM
wDXC7hJm
J7PJkNkL
M2fFsOQx
bCxUbH0B
n8POzkdx
cik08YeB
j/8/hznn
hJkUGm37
yyotLa+s
rpXXKxub
W9s71d29
W53minGX
pTJVdyHV
XIqEuyhQ
8rtMcRqH
krfDwdU4
b99zpUWa
3OAw40FM
e4mIBKNo
LO/B10gV
8bO+6FRr
dt2eiCyC
M4Na4/Cj
9Q0AzU71
0++mLI95
gkxSrT3H
zjAoqELB
JB9V/Fzz
jLIB7XHP
YEJjroNi
MvKIHBun
S6JUmZcg
mbi/Owoa
az2MQ1MZ
U+zr+Wxs
/pd5OUYX
QSGSLEee
sOlHUS4J
pmS8P+kK
xRnKoQHK
lDCzEtan
ijI0V6qY
IzjzKy+C
e1o/q9st
p9a4hKnK
cABHcAIO
nEMDrqEJ
LjBI4RGe
4cVC68l6
td6mpSVr
1rMPf2S9
/wCvvJNF
</latexi
t>
<latexit
sha1_ba
se64="B6
t5L4P5Wm
CHNzoLV7
UGUqh4NK
s=">AAAB
8XicbZDL
SsNAFIYn
9VbrrepS
kcEiuCqJ
C3VZdOOy
BWMLSSiT
6aQdOpmE
mROxhC59
BDcuVNz6
An0Odz6D
L+H0stDW
HwY+/v8c
5pwTpoJr
sO0vq7C0
vLK6Vlwv
bWxube+U
d/fudJIp
ylyaiES1
QqKZ4JK5
wEGwVqoY
iUPBmmH/
epw375nS
PJG3MEhZ
EJOu5BGn
BIzlPfga
iMJ+2uPt
csWu2hPh
RXBmUKkd
jhrfj0ej
erv86XcS
msVMAhVE
a8+xUwhy
ooBTwYYl
P9MsJbRP
uswzKEnM
dJBPRh7i
E+N0cJQo
8yTgifu7
Iyex1oM4
NJUxgZ6e
z8bmf5mX
QXQZ5Fym
GTBJpx9F
mcCQ4PH+
uMMVoyAG
BghV3MyK
aY8oQsFc
qWSO4Myv
vAjuWfW8
ajecSu0K
TVVEB+gY
nSIHXaAa
ukF15CKK
EvSEXtCr
Bdaz9Wa9
T0sL1qxn
H/2R9fED
jJuUqw==
</latexi
t>
x
<latexit sha1_base64="vBWJA9LY6QPbamOLKKoS3HETMkM=">AAAB53icbZA9SwNBEIbn4lc8v6KWNotBsAp3FmojBm0sE/BMIDnC3mYuWbP3we6eGI6AvY2Fiq2/xt7Of+MmsdDEFxYe3neGnZkgFVxpx/myCguLS8srxVV7bX1jc6u0vXOjkkwy9FgiEtkMqELBY/Q01wKbqUQaBQIbweBynDfuUCqexNd6mKIf0V7MQ86oNlb9vlMqOxVnIjIP7g+Uzz/sswcAqHVKn+1uwrIIY80EVarlOqn2cyo1ZwJHdjtTmFI2oD1sGYxphMrPJ4OOyIFxuiRMpHmxJhP3d0dOI6WGUWAqI6r7ajYbm/9lrUyHp37O4zTTGLPpR2EmiE7IeGvS5RKZFkMDlEluZiWsTyVl2tzGNkdwZ1eeB++oclxx6m65egFTFWEP9uEQXDiBKlxBDTxggPAIz/Bi3VpP1qv1Ni0tWD89u/BH1vs3v8yOlg==</latexit><latexit sha1_base64="A9ccdmxJ8Bhrzemvz5zk/mcv4S8=">AAAB53icbZC7SgNBFIbPxltcb1FLm8EgWIVdC7URgzaWCbgmkIQwOzmbjJm9MDMrhiVPYGOhYis+jL2N+DZOLoUm/jDw8f/nMOccPxFcacf5tnILi0vLK/lVe219Y3OrsL1zo+JUMvRYLGJZ96lCwSP0NNcC64lEGvoCa37/cpTX7lAqHkfXepBgK6TdiAecUW2s6n27UHRKzlhkHtwpFM8/7LPk/cuutAufzU7M0hAjzQRVquE6iW5lVGrOBA7tZqowoaxPu9gwGNEQVSsbDzokB8bpkCCW5kWajN3fHRkNlRqEvqkMqe6p2Wxk/pc1Uh2ctjIeJanGiE0+ClJBdExGW5MOl8i0GBigTHIzK2E9KinT5ja2OYI7u/I8eEel45JTdYvlC5goD3uwD4fgwgmU4Qoq4AEDhAd4gmfr1nq0XqzXSWnOmvbswh9Zbz+v55AK</latexit>
1st order coefficients
Example of Scattering coefficients|x
?
|?
�
<latexit sha1_base64="UKn0LYwiDnxBHx5LbGPech2pnps=">AAAB/3icbZDLSsNAFIZPvNZ6i7pw4WZoEQShJC7UZdGNywrGFppQJtNJO3QyCTMTsaTd+CSCGxcqbn0Nd76N04ugrT8MfPznHM6cP0w5U9pxvqyFxaXlldXCWnF9Y3Nr297ZvVVJJgn1SMIT2QixopwJ6mmmOW2kkuI45LQe9i5H9fodlYol4kb3UxrEuCNYxAjWxmrZ+4N7X2kskZ8qNvjBLmvZZafijIXmwZ1CuVryjx8BoNayP/12QrKYCk04VqrpOqkOciw1I5wOi36maIpJD3do06DAMVVBPj5giA6N00ZRIs0TGo3d3xM5jpXqx6HpjLHuqtnayPyv1sx0dB7kTKSZpoJMFkUZRzpBozRQm0lKNO8bwEQy81dEulhiok1mRROCO3vyPHgnldOKc+2WqxcwUQEOoARH4MIZVOEKauABgSE8wQu8Wg/Ws/VmvU9aF6zpzB78kfXxDYhUmAg=</latexit>
<latexit sha1_base64="DAbp+PLfwEoNH5yLdJzXJ4CKPkE=">AAAB/3icbZDLSsNAFIYn9VbrLerChZuhRRCEkrhQl0E3LisYW2hCmUwn7dDJhZkTMbTd+Ay+gRsXKm59DXd9G6cXQVt/GPj4zzmcOX+QCq7AskZGYWl5ZXWtuF7a2Nza3jF39+5UkknKXJqIRDYCopjgMXOBg2CNVDISBYLVg97VuF6/Z1LxJL6FPGV+RDoxDzkloK2WeTB48BQQib1U8cEPdnnLrFhVayK8CPYMKk7ZO3kaOXmtZX557YRmEYuBCqJU07ZS8PtEAqeCDUteplhKaI90WFNjTCKm/P7kgCE+0k4bh4nULwY8cX9P9EmkVB4FujMi0FXztbH5X62ZQXjh93mcZsBiOl0UZgJDgsdp4DaXjILINRAquf4rpl0iCQWdWUmHYM+fvAjuafWsat3YFecSTVVEh6iMjpGNzpGDrlENuYiiIXpGr+jNeDRejHfjY9paMGYz++iPjM9vkDiZjg==</latexit>
DATA
Properties• Non-linear
• Isometric
• Stable to noise
• Covariant with translation
• Invariant to small translation
• Sensitive to the action of rotation
• Linearize the action of small deformation
• Reconstruction properties
21
SJ(x⌧=c) = SJ(x)⌧=c
|c| 2J ) SJ(x⌧=c) ⇡ SJ(x)
SJ(r✓x) 6= SJ(x)
kSJx⌧ � SJxk Ckr⌧k
allow a linear classifier to build class invariant
on informative variability
Ref.: Group Invariant Scattering, Mallat S
Ref.: Reconstruction of images scattering coefficient. Bruna J
kSJxk = kxk
kSJx� SJyk kx� yk
DATAParameters&dimensionality
of the Scattering• For 256x256 images,
• Yet it is possible to reduce it up to
• It depends on a few parameters , the shape of the mother wavelet…
• Identical parameters can be chosen for natural images on different datasets.
22
�x 2 R105
L�x 2 R2000
L, J
It is a generic representation
DATA
2nd order Translation Scattering
23
hg✓
hh
g✓ g✓
J = 3, ✓ 2 {0, ⇡4,⇡
2,3⇡
4}
x0 x1
x2
x3
Order 2Order 1Order 0
Modulus
g✓ h
hg✓
h � 0 Scattering coefficients are only at the output
DATAAffine Scattering Transform
• Observe that |W| is covariant with the affine group Aff(E):
• See |W| as a signal parametrised by some elements of the affine group:
• We can define a WT on any compact Lie group(even not commutative) via:
• The same previous properties hold for this WT/Scattering.
24
|W |[�](g.x) = |(g.x) ? �| = |W |[g.�]x
|x ? j,✓(u)| = x(g), g = (u, r✓, j)
x ?
G (g) =
Z
G (g0)x(g0�1.g)dg0
Ref.: Topics in harmonic analysis related to the Littlewood-Paley theory
Stein EM
Ref.: Group Invariant Scattering, Mallat S
Ref.: PhD, L Sifre
DATA
Separable Roto-Translation Scattering
• Roto-translation group is not separable yet we used a separable wavelet transform on it.
• No averaging along angle (sensitivity increased)
• Separable (simple to implement and fast)
• Equal (slightly better) results as with non separable
25
Ref.: Deep Roto-Translation Scattering for Object Classification. EO and S Mallat
DATA
Separable Roto-Translation scattering
2626
g✓ h
hg✓
hg✓
hh
g✓ g✓
J = 3, ✓ 2 {0, ⇡4,⇡
2,3⇡
4}
x0 x1
x2
x3
Separable convolution that recombines angles on 2nd order
Ref.: Deep Roto-Translation Scattering for Object Classification. EO and S Mallat
DATALinearization of the rotation
• For a deformation:
• In fact the the affine group acts also on the deformation diffeomorphism:
• Decompose it on the affine group:
• A way to linearize the action of the rotation?
27
Work in progress
u� ⌧(u) ⇡ v � ⌧(v) + (I �r⌧)(v)(u� v)
kSx� Sx⌧k C(kr⌧✓k+ ...)
g.(I � ⌧)(u) ⇡ g.(I � ⌧)(v) + g.(I �r⌧)(v)(u� v)
(I �r⌧)(v) = r⌧✓(v)K(v)
DATA
Classification pipeline
• We learn L(select features) and w(select samples) from the data
• Getting L is the most costly part of the algorithm…(Orthogonal Least Square)
28
x �! �x �! L�x �! hL�x,wi � 0
Scattering Transform on YUV
+Log
Supervised dimensionality
ReductionGaussian SVM
DATA
OLS?• Supervised forward selection of features. The
selection is done class per class. (similar to OMP)
• Principle: given a dictionary of feature
• Set
• Find the most correlated feature with
• Pull it from the dictionary, orthogonalize the dictionary and normalize the dictionary.
• Select it. Iterate.
29
{�k}k
y�k
yi = 1 if i is in the class, 0 otherwise
Ref.: On the difference between orthogonal matching pursuit and orthogonal least squares.
T. Blumensath and M. E. Davies.
DATA
Numerical results
30
Dataset Type Paper AccuracyCaltech101 Scattering 79.9
Unsupervised Ask the locals 77.3 RFL 75.3
M-HMP 82.5Supervised DeepNet 91.4
Caltech256 Scattering 43.6Unsupervised Ask the Locals 41.7
M-HMP 50.7Supervised DeepNet 70.6
CIFAR10 Scattering 82.3Unsupervised RFL 83.1
Supervised DeepNet 91.8CIFAR100 Scattering 56.8
Unsupervised RFL 54.2Supervised DeepNet 65.4 Identical
Representation
DATA
Improvement layer wise
31
Method Caltech101 CIFAR10
Translation Scattering order 1 59.8 72.6
Translation Scattering order 2 70.0 80.3
Translation Scattering order 2+ OLS 75.4 81.6
Roto-translation Scattering order 2 74.5 81.5
Roto-translation Scattering order 2+ OLS 79.9 82.3
DATA
How to fill in the Gap?• Adding more supervision in the pipeline
• Building a DeepNet on top of it? Initialising a DeepNet with wavelets filters? Question is opened.
• A more complex classifier could help to handle class variabilities. Fisher vector with scattering?
• Adding a layer means identifying the next complex source of variabilities. Are they geometric? classes?
32
DATA
Other application of ST
• Audio
• Quantum chemistry
• Temporal data, video
• Reconstruction of WT
• Unstructured data…
33
Vincent Lostanlen
Matthew Hirn
Irène Waldspurger
Mia Chen Xu Cheng
DATA
Conclusion
• We provide:
• Mathematical analysis and algorithms
• Competitive numerical results
• Software: http://www.di.ens.fr/~oyallon/ (or send me an email to get the latest!)
34
