[Colloquium] [defense] Dziedzic/Dissertation Defense/Jul 17, 2020

[Margaret Jaffey]

Join Zoom Meeting
https://uchicagostudents.zoom.us/j/94077870948?pwd=NGh4K200RXJPY3JKaFNpanlJUENqZz09

Meeting ID: 940 7787 0948 Password: AdJuWe9804

       Department of Computer Science/The University of Chicago

                     *** Dissertation Defense ***


Candidate:  Adam Dziedzic

Date:  Friday, July 17, 2020

Time:  9:00 AM

Place:  remotely via Zoom

Title: Input and Model Compression for Adaptive and Robust Neural
Networks

Abstract:
The convolutional layers are core building blocks of neural network
architectures. In general, a convolutional filter applies to the
entire frequency spectrum of the input data. Our band-limiting method
artificially constrains the frequency spectra of these filters and
data during training and inference. The frequency-domain constraints
apply to both the feed-forward and back-propagation steps.
Experimental results confirm that Convolutional Neural Networks (CNNs)
are resilient to this compression scheme and show that CNNs learn to
leverage lower-frequency components. In particular, the band-limited
training can effectively control resource usage (GPU and memory). Our
method with 50% compression in the frequency domain results in only a
1.5% drop in accuracy for the ReNet-18 model trained on CIFAR-10 data
while reducing the GPU memory usage by 40% and the computation time by
30% in comparison to their full-spectra counterparts. The models
trained with band-limited layers retain high prediction accuracy and
require no modification to existing training algorithms or neural
network architectures, unlike other compression schemes. We apply the
band-limited models to develop a new solution for a fair wireless
co-existence between Wi-Fi access points and LTE-U (Long Term
Evolution-Unlicensed) base stations.

Band-limited models are also robust to noisy inputs and naturally
extend to perturbation-based defenses that improve robustness to
adversarial attacks. The input perturbation defenses, whether random
or deterministic, are equivalent in their efficacy. However, the
perturbation based defenses offer almost no robustness to adaptive
attacks unless these perturbations are observed during training and a
tuned sequence of noise layers is placed across a network. From the
theoretical perspective, adversarial examples in a close neighborhood
of original inputs show an elevated sensitivity to perturbations in
first and second-order analyses. From the application perspective, we
explore the out-of-distribution robustness of models in terms of their
generalization and detection of anomalous inputs in the NLP (Natural
Language Processing) domain.

Adam's advisor is Prof. Sanjay Krishnan

Login to the Computer Science Department website for details,
including a draft copy of the dissertation:

 https://newtraell.cs.uchicago.edu/phd/phd_announcements#ady

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Margaret P. Jaffey            margaret at cs.uchicago.edu
Department of Computer Science
Student Support Rep (JCL 350)              (773) 702-6011
The University of Chicago      http://www.cs.uchicago.edu
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=