[Colloquium] Rida Assaf Dissertation Defense/Apr 22, 2022

[Megan Woodward]

This is an announcement of Rida Assaf's Dissertation Defense.
===============================================
Candidate: Rida Assaf

Date: Friday, April 22, 2022

Time:  1 pm CST

Remote Location: https://uchicago.zoom.us/j/92653972005?pwd=akdzaE5aVkN5b3hOaFA4UHhEZWNjZz09 Meeting ID: 926 5397 2005 Passcode: 208290


Title: Feature Transformations to Enhance Representation Learning

Abstract: Machine learning (ML) has crowned itself as a breakthrough in a number of domains, such as computer vision and natural language processing, achieving and sometimes exceeding human level performance on certain tasks especially in supervised learning. A major factor driving these success stories is the effort undertaken in designing different artificial neural network architectures that are particularly equipped to handle specialized tasks. For example, convolutional neural networks (CNNs) were designed to leverage spatial locality and other properties assumed in images, whereas recurrent neural networks and attention mechanisms are used with sequential or time-series data. While the majority of datasets available may not exhibit a special structure and is presented in plain tabular form, the most commonly used neural network on tabular datasets is the multi-layer perceptron (MLP).

Similar to how choosing the right learner architecture may be seen as part of the training process that is essential for a good performance, we believe that some effort on the data presentation side could further improve the learning outcome. The underlying idea when using artificial neural networks is that representation learning happens automatically using raw data, unlike classical ML techniques that are still widely applied on tabular datasets, and that are often accompanied by feature selection and transformation strategies. We propose that certain feature transformations could enhance the representation learning process and the subsequent machine learning outcome achieved by artificial neural networks. The key principles behind our proposed transformations are highlighting the entities and relationships described by the data. These transformations can be a direct application of our domain knowledge, for example by manually designing visual representations of the features to be used by a two-dimensional CNN. Another approach is to partition the input feature vector such that each partition represents an entity or a relationship, to be used by a modular MLP (an MLP with multiple input layers). We provide empirical evidence suggesting that such transformations yield better results than baseline MLPs, and require less time, data, and parameters. In cases where domain knowledge is lacking or no clear feature groups are known, the transformation process works by generating a permutation of the feature vector where related features are neighbors, to be used with a one-dimensional CNN to capture the implicit feature groups.

We propose a method to automate the transformation process, and evaluate it empirically using synthetic and real-world datasets. The synthetic datasets are designed in a way that allows different levels of representation of the underlying entities and relationships in the data, providing additional insight into the learning process. The real-world datasets are derived from experiments reported by other approaches that propose automatic feature transformation techniques to enhance deep learning performance. Our results show a clear advantage over these approaches, not only in the learning outcome but also in the simplicity of method.

Advisors: Rick Stevens

Committee Members: Ian Foster, Rick Stevens, and Fangfang Xia



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20220412/0b07ccaf/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Dissertation_draft.pdf
Type: application/pdf
Size: 9207147 bytes
Desc: Dissertation_draft.pdf
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20220412/0b07ccaf/attachment-0001.pdf>