[Colloquium] Re: REMINDER: 1/27 Research at TTIC: Liang Lu, TTIC

[Mary Marre via Colloquium]

When:     Friday, January 27th at noon



Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526



Who:       Liang Lu; TTIC



Title:        Small-footprint Highway Deep Neural Networks for Speech
Recognition

Abstract:
State-of-the-art speech recognition systems typically employ neural network
acoustic models. However, compared to Gaussian mixture models, deep neural
network (DNN) based acoustic models often have many more model parameters,
making it challenging for them to be deployed on resource-constrained
platforms, such as mobile devices. In this talk, I will present the
application of the recently proposed highway deep neural network (HDNN) for
training small-footprint acoustic models. HDNNs are a depth-gated
feedforward neural network, which include two types of gate functions to
facilitate the information flow through different layers. Our study
demonstrates that HDNNs are more compact than regular DNNs for acoustic
modeling, i.e., they can achieve comparable recognition accuracy with many
fewer model parameters. Furthermore, HDNNs are more controllable than DNNs:
the gate functions of an HDNN largely control the behavior of the whole
network using a very small number of model parameters. Finally, HDNNs are
more adaptable than DNNs. For example, simply updating the gate functions
using the adaptation data can result in considerable gains in accuracy.
Teacher-student training will also be studied in this context. I will
demonstrate
these aspects by experiments using the publicly available medium-size AMI
corpus, which has around 80 hours of training data (~28 million data
points).


************************************************************
****************************************************

*Research at TTIC Seminar Series*

TTIC is hosting a weekly seminar series presenting the research currently
underway at the Institute. Every week a different TTIC faculty member will
present their research.  The lectures are intended both for students
seeking research topics and adviser, and for the general TTIC and
University of Chicago communities interested in hearing what their
colleagues are up to.

To receive announcements about the seminar series, please subscribe to the
mailing list: https://groups.google.com/a/ttic.edu/group/talks/subscribe

Speaker details can be found at: http://www.ttic.edu/tticseminar.php.

For additional questions, please contact Nathan Srebro at nati at ttic.edu
<mcallester at ttic.edu>





Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 504*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*

On Thu, Jan 26, 2017 at 4:39 PM, Mary Marre <mmarre at ttic.edu> wrote:

> When:     Friday, January 27th at noon
>
>
>
> Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526
>
>
>
> Who:       Liang Lu; TTIC
>
>
>
> Title:        Small-footprint Highway Deep Neural Networks for Speech
> Recognition
>
> Abstract:
> State-of-the-art speech recognition systems typically employ neural
> network acoustic models. However, compared to Gaussian mixture models, deep
> neural network (DNN) based acoustic models often have many more model
> parameters, making it challenging for them to be deployed on
> resource-constrained platforms, such as mobile devices. In this talk, I
> will present the application of the recently proposed highway deep neural
> network (HDNN) for training small-footprint acoustic models. HDNNs are a
> depth-gated feedforward neural network, which include two types of gate
> functions to facilitate the information flow through different layers. Our
> study demonstrates that HDNNs are more compact than regular DNNs for
> acoustic modeling, i.e., they can achieve comparable recognition accuracy
> with many fewer model parameters. Furthermore, HDNNs are more controllable
> than DNNs: the gate functions of an HDNN largely control the behavior of
> the whole network using a very small number of model parameters. Finally,
> HDNNs are more adaptable than DNNs. For example, simply updating the gate
> functions using the adaptation data can result in considerable gains in
> accuracy. Teacher-student training will also be studied in this context. I
> will demonstrate these aspects by experiments using the publicly
> available medium-size AMI corpus, which has around 80 hours of training
> data (~28 million data points).
>
>
> ************************************************************
> ****************************************************
>
> *Research at TTIC Seminar Series*
>
> TTIC is hosting a weekly seminar series presenting the research currently
> underway at the Institute. Every week a different TTIC faculty member will
> present their research.  The lectures are intended both for students
> seeking research topics and adviser, and for the general TTIC and
> University of Chicago communities interested in hearing what their
> colleagues are up to.
>
> To receive announcements about the seminar series, please subscribe to the
> mailing list: https://groups.google.com/a/ttic.edu/group/talks/subscribe
>
> Speaker details can be found at: http://www.ttic.edu/tticseminar.php.
>
> For additional questions, please contact Nathan Srebro at nati at ttic.edu
> <mcallester at ttic.edu>
>
>
>
>
> Mary C. Marre
> Administrative Assistant
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 504*
> *Chicago, IL  60637*
> *p:(773) 834-1757 <(773)%20834-1757>*
> *f: (773) 357-6970 <(773)%20357-6970>*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20170127/5c1c73c6/attachment.html>