CUNY-NLP: Stanislas Lauly (NYU)

MAR 03, 2017 | 2:30 PM TO 4:00 PM



The Graduate Center
365 Fifth Avenue




March 03, 2017: 2:30 PM-4:00 PM




Stanislas Lauly (NYU) presents "Document Neural Autoregressive Distribution Estimation." All are welcome.

We present an approach based on feed-forward neural networks for learning the distribution of textual documents. This approach is inspired by the Neural Autoregressive Distribution Estimator (NADE) model, which has been shown to be a good estimator of the distribution of discrete-valued  high-dimensional vectors. We present how NADE can successfully be adapted to the case of textual data, retaining from NADE the property that sampling or computing the probability of observations can be done exactly and efficiently. The approach can also be used to learn deep representations of documents that are competitive to those learned by the alternative topic modeling approaches. Finally, we describe how the approach can be combined with a regular neural network N-gram model and substantially improve its performance, by making its learned representation sensitive to the larger, document-specific context.

For questions, contact the CUNY-NLP Talk organizer, Michelle Renee Morales: m​