[Theory] [TTIC Talks] 11/1 Research at TTIC: Jingyan Wang, TTIC

[Brandie Jones via Theory]

*When:*         November 1st* at 12:30pm CT  *


*Where:*        Talk will be given *live, in-person* at

                       TTIC, 6045 S. Kenwood Avenue

                        5th Floor, Room 530


*Virtually:*    via Panopto (Livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=e4e73f4e-ffcb-4639-b0e2-b1a901057968>
)



*Who:*           Jingyan Wang, TTIC



*Title*:           Learning from people: Data elicitation and bias
mitigation

*Abstract*:   Evaluation – to estimate the quality of items or people –
arises in many real-world applications such as admissions, grading, hiring,
and peer review. In this talk, I will discuss two directions of my research
towards understanding and improving evaluation problems.

(1) Bias mitigation: We consider bias mitigation in a sequential setting,
where an evaluator rates candidates in an online, irrevocable fashion. We
propose a natural model for the evaluator's rating process that captures
the lack of calibration inherent to this task, supported by the psychology
literature, theoretical results, and crowdsourcing experiments. Under this
model, we propose a near-linear online algorithm for bias correction and
show that it is information-theoretically optimal, outperforming the de
facto approach of using the ranking induced by the raw scores.

(2) Data elicitation: Human feedback often comes in the form of ratings or
rankings (such as pairwise comparisons, triplet comparisons, or total
orderings). We introduce a new type of query scheme termed the “perceptual
adjustment query” (PAQ), which combines advantages from both ratings and
rankings. Operationally, PAQ is implemented by having the user adjust a
sliding bar on a continuous spectrum. We showcase PAQ in a metric learning
problem, which gives rise to a new matrix estimation problem to which
standard estimators cannot be applied. Consequently, we develop a two-stage
estimator and provide its sample complexity guarantees, along with
simulation and a preliminary user study on a color perception task.

***********************************************************************************************

*Masks are optional in all common areas. **Full visitor guidance is
available at ttic.edu/visitors <http://ttic.edu/visitors>.*

***********************************************************************************************

*Research at TTIC Seminar Series*



TTIC is hosting a weekly seminar series presenting the research currently
underway at the Institute. Every week a different TTIC faculty member will
present their research.  The lectures are intended for students
seeking research topics and advisors and for the general TTIC and
University of Chicago communities interested in hearing what their
colleagues are up to.


--
*Brandie Jones *
*Executive **Administrative Assistant*
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20241028/95fe998b/attachment.html>