Neural Evolution and Selection

Research Project at UMass Amherst
2020 - present

Modern neural networks usually have deep architectures with millions of parameters and connections. To make these models work well, extensive manual efforts in design and tuning are often required. Our work focus on exploring automated evolution of deep neural networks by extending the ideas of selection in genetic algorithms.


Evolving Neural Selection with Adaptive Regularization
Li Ding and Lee Spector
[GECCO 2021: Workshop on NeuroEvolution at Work] [paper] [presentation]

Cognitive Load Estimation

Research Project at MIT
Supported by Veoneer and MIT AHEAD Consortium
2018 - present

How do we measure the state of the human mind? Cognitive load has been shown to be an important variable for understanding human performance on a variety of tasks including public speaking, education, machine operation, and driving. Our research focuses on vision-based non-contact cognitive load estimation using machine learning approaches on visual features including pupil dynamics, glance behaviors, and facial expressions.


Joint Cognitive Load Estimation and Eye Landmarks Detection in the Wild
Li Ding, Jack Terwilliger, Aishni Parab, Meng Wang, Bruce Mehler, Bryan Reimer, and Lex Fridman
[Under Review]

Real-time Cognitive Load Estimation with Multi-source Features and Individual Normalization
Meng Wang, Li Ding, Pnina Gershon, Bruce Mehler, and Bryan Reimer
[Under Review]

Driving Scene Perception and Edge Case Enumeration

Research Project at MIT
Supported by Toyota
2017 - 2020

Solving the driving scene perception problem for driver-assistance systems and autonomous vehicles requires accurate and robust model performance in various driving scenarios, including those rarely-occurring edge cases. Aiming at developing a real-time perception system prototype, our research involves novel methods for video scene segmentation, large-scale driving data collection, semi-automated annotation, and edge case enumeration.


Perceptual Evaluation of Driving Scene Segmentation
Li Ding, Rini Sherony, Bruce Mehler, and Bryan Reimer
[IEEE IV 2021]

MIT-AVT Clustered Driving Scene Dataset: Evaluating Perception Systems in Real-World Naturalistic Driving Scenarios
Li Ding, Michael Glazer, Meng Wang, Bruce Mehler, Bryan Reimer, and Lex Fridman
[IEEE IV 2020: NDDA Workshop (oral presentation)] [paper]

Value of Temporal Dynamics Information in Driving Scene Segmentation
Li Ding, Jack Terwilliger, Rini Sherony, Bryan Reimer, and Lex Fridman
[IEEE Transactions on Intelligent Vehicles] [paper] [arXiv] [dataset]

Black Betty: MIT Human-Centered Autonomous Vehicle

Research Project at MIT
Supported by Toyota and Veoneer
2018 - 2019

The interaction between human and machine with growing intelligence challenges our assumptions about the limitations of human beings at their worst and the capabilities of AI systems at their best. We explore human-centered autonomous vehicle as an illustrative case study of concepts in shared autonomy. The project involves research on human machine collaboration, computer vision, and perception/control systems for semi-autonomous driving.


Project Page:

Arguing Machines: Human Supervision of Black Box AI Systems that Make Life-Critical Decisions
Lex Fridman, Li Ding, Benedikt Jenik, Bryan Reimer
[CVPR 2019: Workshop on Autonomous Driving] [paper] [arXiv]

Human Action Recognition in Untrimmed Videos

Research Project at the University of Rochester
Supported by NSF BIGDATA
2017 - 2018

One of the major challenges in video understanding is to localize and classify human actions in long, untrimmed videos. We proposed a new temporal model for action recognition and a novel iterative alignment approach to address the weakly supervised action localization task. The model was able to learn only from the transcript (ordering of actions) during training, and predict the exact timing of each action.


Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Li Ding, Chenliang Xu
[CVPR 2018] [paper] [arXiv] [poster] [code]

TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation
Li Ding, Chenliang Xu
[Technical Report] [arXiv]