If interested in joining my lab as a PhD student, please click here.

I’m a tenured Associate Professor of Computer Science at the University of Rochester. I have secondary appointments in Brain and Cognitive Sciences, The Goergen Institute for Data Science, and The Center for Visual Science. My lab’s main focus is basic research in deep learning, especially prerequisite capabilities for artificial general intelligence (AGI). I have worked on deep continual learning, multi-modal scene understanding, visual question answering, medical computer vision (computational pathology and radiology), self-supervised learning, semantic segmentation, object recognition, object detection, active vision, object tracking, and more. Beyond AI, I also have a background in cognitive science, primate vision, theoretical neuroscience, and eye tracking.

From 2018 – 2022, I was a member of Paige’s executive leadership team, where I oversaw AI research activities at the start-up. Paige is revolutionizing pathology and oncology by creating clinical-grade AI systems to help pathologists diagnose cancer. Among other achievements, my team developed Paige Prostate, the first FDA-approved AI system in pathology. I helped grow the company from fewer than 10 employees when I joined to over 180 people. I also led Paige’s patent initiative, and I have over 50 granted patents. Currently, I’m a member of Paige’s Scientific Advisory Board.

Previously, I was a tenured Associate Professor in the Carlson Center for Imaging Science at the Rochester Institute of Technology (RIT). At RIT, I co-founded the Center for Human-aware AI (CHAI), and I served as its Associate Director for four years. I was also a member of RIT’s McNair Scholars Advisory Board, part of RIT’s Division of Diversity and Inclusion. From 2019 – 2022, I was a visiting professor at Cornell Tech in New York City, where I taught a course on Deep Learning for four years to about 100 graduate students annually.

I received my PhD from UC San Diego (UCSD), was a postdoctoral scholar at Caltech, and worked as a scientist at NASA JPL.

See my publications page for links to specific projects, code, and datasets.

Google Scholar | LinkedIn | Twitter | YouTube

Recent News

2024/09: Our paper on understanding out-of-distribution generalization was accepted to NeurIPS-2024!
2024/07: Our paper on the foundation model Virchow was published in Nature Medicine!
2024/03: I was elected as a Senior Member of AAAI!
2023/12: My PhD student Robik Shrestha defended his dissertation on VQA and bias robust AI! He is joining Amazon AGI.
2023/09: Won an NSF grant with Dhireesha Kudithipudi and others for $2M!
2023/09: Won a DoE grant with Riccardo Betti and others for $3M to use generative AI to advance nuclear fusion research!
2023/08: Lab alumnus Dr. Ron Kemker won the Harold Brown Award from the USAF!
2022/08: I was honored to be profiled in Quanta Magazine.
2022/07: Our paper on OccamNets was accepted to ECCV-2022 as an Oral!
2022/07: I am now an Associate Professor of Computer Science at the University of Rochester, where I’ll be leading various AI activities!
2022/06: My PhD student Usman Mahmood defended his dissertation on detecting and mitigating bias in AI for radiology!!
2022/06: My PhD student Manoj Acharya defended his dissertation and will be joining Amazon!!
2022/04: My PhD student Tyler Hayes defended her dissertation and will be joining Naver Labs!
2021/09: I am honored to have received an NSF CAREER Award!
2021/09: Led by Cissi Alm, we won an NSF NRT award for $2M to improve graduate student training in AI!
2021/09: Paige Prostate received FDA clearance! I oversaw the AI research activities for the system.
2021/05: I was awarded tenure at RIT and promoted to Associate Professor!
2021/03: With SRI, we won an award from DARPA for Lifelong Learning on Starcraft 2!

Older News

2020/09: Our paper analyzing metrics and bias in VQA-CP was accepted to NeurIPS-2020!
2020/08: My PhD student Ryne Roady defended his dissertation on open set recognition!
2020/07: Our REMIND paper for streaming CNN training was accepted to ECCV-2020!
2020/06: Our paper on deep streaming linear discriminant analysis won best paper award at CLVISION!
2020/06: Gave a talk at CVPR-2020 on improving continual learning.
2020/06: Gave a talk at CVPR-2020 on Paige’s cancer detection technology.
2020/05: Paper accepted to ACL examining methods for VQA with changing priors.
2020/03: My PhD student Kushal Kafle defended his dissertation and will be joining Adobe Research!!
2019/10: Paper accepted to WACV-2020 that achieves super-human performance on chart question answering!
2019/09: Helped RIT win the Facebook OpenEDS Semantic Segmentation Challenge!
2019/06: Won an NSF grant for multi-modal streaming learning!
2019/06: Won a grant with Christy Tyler to detect invasive plant species!
2019/04: Gave an invited talk at SUNY Buffalo on memory replay and the hippocampus
2019/02: Our paper on universal VQA models was accepted to CVPR-2019!
2019/02: Our paper on Visual Query Detection was accepted to NAACL-2019!
2019/01: Our paper on memory efficient experience replay for streaming learning was accepted to ICRA-2019!
2019/01: Became a Visiting Professor at Cornell Tech and will be teaching Deep Learning. I’ll be at RIT 1-2 days per week.
2018/12: I am co-organizing the NAACL-2019 Workshop on Shortcomings in Vision and Language (SiVL).
2018/11: Joining Paige as a Senior AI Scientist to lead a group working on improving detection and treatment of cancer.
2018/10: Won a $206K DARPA/ARL award for lifelong learning as PI!
2018/10: My lab’s TallyQA paper was accepted to AAAI-2019. It is the largest VQA counting dataset.
2018/09: Joined RIT’s McNair Scholars Advisory Board to help organize training programs for underrepresented students.
2018/08: Panelist discussing Artificial General Intelligence on NPR Member Station WXXI’s Connections with Evan Dawson.
2018/06: Our web-app for Visual Question Answering was relaunched: http://www.askimage.org/
2018/06: Released EarthMapper software toolbox for low-shot semantic segmentation.
2018/05: Appointed Associate Director of RIT’s Center for Human-aware AI (CHAI).
2018/04: Ron Kemker became the first person to defend his PhD in my lab! Congratulations!!
2018/03: Two papers on semantic segmentation for remote sensing accepted in TGRS and P&RS.
2018/03: Check out EarthMapper, our toolbox for low-shot semantic segmentation.
2018/02: Won a $352K grant from USAF Materiel Command as Co-PI (Andreas Savakis is PI)!
2018/02: Our DVQA paper that does VQA for data visualizations was accepted to CVPR-2018!
2018/01: Our FearNet paper for lifelong learning was accepted to ICLR-2018!
2018/01: Teaching IMGS 682 – Image Processing and Computer Vision.
2017/11: Adobe Research has given my lab a gift of $22,000. Thank you Adobe Research!
2017/11: Won a $33K NGA Phase 1 SBIR with our collaborators at Intelligent Automation, Inc. to do target detection.
2017/11: My lab’s paper on measuring catastrophic forgetting was accepted to AAAI-2018!
2017/10: I was an invited speaker at the ICCV-2017 Workshop on Closing the Loop between Vision and Language.
2017/08: Our new VQA dataset, TDIUC, is now publicly available.
2017/08: Our RIT-18 dataset for multispectral semantic segmentation is now publicly available.
2017/08: Teaching the course Deep Learning for Vision.
2017/07: My paper with Dhireesha Kudithipudi’s lab on video classification was accepted to ICRC-2017.
2017/07: Our paper analyzing VQA algorithms using TDIUC was accepted to ICCV-2017.
2017/06: My first robotics paper, which is on using deep learning for detecting good grasps, was accepted to IROS-2017.
2017/06: My lab’s critical review paper on the state of VQA was accepted by the journal CVIU.
2017/05: I am co-organizing the ICCV-2017 workshop on Mutual Benefits of Cognitive and Computer Vision (MBCC).
2017/02: I’m an area chair for ICIP-2017.
2017/01: Teaching IMGS 682 – Image Processing and Computer Vision.
2016/12: Our paper on self-taught deep learning models for semantic segmentation of hyperspectral images was accepted to IEEE TGRS.
2016/08: Teaching IMGS 789 – Deep Learning for Vision.
2016/06: Our paper on Answer-Type Prediction for Visual Question Answering appeared at CVPR-2016.
2016/05: I won the RIT College of Science Rising Star Award.
2016/03: Our paper on using gnostic fields and saliency for tracking appeared at WACV-2016.
2016/01: Started teaching IMGS 682 – Image Processing and Computer Vision.
2015/08: Started as an assistant professor at RIT. My lab: http://klab.cis.rit.edu/
2015/07: The VAIS dataset for ship classification is now available for download.
2015/07: Our paper on combining hierarchical ICA with Gnostic Fields appeared at COGSCI-2015.
2015/06: Our paper on the VAIS dataset for ship classification appeared at the CVPR-2015 Perception Beyond the Visible Spectrum workshop.
2015/01: Our paper on the Multi-Fixation Pattern Analysis, a new method for analyzing eye movement data, appeared in Vision Research.

Outreach & Service

For nearly two decades, I’ve endeavored to promote diversity in graduate education. During my time at RIT, I was an advisory board member and faculty mentor for the RIT McNair and LSAMP Scholars programs, which are aimed at preparing first-generation college students and members of underrepresented minority groups for graduate school. I gave workshops on improving grades, conducting research, overcoming challenges, and applying to graduate school. Before joining RIT, I gave talks at the California Forum for Diversity and at several California State Universities to help undergraduates obtain a better idea of how to get into a PhD program and what is expected of them once they are accepted. During my PhD, I was a mentor to high school students at The Preuss School, which serves low income students.

I have been active in organizing academic conferences. I was General Chair for the NSF sponsored 5th Annual inter-Science of Learning Center Conference (iSLC). I was an area chair for ICIP-2017, and I have helped organize workshops at various artificial intelligence conferences.

Biography

I grew up in a tiny town in rural Oklahoma, where I first began to explore artificial intelligence in 1996 by creating “bots” to play online multiplayer computer games in high school. As an undergraduate at Oklahoma State University, I double majored in philosophy and computer science. Subsequently, I earned a M.S. in computer science from the University of Southern California (USC) , with an emphasis in artificial intelligence and neuroscience while working with Michael Arbib, an early pioneer in computational neuroscience and neural networks. I then went on to earn a Ph.D. in computer science at the University of California, San Diego (UCSD), where I was a member of Gary Cottrell’s research group. Afterwards, I became a Caltech Postdoctoral Scholar, where I worked at NASA’s Jet Propulsion Laboratory (JPL) as part of the Maritime and Aerial Perception group in the Robotics and Mobility Section. After eight months, JPL hired me as a Research Technologist, where I helped develop artificial vision systems for autonomous ships using deep learning. In August 2015, I joined the Chester F. Carlson Center for Imaging Science at RIT as a Professor, and I received tenure in 2021. In 2018, I joined Paige as one of the first employees, where I have overseen AI research for building computational pathology systems for improving cancer diagnosis and treatment. Most recently, I joined the University of Rochester as an Associate Professor of Computer Science. I am a recipient of an NSF CAREER award.