Abhinav Shrivastava's Academic Website

tl;dr

Assistant Professor,
University of Maryland, College Park

Note to prospective students: We hire PhD students in computer vision, machine learning, and related fields every year. Find more about the graduate program here.

If you are interested in working with me, mention my name in your research statement.

News

02/20:

PatchVAE accepted at CVPR 2020.
02/20:

Kyungjun's paper won Best Paper Award (Applications) at WACV 2020!
01/20:

Serving as an Area Chair for CVPR 2021.
12/19:

One paper accepted at ICLR 2020.
11/19:

Two papers accepted at AAAI 2020.
10/19:

Two papers accepted at WACV 2020.
07/19:

EvalNorm accepted at ICCV 2019.
02/19:

One paper accepted at CVPR 2019.
08/18:

Started as Assitant Professor at University of Maryland, College Park.
07/18:

Two papers accepted at ECCV 2018.
07/18:

Google AI blog post on Self-Supervised Tracking via Video Colorization.

05/18:

Serving as an Area Chair for CVPR 2019.
02/18:

Serving as an Area Chair for ECCV 2018.
08/17:

Started as Visiting Research Scientist at Google.
08/17:

Successfully defended my thesis & Graduated!
07/17:

Google Research blog post on Revisiting the Unreasonable Effectiveness of Data
06/17:

Accepted an Assistant Professor position at UMD (starting in Fall 2018).
06/17:

Paper accepted at ICCV 2017:
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (spotlight).
04/17:

Serving as an Area Chair for CVPR 2018.
03/17:

Paper accepted at CVPR 2017:
A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection.
12/16:

Our work on Top-down Modulation achieves state-of-the-art single model performance on COCO detection!
10/16:

Our team stood 4th in the 2016 COCO object detection challenge (leaderboard, see Challenge2016), using the Priming and Feedback model from our ECCV paper (with some bells and whistles). It is the best VGG16 single-model entry, above a couple of ResNet entries!
08/16:

Code released for OHEM.
07/16:

Paper accepted at ECCV 2016:
Contextual Priming and Feedback for Faster R-CNN.
03/16:

Two papers accepted at CVPR 2016:
Training Region-based Object Detectors with Online Hard Example Mining (oral), and
Cross-stitch Networks for Multi-task Learning (spotlight).
03/16:

A hackathon project during my internship at Microsoft Research is the basis of the 'Seeing AI' product. During my Masters, I worked on a similar project on Assistive Technologies using a first person vision device (read more).
12/15:

Our team stood 4th in the 2015 MS COCO object detection challenge (leaderboard, see Challenge2015). Our entry used hard example mining to train Fast R-CNN (and added some bells and whistles).

About me

I am an Assistant Professor in the Department of Computer Science at University of Maryland, College Park, with a joint appointment in the Institute of Advanced Computer Studies (UMIACS). Before this, I spent 1 year as a Visiting Research Scientist at Google Research.

In August 2017, I received my PhD in Robotics and Artificial Intelligence from the Robotics Institute, Carnegie Mellon University, where I was advised by Abhinav Gupta. My PhD thesis, Discovering and Leveraging Visual Structure for Large-scale Recognition, was supported by Microsoft Research PhD Fellowship for 2014-16.

Before joining PhD, I received my Masters from the Robotics Institute, under the supervision of Alyosha Efros and Martial Hebert. Prior to that, I received my BTech in Computer Science and Engineering from JIIT (Noida, India).

I have also enjoyed working with awesome researchers and engineers in industry, including internships at Google Research and Microsoft Research (details).

Download CV (pdf)

Research Experience

August, 2018 - present

Assistant Professor

University of Maryland, College Park

Joint appointments in Department of Computer Science and Institute of Advanced Computer Studies (UMIACS).
(details to follow)

September, 2017 - August, 2018

Visiting Research Scientist - Google Research

(details to follow)

August, 2016 - July, 2017

Research Assistant - Google Research

Working with Abhinav Gupta, Rahul Sukthankar, and Jitendra Malik on incorporating feedback in object detection models.

Summer, 2015

Research Intern - Microsoft Research

Worked with Ross Girshick and Larry Zitnick on object detection and semi-supervised learning.

Summer, 2013

Research Intern - Google Research

Worked with Mark Segal, Rahul Sukthankar and Thomas Leung on incorporating image geometry in deep neural networks.

Summer, 2012

Research Intern -Microsoft Research

Worked on large-scale indexing and nearest-neighbor search for high-dimensional data with Sanjeev Mehrotra and Jin Li.

Fall, 2010

Research Associate III -- Robotics Institute, Carnegie Mellon University

Continued Masters research on image matching and retrieval, real-time assistive systems, and object detection; and worked on large-scale semi-supervised learning algorithm.

Education

August, 2012 - August, 2017

PhD, Artificial Intelligence

Robotics Institute, Carnegie Mellon University

Working on discovering the underlying regularities, or structure, in our visual world and leveraging it in large-scale recognition algorithms and systems. This work spans a wide range of recognition tasks, and includes frequent collaborations with researchers from both academia and industry.
Find my dissertation here.

August, 2010 - December, 2011

M.S., Aritificial Intelligence

Robotics Institute, Carnegie Mellon University

Worked on data-driven visual similarity for image matching and retrieval, real-time assistive systems, and object detection.

July, 2006 - May, 2010

BTech., Computer Science and Engineering

Jaypee Institute of Information Technology

Thesis on 'A Hypermedia-development Tool for Movie-based Comic-strip Rendering'.

Teaching Experience

Introduction to Deep Learning: CMSC428L, Spring 2019
Advanced Techniques in Visual Recognition and Learning: CMSC828I, Fall 2018

Teaching Assistant Experience

Teaching Assistant, Geometry-based Methods in Vision, CMU; Spring 2013
Teaching Assistant, Data Structures, JIIT; 2008-09
Teaching Assistant, Microprocessors and Controllers, JIIT; 2008-09

Area Chair

Conferences: CVPR'18, ECCV'18, CVPR'19, CVPR'21

Reviewer

Conferences: CVPR'12-17/20, NIPS'12-15, ECCV'12/14/16, ICCV'11/13/15/17/19, ACCV'12-16, SIGGRAPH'14, AAAI'15, 3DV'14-15
Journals: IJCV, TPAMI, CVIU, TKDE

Department Services (Carnegie Mellon University)

Master's Admissions Committee, Robotics Institute; 2015-16
Master's Thesis Committee: Shreyansh Daftry, Krishna Kumar Singh; 2014-17
Ph.D. Qualifier Committee: Aayush Bansal, Ishan Misra, Xiaolong Wang; 2015-17

Interests

I enjoy working on artificial intelligence, particularly computer vision and machine learning; and my research interests include related fields such as robotics, graphics, natural language processing, human-computer interaction, systems, data mining, and cognitive and computational neuroscience.

My long term goal is to equip machines with visual perception abilities, which enables them to understand and respond to their surroundings.

Publications

PatchVAE: Learning Local Latent Codes for Recognition Kamal Gupta, Saurabh Singh, Abhinav Shrivastava IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020 new

Scalable Model Compression by Entropy Penalized Reparameterization Deniz Oktay, Johannes Ballé, Saurabh Singh, Abhinav Shrivastava International Conference on Learning Representations (ICLR), 2020 new pdf

Detecting Human-Object Interactions via Functional Generalization Ankan Bansal, Saketh Rambhatla, Abhinav Shrivastava, Rama Chellappa Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 2020 new pdf

Generate, Segment and Refine: Towards Generic Manipulation Segmentation Peng Zhou, Bor-Chun Chen, Xintong Han, Mahyar Najibi, Abhinav Shrivastava, Ser-Nam Lim, Larry Davis Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 2020 new pdf

Hand-Priming in Object Localization for Assistive Egocentric Vision Kyungjun Lee, Abhinav Shrivastava, Hernisa Kacorri IEEE & CVF Winter Conference on Applications of Computer Vision (WACV), 2020 new oral presentation best paper award (applications) pdf

Boosting Standard Classification Architectures Through a Ranking Regularizer Ahmed Taha, Yi-Ting Chen, Teruhisa Misu, Abhinav Shrivastava, Larry Davis IEEE & CVF Winter Conference on Applications of Computer Vision (WACV), 2020 new pdf

EvalNorm: Estimating Batch Normalization Statistics for Evaluation Saurabh Singh, Abhinav Shrivastava IEEE International Conference on Computer Vision (ICCV), 2019 new bibtex / pdf

Relational Action Forecasting Chen Sun, Abhinav Shrivastava, Carl Vondrick, Rahul Sukthankar, Kevin Murphy, Cordelia Schmid IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 bibtex / pdf

Actor-centric Relation Network Chen Sun, Abhinav Shrivastava, Carl Vondrick, Kevin Murphy, Rahul Sukthankar, Cordelia Schmid European Conference on Computer Vision (ECCV), 2018 bibtex / pdf

Tracking Emerges by Colorizing Videos Carl Vondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, Kevin Murphy European Conference on Computer Vision (ECCV), 2018 bibtex / pdf
In Media: Google AI Blog

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era Chen Sun, Abhinav Shrivastava, Saurabh Singh, Abhinav Gupta IEEE International Conference on Computer Vision (ICCV), 2017 spotlight presentation bibtex / pdf
In Media: Google Research Blog, Wired

Beyond Skip Connections: Top-Down Modulation for Object Detection Abhinav Shrivastava, Rahul Sukthankar, Jitendra Malik, Abhinav Gupta arXiv [cv.CV], 2016 bibtex / pdf

A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection Xiaolong Wang, Abhinav Shrivastava, Abhinav Gupta IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017 bibtex / pdf / code

Contextual Priming and Feedback for Faster R-CNN Abhinav Shrivastava, Abhinav Gupta European Conference on Computer Vision (ECCV), 2016 bibtex / pdf / poster

Training Region-based Object Detectors with Online Hard Example Mining Abhinav Shrivastava, Abhinav Gupta, Ross Girshick IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 oral presentation bibtex / pdf / code / video / poster / slides (pptx)

Cross-stitch Networks for Multi-task Learning Ishan Misra*, Abhinav Shrivastava*, Abhinav Gupta, Martial Hebert (*equal contribution) IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 spotlight presentation bibtex / pdf / poster / slides

Mid-level Elements for Object Detection Aayush Bansal, Abhinav Shrivastava, Carl Doersch, Abhinav Gupta arXiv [cs.CV], 2015 bibtex / pdf / analysis

Watch and Learn: Semi-supervised Learning of Object Detectors from Videos Ishan Misra, Abhinav Shrivastava, Martial Hebert IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 bibtex / pdf / project page / poster

Applying artificial vision models to human scene understanding Elissa M. Aminoff, Mariya Toneva, Abhinav Shrivastava, Xinlei Chen, Ishan Misra, Abhinav Gupta, Michael Tarr Frontiers in Computational Neuroscience, 2015 bibtex / pdf

Enriching Visual Knowledge Bases via Object Discovery and Segmentation Xinlei Chen, Abhinav Shrivastava, Abhinav Gupta IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014 bibtex / pdf / project page / code / poster / supplement

Data-driven Exemplar Model Selection Ishan Misra, Abhinav Shrivastava, Martial Hebert IEEE Winter Conference on Applications of Computer Vision (WACV), 2014 oral presentation best student paper award bibtex / pdf / project page / slides (pptx)

Building Part-based Object Detectors via 3D Geometry Abhinav Shrivastava, Abhinav Gupta IEEE International Conference on Computer Vision (ICCV), 2013 bibtex / pdf / project page

NEIL: Extracting Visual Knowledge from Web Data Xinlei Chen, Abhinav Shrivastava, Abhinav Gupta IEEE International Conference on Computer Vision (ICCV), 2013 oral presentation bibtex / pdf / project page / code / test code / video / poster / slides (pptx) In Media: CNN, Discover Magazine, Newsweek, Forbes, Yahoo! News, BBC News, AP, Business Insider, Slashdot, Engadget, Techradar

Constrained Semi-Supervised Learning using Attributes and Comparative Attributes Abhinav Shrivastava, Saurabh Singh, Abhinav Gupta European Conference on Computer Vision (ECCV), 2012 oral presentation bibtex / pdf / project page / video / slides (pptx)

Data-driven Visual Similarity for Cross-domain Image Matching Abhinav Shrivastava, Tomasz Malisiewicz, Abhinav Gupta, Alexei A. Efros ACM Transaction of Graphics, (Proceedings of SIGGRAPH Asia), 2011 oral presentation bibtex / pdf / project page / code / video / supplement / data / slides (pptx) In Media: Techcrunch, Science Daily, Y! Hacker News, New Atlas (formerly Gizmag)

Invited Papers and Technical Reports

Real-time Household Object Detection from First-person's view using Exemplar-SVMs Abhinav Shrivastava, Abhinav Gupta, Alexei A. Efros In IEEE Workshop on Egocentric Vision at CVPR, 2012 extended abstract & poster project page, code and demo

Exemplar-SVMs for Visual Object Detection, Label Transfer and Image Retrieval Tomasz Malisiewicz, Abhinav Shrivastava, Abhinav Gupta, Alexei A. Efros In International Conference on Machine Learning (ICML), 2012 invited applications talk & extended abstract pdf / slides

Measuring and Increasing the capacity of Natural HOG Statistics Tinghui Zhou, Abhinav Shrivastava, Guillaume Obozinski, Abhinav Gupta, Alexei A. Efros Technical Report, Carnegie Mellon University MS thesis (T. Zhou) pdf / supplement

HOG and Spatial Convolution on SIMD Architecture Ishan Misra, Abhinav Shrivastava, Martial Hebert Technical Report, Carnegie Mellon University pdf / code

Selected Awards and Honors

Best Paper Award (Applications), IEEE Winter Conference on Applications of Computer Vision; 2020
Outstanding Reviewer Award, IEEE CVPR; 2015
Microsoft Research Ph.D. Fellowship; 2014-16
CNNs Top-10 Ideas 2013 (Thinking Tech)
Best Student Paper Award, IEEE Winter Conference on Applications of Computer Vision; 2014
Selected for Google Graduate CS Forum; 2012
Research Highlight of the week, Computing Community Consortium; 2011
Vice Chancellor Gold Medal (awarded to Rank 1 out of 120), Dept. of Computer Science and Engineering, JIIT; 2006-10

Selected Media Coverage

Self-Supervised Tracking via Video Colorization, Google AI Blog; 2018
Revisiting the Unreasonable Effectiveness of Data, Google Research Blog; 2017
AI And 'Enormous Data' Could Make Tech Giants Harder to Topple, Wired; 2017
Thinking Tech (Top-10 Ideas), CNN; 2013
The Greatest Hits, and Misses, of an Image-Learning AI, Discover Magazine; April 2015
Watch out, WATSON. You've got competition, Newsweek; 2013
Computer Learns Common Sense From The Internet, Forbes; 2013
New research aims to teach computers common sense, Yahoo! News; 2013
Computer uses images to teach itself common sense, BBC News; 2013
New research aims to teach computers common sense, AP; 2013
Researchers Are Trying To Teach Computers Common Sense, Business Insider; 2013
CMU AI Learning Common Sense By Watching the Internet, Slashdot; 2013
Carnegie Mellon computer learns common sense through pictures, shows what it's thinking, Engadget; 2013
Meet NEIL, the computer that thinks like you do, Techradar; 2013
CMU Researchers One-Up Google Image Search And Photosynth With Visual Similarity Engine, Techcrunch; 2011
Computerized method for matching images in photos, paintings, sketches created, Science Daily; 2011
A better search for visually similar images, Y! Hacker News; 2011
CMU algorithm matches sketches, paintings to photographs, New Atlas (formerly Gizmag); 2011

Selected Talks, Seminars and Lectures

Top-down Mechanisms in Bottom-up Deep Networks

workshop

Workshop on Deep Learning, University of Maryland, College Park; May 2017

The Small and the Rare: the Twin Menace of Visual Recognition

research tech talk

Faceook AI Research (FAIR); Jun. 2017

research tech talk

4Catalyzer; Jun. 2017

colloqium

CS Colloqium, University of Maryland, College Park; Mar. 2017

seminar

GRASP Seminar, University of Pennsylvania; Feb. 2017

Training Region-based Object Detectors with Online Hard Example Mining

conference

CVPR; Jun. 2016; video

NEIL: Extracting Visual Knowledge from Web Data

CMU VASC Seminar; Nov. 2013

conference

ICCV; Dec. 2013; video

course

Guest Lecture: Visual Recognition, University of Pittsburgh; Feb. 2015

Constrained Semi-Supervised Learning using Attributes and Comparative Attributes

CMU VASC Seminar; Sep. 2012

conference

ECCV; Dec. 2012; video

course

Guest Lecture: Visual Recognition, University of Pittsburgh; Feb. 2015

Data-driven Visual Similarity for Cross-domain Image Matching

conference

SIGGRAPH Asia; Dec. 2011

course

Guest Lecture: Visual Recognition, University of Pittsburgh; Feb. 2015

Overview of Object Detection with historical context

course req.

Learning-based Methods in Vision, CMU; Oct. 2013

Semantic vs. Visual Subcategories in Computer Vision and Neuroscience

course req.

The Visual World as seen by the Neurons and Machines; Mar. 2014

Building Part-based Object Detectors via 3D Geometry

CMU VASC Seminar; Nov. 2013

Tutorial on Caffe toolbox

course req.

Big Data Approaches in Vision, CMU; Sep. 2014

Vanishing Point Estimation, and applications to Scene-layout Estimation

course

Guest Lecture, Geometry-based Methods in Vision, CMU; 2013-16

Indexing in High-dimensional spaces (for large-scale nearest neighbor search)

industry

Bing, Microsoft; Aug. 2012

Tutorial, CMU; Sep. 2012

Tutorial and Workshop on Automated Robotics (Micro-mouse)

course

Microprocessors and Controllers, JIIT; 2008-09

Guest Lecture, Computer Society of India (CSI) Week, IGIT, Indraprastha (IP) University; 2008

Guest Lecture, IEEE Week, NIEC; 2008

Workshop, IEEE Winter Academic Program, JIIT; 2008

Selected Robotics Competitions (undergrad)

Finalists, Robo-Relay, IIT, Kharagpur; 2008
Runner-up, Line Follower, Delhi College of Engineering; 2008
Finalist, Maze Ablaze, Delhi College of Engineering; 2008
Winner, Cross Terrain Racing, USIT, Indraprastha (IP) University (New Delhi); 2007
Winner, Trash Collection, IGIT, Indraprastha (IP) University (New Delhi); 2007
Runner-up, Chequered Flag, IGIT, Indraprastha (IP) University (New Delhi); 2007

Selected Positions Held (undergrad)

Technical Research Coordinator, Creativity and Innovation Cell in Robotics, JIIT; 2008-09
Sun Campus Ambassador (for Sun Microsystems Inc.), JIIT; 2008
President, JIIT Youth Club (student union), JIIT, 2008-09
Team Leader, Microsoft Go-Alive Challenge, JIIT; 2008
Treasurer, EBULLIENCE, JIIT; 2007
Chief Project Coordinator, Multimedia Project (2D Graphics) (managing more than 800 students), JIIT; 2007