UMD Ph.D. Student Snehesh Shrestha's Software Uses AI to Teach You How to Play the Violin

On the ground floor of one of the new computer buildings at the University of Maryland, Anna Kelleher played her centuries-old violin while a program running on a laptop in front of her told her to do things such as raise her chin or widen her stance.

These were common mistakes that Kelleher knows not to do. After all, she’s a graduate student studying violin performance. But she also teaches violin to others, and the program she was demonstrating might someday help those she teaches to play even better.

Believers in artificial intelligence say the program will radically transform our lives in so many ways.

It’s designed by Snehesh Shrestha, a Ph.D. candidate at the Department of Computer Science, and is the perfect example of how the University of Maryland is building bridges between AI and every other academic program on campus.

A simple webcam found on your laptop, or even your phone, captures enough movement and audio from your performance that the AI program can tell what you’re doing wrong. Whether your stance is too wide or narrow, to whether or not your chin is in the right spot, it can see and also hear everything you’re doing right and wrong.

The program was designed to try “to understand the whole space, not just blindly building a technology, but understanding how can we fill the gaps that are currently there in the entire music learning process,” Shrestha said. “And by identifying gaps where we can empower the teacher and the students, we could really build something a lot more powerful than just building a single technology. And that really was like the starting point of exploring into what the technology can provide towards the future direction of music education.”

On the monitor, the teacher can see the student in 3D — every angle imaginable — to see how they stand and how they move. Technology, including a piece that looks sort of like a smartwatch, can also send cues to the student through vibrations in the wrist.

Click HERE to read the full article