Engineer develops robotic signature arm for deafblind people

0

Samantha Johnson, bioengineer and founder of Tatum Robotics, vividly remembers meeting a deafblind person for the first time.

She was in her second year at Northeastern University, where she was taking an American Sign Language course, which required her to learn more about the local Deaf community by attending local Deaf events. At one of the events, she met a deafblind woman.

“I thought it was really like magic how they could kind of understand [what was being communicated] purely in their hands,” Johnson says.

Deafblind people communicate via tactile ASL with the assistance of a partner or interpreter, holding their dominant hand while the interpreter signs. Signing with the woman, Johnson immediately thought of a robotic device that could sign for deafblind people. But she didn’t have the time or the skills to develop such a device in her second year, so she put the idea away in her head.

Last month, however, Johnson, who is now 23 and a 2021 College of Engineering alumnus with a combined bachelor’s and master’s degree in bioengineering, was announced as the first winner of the 2022 Innovator Awards presented by the Northeastern’s Women Who Empower Platform in the recent graduate category for the development of a robotic arm capable of signing in ASL.

The award comes with a $22,000 cash prize that Johnson plans to spend on further research and development, as well as hiring deafblind consultants to help brainstorm ideas and evaluate prototypes.

Samantha Johnson, founder of Tatum Robotics and first place winner of the 2022 Innovator Awards in the Young Graduates category presented by Women Who Empower, graduated with a combined bachelor’s and master’s degree in bioengineering from the College of Engineering in 2021. Photo by Matthew Modoono/Northeastern University

Johnson grew up in Hudson in central Massachusetts and describes herself as an optimistic person with high energy, who watches over everyone and makes sure everyone is happy.

“I really see that we can accomplish anything we want,” Johnson says. “I can work 20 hours a day, especially if I see progress, I can continue. »

Johnson began working with special needs students in middle school through a conversation club that helped them practice making eye contact, asking questions and practicing active listening.

In high school, Johnson was one of the first members to join a unified track team, where athletes with special needs competed alongside partners with disabilities. This experience allowed her to learn more about the impact of assistive technologies, including wheelchairs and prosthetics, which gave students abilities they otherwise wouldn’t have had, she says.

“I decided to do bioengineering on the assumption that I would either do prosthetics or assistive technology,” says Johnson, who still works as a life skills assistant for adults with disabilities after work. .

Johnson revived his idea of ​​creating a communication device for the deafblind during his senior year of college. In 2020, she was supposed to work on her thesis in a lab, but all the labs closed due to the COVID-19 pandemic with the university. Instead, Johnson moved on to work on the thesis in her small studio.

Tatum Robotics, a company founded by Samantha Johnson, is developing a low-cost, anthropomorphic robotic hand that will spell tactile sign language and a safe, compliant robotic arm to help deafblind people receive information and communicate with the world without an interpreter. Photo by Matthew Modoono/Northeastern University

She installed 3D printers, bought soldering irons and started building a signature hand in conjunction with the Deaf-Blind Contact Center in Allston. She designed all the parts herself, 3D printed them, assembled them, and started writing code. After the first prototypes were built, his father Todd Johnson, a software engineer, took over the development of software for the hand in order to get it signed.

To ensure the device is most effective, Johnson enlisted the help of the Northeast Experiential Robotics Institute and its director Taskin Padir; assistant professor of biology Jon Matthis and his motion capture software project FreeMoCap; associate professor in bioengineering and thesis supervisor Chiara Bellini; and Assistant Professor of Chemistry and Chemical Biology Mona Minkara. She has also established a collaboration with the New Dexterity group at the University of Auckland.

“Samantha used a truly experiential approach to her project,” says Padir. “Rather than looking for an application for a technology, she identified an unmet need within the deafblind community and set a course for her project to have an impact. This is the recipe for success. »

He also notes that Johnson is a true entrepreneur and go-getter, able to attract talent and motivate her team towards her project goals.

By the time Johnson graduated from the Northeast, she had created a hand-wrist system that could spell words texted from a computer with her fingers. Johnson sought employment, but none of the options involved working directly alongside disabled communities, which was important to her.

By a twist of fate, the Canadian National Institute for the Blind saw an article about Johnson’s work and offered him a small grant to continue working on a robotic arm. This allowed him to start his own company, Tatum Robotics, and move to a new space at MassRobotics, an innovation workspace in Seaport.

Tatum Robotics currently has one full-time employee, linguist and Northeast alumnus Nicole Rich, two graduate co-ops and a large number of volunteers, Johnson said. His father continues to be the lead software architect, and his mother, Tammy Johnson, who is an accountant, helps manage the finances.

“We’re hoping that once we get the funding, we can hire a few more people, especially more deafblind people on staff,” Johnson said.

Since the thesis project became a real business, Johnson has forged collaborations with the Perkins School for the Blind, the National Technical Institute for the Deaf, and the Helen Keller National Center. She brings in deafblind collaborators as often as possible to better understand tactile ASL and the needs of the community.

Tatum Robotics is currently continuing to work on a low-cost, anthropomorphic robotic hand that will finger-spell tactile sign language and a secure, compliant robotic arm so the system can sign more complex words and sentences. They are creating a user interface and gesture recognition software with the help of some clubs in the North East, so that deafblind people can respond and have two-way communication on their own, similar to video relay services that see but the deaf and hard of hearing. and people with speech disabilities use in real time through a sign language interpreter.

“We bring in native deaf and deafblind signers to sign so we can actually map their signature moves directly to our robot,” Johnson says.

Brian Mansur, program director at the Deaf-Blind Contact Center, says he sees a lot of potential in the devices Johnson is working on. They would make things like reading newspapers and emails, going online, texting a friend, or even watching movies accessible to deafblind people.

“If someone comes in and tries to sign in their hand all day, it’s too big a task to get any information,” Mansur says. “They can touch a robot anywhere and anytime and get nuances and changes that happen in language because language is constantly changing.”

Deafblind people would become more employable with these communication devices, Mansur says.

He highlights how committed Johnson and Tatum Robotics are to the project.

“It’s very impressive. I haven’t seen anyone dedicated to it yet,” Mansur says.

Johnson has received positive feedback from other engineers and entrepreneurs, and they often suggest focusing on other, larger audiences or more lucrative markets, Johnson says.

“The technology itself is new,” Johnson says. “What we are doing could do more than benefit deafblind people.”

But for now, she prefers to focus on the deafblind community, which has been historically underserved, and, perhaps, expand to other areas later.

“We can really have such an impact on the lives of these people who currently have no means of communication. And it could be the first,” Johnson says.

For media inquiries, please contact [email protected]

Share.

Comments are closed.