It’s not far-fetched to imagine that one day it will be commonplace for manufacturing and warehouse workers to get trained using augmented reality headsets. In fact, a group of Northeastern University researchers are in the process of developing and testing that kind of technology today.

Mohsen Moghaddam, a professor of mechanical and industrial engineering at Northeastern, is leading a project that takes advantage of mixed reality headsets from companies like Microsoft to develop AI-based coaches used to train facility workers.

In an interview, Moghaddam played out one scenario where these types of assistants would be ideal.

“You put the headset on, and it instantly scans the environment and provides you with instruction in the form of audio, text, video and immersive animations that are superimposed in 3D space on top of the physical objects in your scene,” he says. “Let’s say you need to unscrew a bolt. It can point to that bolt and provide you with an animation of how you’re supposed to do that on screen.”  

The project is partially getting funded by the National Science Foundation’s Future of Work at the Human-Technology Frontier program, an initiative set up by the foundation to help support researchers working on projects that use next-generation technologies to help reinvent and improve the workplace.  

Moghaddam was one of a dozen Northeastern researchers spotlighting their work at a recent NSF private investigator’s meeting held at the university’s Interdisciplinary Science and Engineering Complex.

The two-day event served as an opportunity for researchers to showcase their work and learn from others in the community. Taskin Padir, a professor of electrical and computer engineering at Northeastern, was the event’s primary host. Among the event speakers was the state’s secretary of labor and workforce development, Lauren E. Jones, a Northeastern graduate.

Let’s say you need to unscrew a bolt. It can point to that bolt and provide you with an animation of how you’re supposed to do that on screen.

Mohsen Moghaddam, a professor of mechanical and industrial engineering at Northeastern

“Northeastern has been an active participant in the program,” Padir says. Five NSF-backed Northeastern projects were highlighted during the event that touched on a range of topics, including food processing, digital manufacturing and work for neurodivergent individuals.

In 2019, Padir was awarded a grant from NSF’s Human’s Technology Frontier Project to investigate how robots could be used to collaboratively work with humans working in seafood processing.

He worked on the project in collaboration with several colleagues, including Alicia Modestino, a public policy, urban affairs and economics professor at Northeastern. She was in charge of analyzing the project from an economics perspective.

During a panel, Modestino said the pandemic and the global supply chain problems that followed highlight the need for automation in the manufacturing and processing spaces.

More work needed to be done in-house and robots helped answer the call.

“We actually had a moment in time where it was actually essential for us to do more seafood processing in the United States,” Modestino said. “You probably don’t know this but we actually export a lot of the fish that we catch in the United States to get processed in China, and then we re-import to eat because we don’t have enough capacity to do it here in the United States. Imagine what that looked like during Covid?”

Collaborative robots were used to help with the packaging of goods and other repetitive tasks. They proved useful in helping improve throughputs and allowed workers to social distance from one another, she said.

The work Northeastern professors are doing to help advance the workplace even extends into the video game realm.

Leanne Chukoskie, a professor in the Bouvé College of Health Sciences, is supporting neurodivergent people by helping them get work experience. Specifically, her team is taking advantage of funding from the NSF to create video game internship opportunities for adults.

It initially started as an in-house internship project, but has since expanded and now has the support of video game developer Ubisoft.

“We actually have mentors from Ubisoft who either themselves identify as neurodivergent or have people in their lives who are neurodivergent,” she says. “They are mentoring our folks.” 

Chukoskie is also taking advantage of “supportive tools” like virtual reality headsets that provide biometric information to conduct mock interviews. The feedback the headsets provide helps candidates prepare for interviews with potential employers.

While there’s often talk about robots and AI taking jobs away from people, Saiph Savage, a Northeastern computer science professor, is exploring how artificial intelligence can help rural workers in places like the Appalachian region use AI to get jobs.

Through her NSF-backed project, Savage was able to develop an open-sourced web plug-in workers can install on their browser that allows them to get more information about the types of positions available to them.

“For example, there are tools that can recommend tasks for them to do on digital labor market spaces, so that they can earn wages,” she says. “[We also developed] tools that can recommend certain tasks for them to do so they can develop certain skills that will allow them to get access to new jobs.”

For the project, Savage and her team conducted interviews with rural workers to gauge what types of problems they face in securing jobs.

“We also started working with the public infrastructure that exists in rural regions in order for them to have access to the internet and a nice space for working,” she says.

Cesareo Contreras is a Northeastern Global News reporter. Email him at c.contreras@northeastern.edu. Follow him on Twitter @cesareo_r.