In our inaugural edition, From the Future looked at AI research at Notre Dame related to broad philosophical, political and practical issues. This week, we profile three Notre Dame researchers using AI for unusual or unexpected applications in fields as diverse as peace studies, science and the humanities. These projects demonstrate how AI disruption will be truly ubiquitous, but also how Notre Dame is a leader in conjuring creative ways to use AI to advance scholarship in a multidisciplinary fashion.
Reading into the past with AI vision
Walter Scheirer, Dennis O. Doughty Collegiate Associate Professor, Computer Science and Engineering
Courtesy of Walter Scheirer
As far back as his undergraduate years, Walter Scheirer, Dennis O. Doughty Collegiate associate professor of computer science and engineering, envisioned novel applications for AI. His vision, as it were, was that computers themselves would be able to see.
“I thought [computer vision] was interesting, because it’s a really broad capability,” Scheirer said. “My background is pretty diverse, so I was interested in social science questions and humanistic questions … I wanted to develop increasingly sophisticated algorithms that could help on problems that I thought were really hard and interesting in the world of digital humanities.”
Years later in 2017, a group from Roma Tre University contacted Scheirer about a project that was right up his alley of computer vision in the humanities. The group had been working on transcribing Latin manuscripts at the Abbey Library of St. Gall in Switzerland and was hoping to leverage technology to improve this process.
Scheirer joined forces with classicists and medievalists from around Notre Dame and got to work on an AI program that could transcribe the ancient documents into text form. The AI was trained using supervised learning, where computer scientists would feed examples of already transcribed texts into their AI so it could learn to “read” Latin.
Scheirer noted that training the AI involved interdisciplinary collaboration. Computer scientists had to do the technical programming, but humanities scholars had to help explain the basics of medieval texts and provide the input data (the sample transcriptions) required for machine learning.
“There was a lot of generosity and sharing [of] information on both sides,” Scheirer said. “We didn't know basically anything about manuscript studies when we started this project, but on the other side, the humanists didn't know anything about AI …So we needed each other.”
After seeing positive results with Latin, Scheirer has applied similar computer vision programs to transcribing or translating materials in other languages. Projects have revolved around more mainstream languages like French or forms of English, but also obscure dialects like Ge’ez, an ancient Ethiopian script.
But Scheirer has even more ambitious ideas for how AI vision can help the humanities. Scheirer mentioned the concept of “distant reading,” where researchers could have AI analyze thousands of works to discern broad patterns or connections between these texts and the knowledge within them.
“You can't read all the books, but what if you could summarize information in many thousands of books?” Scheirer said. “Can you see trends over time? What ideas come into fashion? What ideas are around for a little while and go out of fashion? [It’s] like the historical progression of knowledge, right? You can use textual analysis algorithms to do that.”
As interest increases in the emerging field of digital humanities, Scheirer sees AI as a tool that will expand the possibilities of humanities scholarship.
“It opens up new modes of inquiry,” Scheirer said. “We're able to pose questions that we couldn’t pose before and start to tackle them using these computational tools.”
Putting the “science” in computer science
Jian-xun Wang, Assistant Professor, Aerospace and Mechanical Engineering
Courtesy of Jian-xun Wang
That AI would have a multitude of applications in the sciences is perhaps not unexpected. Computer science is, after all, a science itself with a close relationship to other formal and natural fields.
Indeed, Jian-xun Wang, assistant professor of aerospace and mechanical engineering, finds that in his research revolving around AI in scientific applications, it can be difficult to discern whether one is a computer scientist, another kind of scientist or all of the above.
“In scientific [machine learning], it is hard to define computer scientists and domain scientists,” Wang wrote in an email. “Most of the time, we are both, since you need to understand the physics behind the problem and also machine learning techniques to create new scientific AI models. But we also closely collaborate with pure domain/computer scientists to have deep expertise in each field. We view ourselves as the bridge.”
Wang’s “Computational Mechanics & Scientific Artificial Intelligence” (CoMSAIL) Lab at Notre Dame works on a range of projects at the intersection of AI and scientific research.
A significant portion of Wang’s research has, dating back to his PhD days, involved using AI to model complex physical phenomena like fluids. In these cases, AI models are advancing research related to physics, but physics is also advancing the development of AI by providing knowledge and principles upon which AI models can be improved. In this sense, there is a reciprocity to the research — computer science and natural science are mutually advanced.
Physics-informed AI programs have numerous applications in their own right. The CoMSAIL Lab has used AI to model fluids in engineering applications, turbulent water flows related to ships in maritime or naval contexts, and blood flows in cardiovascular systems.
While direct scientific applications are helpful, Wang indicated that AI’s most disruptive impact on science may come in a different, perhaps unexpected way. By using AI not for “doing” science per se but for reading papers and conducting literature reviews, scientists could — in a manner analogous to the “distant reading” mentioned above — drastically enhance their ability to extract knowledge, make connections and advance research in a transformative fashion.
“AI has the potential to revolutionize the way scientific research is conducted, from streamlining data collection and analysis to generating new scientific hypotheses,” Wang wrote. “AI algorithms can help researchers process and analyze large data sets, identify patterns and relationships that would be difficult for humans to detect and make predictions that can guide future experiments.”
While the use of AI for textual research can help any scientific field, Wang mentioned genomics, drug development and climate forecasting as areas that may especially benefit from AI readers combing over massive amounts of data and recognizing patterns.
However, Wang noted that AI disruption is not inevitable. Researchers will need to make an active effort to ensure AI systems are functionally safe and responsibly applied.
“There will need to be continued investment in the development and application of AI technologies in scientific research to ensure that their full potential is realized,” Wang wrote.
Using AI to track online violence
Tim Weninger, Frank M. Freimann Associate Professor of Engineering, Computer Science and Engineering
Courtesy of Tim Weninger
When Tim Weninger, Frank M. Freimann associate professor of engineering, first visited USAID (U.S. Agency for International Development), he said he felt a bit out of his element.
“I was the only computer scientist within ten miles of that place,” Weninger said. “Computer scientists are more interested in making the latest startup than going and engaging with the State Department or with USAID. So I was a fish out of water.”
Rather than going the stereotypical startup route, Weninger wanted to apply his knowledge of computer science and AI to solve problems related to peace and violence.
Now, Weninger and colleagues from across campus work on a USAID-funded project that tracks indicators of violence on social media.
The project originated in the Indo-Pacific region where Weninger explained that “Facebook is the internet” — the primary source for communicating with others, for consuming media and for accessing information. In this region, as in many parts of the world, Facebook has proven to be a platform for propagating misinformation via altered images and videos, sometimes with violent connotations.
That’s where AI comes in. Weninger and his team created a program to comb social media for doctored content that promotes violence or hatred. Their AI can sift through millions of images per day, a task impossible for humans, and identify warning signs of violence.
“We can learn a lot about a country and a culture and a people and what they're afraid of by looking and watching how they manipulate images,” Weninger said. “With AI tools, I can discover that and tell us the story that is kind of emerging.”
After success in Indonesia with their AI, Weninger and his team are expanding into other countries, most recently Ukraine and France. This initial project with USAID has also led to more AI-driven collaborations planned for the future.
Despite being a computer scientist, Weninger’s roster of postdocs includes people from diverse backgrounds, like peace studies. While Weninger believes in AI’s power to solve social scientific or political problems, he also recognizes that computer scientists need to collaborate to achieve impact in these spaces.
“Computer scientists pretend that we can solve the world's problems through technology, but we can't.” Weninger said. “We can bring technology to bear on problems but we need subject matter experts who can guide us in these problems. That's why I have hired folks from [the Kroc Institute] or from [the Pulte Institute] — because they're experts.”
Moving forward, Weninger thinks AI will have a disruptive role in the social sciences. And he sees Notre Dame as a leader in advancing new AI applications that focus on helping people in the way that his work has done.
“We have a different tact here at Notre Dame,” Weninger said. “It's about the ‘why’ of [AI], the ‘what are we going to use [AI] to solve.’ And the problems or needs we are going to solve are not necessarily the next Uber, but the next way of helping people better the environment or better how they communicate or better the democratic principles of their local community. That's the distinctively Notre Dame part of this.”
