Joining forces with BMW and Google

From 24 to 27 June 2024, researchers from NordSTAR at OsloMet met leading researchers and experts from the BMW Research Group (bmwgroup.com) and Google Cloud (cloudonair.withgoogle.com) in Munich, Germany.

Here they put their heads together to explore ground-breaking developments in artificial intelligence, machine learning and "Big Data".

As told by Professor Pedro Lind, NordSTAR and OsloMet AI Lab, the visit had several important highlights:

• insightful presentations from NordSTAR researchers on its new AI Trustworthiness scale, reliability of artificial intelligence, cyber security, and ongoing projects with Eye-Tracking data
• fascinating lectures by BMW experts on "Knowledge Graph"-enhanced large language models, reliability assessment and innovative topic modelling
• an engaging workshop about generative artificial intelligence organized by Google Cloud, with discussions on trust and transparency in artificial intelligence, and a demo of Google Cloud Gemini

Reliability and AI trustworthiness in focus

The experts from Google and BMW emphasized the critical importance of assessment methods for trustworthy artificial intelligence ("AI Trustworthiness") in today's rapidly developing technological landscape.

This is something that NordSTAR has been working on, and one of the most recent achievements was the validation of a user’s trustworthiness scale for AI tools and algorithms. 

NordSTAR was thus able to engage in fruitful discussions that could lay the foundation for future collaboration with these two companies.

“This is about exploring new applications for our artificial intelligence trustworthy questionnaire, assessing joint research projects, and laying the foundations for cooperation to promote artificial intelligence in the automotive industry”, says Pedro Lind.

Great interest in the AI Trustworthiness scale NordSTAR has developed

The background for the contact with BMW is that Pedro Lind met PhD Simon Schramm at a summer school in 2022. He organizes activities with all the 160 PhDs in machine learning at BMW worldwide, and then he told him about the NordSTAR centre at OsloMet.

He was very curious to know more about how to assess the trustworthy of artificial intelligence tools, and he heard about the method NordSTAR had developed.

“We have kept in touch, and when we validated the AI Trustworthiness scale, we have developed, we shared the results with him. He then invited us to BMW in Munich, for a meeting with the rest of his group.”

Can become important partners

“I think BMW can become an important partner, where we can test our AI Trustworthiness scale on various artificial intelligence tools that they are developing, for example for self-driving cars.”

“BMW has its own research and development department, but so far, they have not focused so much on assessment tools for the trustworthiness of artificial intelligence. A collaboration on this can therefore be fruitful for both parties.”

Lind points out that the scale NordSTAR has developed is able to show the level of trustworthiness of artificial intelligence tools from the user's perspective.

“And we now want to combine this with other aspects of trustworthiness such as objective performance, explainability, reliability, security and privacy.”

“All this contributes to trust and trustworthiness, but how to assess them specifically in the context of artificial intelligence is still an open problem.”

Common problems provide opportunities

“We understood that BMW and Google have exactly the same problems as everyone else, that they want their tools to be trustworthy, but that they don't know how to evaluate them in a standardized way.”

“And that is exactly what NordSTAR focuses on, developing meaningful frameworks that can assess the trustworthiness of artificial intelligence tools.”

This research activity is very interdisciplinary, involving both technical competence, in for example the development of algorithms, as well as social science competence in, for example, human interaction with computer tools.

“During the first months of 2025, NordSTAR will organize a final workshop about its ongoing activities, before we submit an application for a national a centre of research excellence (SFF), and then we will of course invite our partners from BMW and Google.”

“We have now also started a white paper, i.e. an article stating NordSTAR positioning and the needs and unanswered questions within the trustworthiness and sustainability assessment of artificial intelligence tools.”

There will also be a discussion about the need to develop the assessment tool for the trustworthiness of artificial intelligence.

“I think this is a good opportunity for cooperation. It is a win-win situation for us in OsloMet, in NordSTAR, and for BMW and Google in Germany.”

“It has also turned out that BMW was interested in our FRIPRO project, Virtual-Eye, within which we develop methods for eye-tracking data analysis which we then apply, not only in the field of intelligent health, but also in the development of new search strategies in autonomous search engines.”

Particularly about the latter, BMW researchers became very interested, and a couple of them are planning to visit us during the autumn to get to know our eye tracking lab and our research activities in eye tracking better.”

Can bring big gains

Collaborating with large, multinational companies can bring great benefits, Lind believes.

“I think this could be decisive for us now that we are really starting to write applications for large projects and will apply to become a centre for outstanding research next year.”

“I also believe that it will increase our chances when we want to apply for large external funds in research, and that it can contribute to developing OsloMet as a university.

“And regardless of the outcome, I think it was very nice to see interest from both sides in continuing to develop cooperation between us, BMW and Google. I think it also opens the doors to involve young PhD and master's students, and postdocs, in collaboration with the non-academic sectors. It can, for example, increase their chances of being employed in these large companies after completing their studies or doctorate.”

This also applies to teaching activities, says Lind:

“As we know a university can only be a university, if it keeps focus and investment on both teaching and research. I think NordSTAR and the OsloMet AI Lab are contributing significantly to both. As a teacher, I also believe that having these non-academic companies as partners in research activities will also attract students who want to develop their skills.”

In Munich, in addition to Professor Pedro Lind, other OsloMet staff took place in the visiting team: 

• senior adviser Vera Østensen, who is the coordinator of OsloMet's commitment to outstanding research environments; 
• Researcher II Yuri Kasahara from NIBR and NordSTAR's partner, who was involved in the development and validation of NordSTAR AI Trustworthiness scale; 
• Professor Lothar Fritsch, expert in cybersecurity; 
• postdoctoral researcher Tumaini Kabudi, who also contributed to the development of the new scale

The team is now about to submit an article introducing and describing the scale.

The picture at the top of the article shows some of those who met at BMW in Munich. From top-down and left-right: Tumaini Kabudi, Karl Volkenandt (BMW), Hugo Math (BMW), Yuri Kasahara, Vera Østensen, Nadja Sauter (BMW), Osman Aksu (BMW) and Pedro Lind.