Jump to content
Medical informatics

WisPerMed team wins international AI competition

Published

Tabea Pakull (l.) and Hendrik Damm

UAS doctoral students Tabea Pakull and Hendrik Damm demonstrated outstanding knowledge and skills in the development of AI programs at this year's international BioNLP competition. As a team from the WisPerMed Research Training Group, they achieved a very good place twice.

"BioNLP" stands for "Biomedical Natural Language Processing", which refers to the processing of biomedical terminology using artificial intelligence. At the Association for Computational Linguistics (ACL) conference of the same name, researchers from all over the world present their latest findings in the field of AI understanding of biological and medical texts. In addition, the ACL organizes several tasks each year for researchers to compete against each other.

Hospital documentation

The "Discharge Me!" task involved hospital documentation, i.e. the complete listing of all examinations and their results as well as treatments and developments from admission to discharge. In everyday hospital life, this documentation is usually extensive and time-consuming for medical staff to compile.

The task was to have AI programs create as much of this documentation as possible using a given data set of 25 discharge letters. The UAS team succeeded better than all 16 international competing teams: Tabea Pakull and Hendrik Damm achieved the highest score and thus first place. The results were not only technically evaluated, but also evaluated and confirmed by three clinicians.

The second task, "BioLaySumm", requires the translation of more than a thousand complex scientific texts using AI - in such a way that they can be understood by non-specialists without compromising scientific precision. 54 international teams took part. As a guide for the evaluation, the organizers had the task completed using a standard AI method and defined the result as a "baseline".

Thousand-text training

They not only have to translate the scientific formulations into understandable language. It also has to recognize where additional explanations are needed to compensate for the lack of specialist and background knowledge among laypeople.

Tabea Pakull and Hendrik Damm tested different AI text models and ultimately combined several in order to achieve the best possible results. They relied exclusively on open source software to ensure the traceability of the results - unlike other teams that used models such as ChatGPT, which are powerful but not fully transparent as commercial products.

They then developed four different approaches for the AI to take to the texts. They had the AI imitate the mindset of a science communicator to process the texts. Another approach was to provide it with step-by-step instructions. A third approach was to give the AI as much freedom as possible to take its own approach. Using these methods, they ran through each individual text.

"Most innovative approach"

This put them in fourth place out of 54 teams. They exceeded the baseline by 5.5 percentage points - the first-placed teams were only a further 1.5 percentage points higher and many teams did not even reach the baseline. They also received the award for the "most innovative approach" for their method of combined models and approaches.

Tabea Pakull and Hendrik Damm were accompanied by Prof. Dr. Christoph M. Friedrich from the Faculty of Computer Science at Fachhochschule Dortmund and transfusion physician Prof. Dr. Peter Horn from Essen University Hospital. Co-authors of the projects are Bahadır Eryılmaz, Helmut Becker, Ahmad Idrissi-Yaghir, Henning Schäfer, Sergej Schultenkämper, Peter A. Horn and Christoph M. Friedrich.

The competition for "BioLaySumm" was led by Tomas Goldsack and Carolina Scarton (both University of Sheffield), Matthew Shardlow (Manchester Metropolitan University) and Chenghua Lin (University of Manchester).

The competition for "Discharge Me!" was led by Justin Xu, Jean-Benoit Delbrouck, Andrew Johnston, Louis Blankemeier and Curtis Langlotz, who all work at Stanford University.

Notes and references

This site uses cookies to ensure the functionality of the website and to collect statistical data. You can object to the statistical collection via the data protection settings (opt-out).

Settings(Opens in a new tab)