Table 1 Task completion rates of multi-agent collaboration system (MACS) on 21 tasks: comparison of the basic version (text input), +retrieval augmented generation (RAG) module (text input), and +voice modules (voice input). User confirmation and the memory module were disabled during this experiment.

From: Proof of concept for voice based MRI scanner control using large language models in real time guided interventions

  

Task completion rate\(^\textrm{a}\)

Task (example command)

Related Tools and Perceptrons

baseb

+RAGb

+RAGc

+RAG+Voicec

Ask the current field of view (FOV) size.

getFieldOfViewRead

\(100\%\)

\(100\%\)

/

\(100\%\)

Set the FOV to 500 mm.

setFieldOfViewRead

\(100\%\)

\(100\%\)

/

\(100\%\)

Expand or reduce the FOV size.

get-, setFieldOfViewRead

\(100\%\)

\(100\%\)

/

\(100\%\)

Display the position of the 1st slice group.

getSlicePositionPcs

\(100\%\)

\(100\%\)

/

\(100\%\)

Set the 1st slice group to (10, -10, 0).

setSlicePositionPcs

\(100\%\)

\(100\%\)

/

\(100\%\)

Move 1st slice group on random patient coordinate system (PCS) axes.

get-, setSlicePositionPcs

\({\mathbf {8\%}}^{*}\)

\({\mathbf {80\%}}^{*}\)

/

\(80\%\)

- on one PCS axis.

get-, setSlicePositionPcs

/

\(100\%\)

\(100\%\)

/

- on two PCS axes.

get-, setSlicePositionPcs

/

\(76\%^{*}\)

\(100\%^{*}\)

/

- on three PCS axes.

get-, setSlicePositionPcs

/

\(56\%\)

\(84\%\)

/

- AVERAGE

get-, setSlicePositionPcs

/

\({\mathbf {77.3\%}}^{*}\)

\({\mathbf {94.7\%}}^{*}\)

/

List all templates available for a remote control.

getTemplates

\(100\%\)

\(100\%\)

/

\(100\%\)

Open the template for beat sequence\(^{d}\).

openTemplate

\(100\%\)

\(100\%\)

/

\(80\%\)

Save the changes and close the template.

closeTemplate

\(100\%\)

\(100\%\)

/

\(100\%\)

Start one pre-opened sequence.

startSequence

\(96\%\)

\(100\%\)

/

\(100\%\)

Start one non-pre-opened sequence.

get-, openTemplate, startSequence

\(100\%\)

\(100\%\)

/

\(100\%\)

Stop the sequence immediately.

stopSequence

\(100\%\)

\(100\%^{*}\)

/

\({\mathbf {60\%}}^{*}\)

Pause the currently active sequence temporarily.

pauseSequence

\(100\%\)

\(100\%\)

/

\(100\%\)

Restart the paused sequence.

continueSequence

\(96\%\)

\(100\%\)

/

\(100\%\)

Try to gain operational control over the host.

requestControl

\(100\%\)

\(100\%\)

/

\(100\%\)

Reverse the control of the host.

releaseControl

\(100\%\)

\(100\%\)

/

\(80\%\)

Move the table to position 450.

moveTable, Scanner Table State

\(96\%\)

\(100\%\)

/

\(80\%\)

Move the table a little out.

moveTable, Scanner Table State

\({\mathbf {44\%}}^{*}\)

\({\mathbf {84\%}}^{*}\)

\({\mathbf {100\%}}\)

\(100\%\)

Check the editing option of the open template.

Sequence State

\(100\%\)

\(100\%\)

/

/

Show the status of the currently running templates.

Sequence State

\(100\%\)

\(100\%\)

/

/

Check if the control can be rescheduled.

Host Control State

\(100\%\)

\(100\%\)

/

/

Total

\({\mathbf {92.4\%}}\)

\({\mathbf {98.2\%}}\)

\({\mathbf {99.7\%}}^{*}\)

\({\mathbf {93.3\%}}^{*}\)

  1. \(^{a}\) For each task, gpt-4-0125-preview was used to generate 25 semantically similar commands for repeated testing with text input (on base and +RAG version). The commands used in the voice test were 5 randomly selected commands from the 25.
  2. \(^{b}\) MACS driven by gpt-4-0125-preview.
  3. \(^{c}\) MACS driven by gpt-4o-2024-05-13.
  4. \(^{d}\) The ’beat’ sequence, a product of Siemens, is an MR imaging sequence used to track the needle in real time. Fisher’s exact test: * \(P < 0.05\). Untested commands (marked with ‘/’) are explained in the Methods–Evaluation section.
  5. Significant values are in bold.