Fig. 1: Patient video recording and processing. | npj Digital Medicine

Fig. 1: Patient video recording and processing.

From: Hospitalization prediction from the emergency department using computer vision AI with short patient video clips

Fig. 1

a Video recording was conducted using secured mobile devices and lasted ~5 min, with the camera at the base of the patient’s bed, and the patient as upright as tolerated. Video recording did not interfere with patient care, and was paused if the clinical team needed to interact with the patient. Videos were spot-checked regularly to ensure adherence to study protocol. The figure was created with BioRender.com. b The video recording (did not include audio) was then processed via the ImageBind video processing pipeline, which involves uniformly sampling up to five 2-s clips at 1 frame per second and taking three spatial crops (left, middle, right) per clip. Then, these fifteen spatially cropped clips are passed into the vision encoder and the resulting output is the mean of the clips’ 1024-dimensional representations. (Subject in the figure is a co-author, not patient).

Back to article page