Introduction

The web was created to facilitate information-sharing between universities and institutes. Initially, it was a medium to exchange simple documents with links and basic formatting. After decades of development and innovation, modern browsers can play high-quality video streams, complex animations, games, and simulations and perform other computationally intensive tasks natively without using any third-party plugins. The possibility of plugin-free native application execution inside browsers came with the 2014 release of the asm.js specification [1]. Asm.js was introduced as a strict subset of JavaScript to be used as a low-level target language for compilers. It allows the browsers to perform ahead-of-time optimizations on the code and to run it with superior performance to unoptimized JavaScript. The de-facto compiler toolchain for asm.js is Emscripten [2]. It can compile C/C++ code bases by converting Low Level Virtual Machine (LLVM) [3] bitcode to asm.js. An early demonstration of the viability and ease of use of the toolchain was the porting of Unreal Engine 3 to the web, which was developed as a collaboration of Mozilla and Epic Games in 4 days [4]. The successor to asm.js is WebAssembly, which first appeared in 2017 [5]. It has similar goals to asm.js, most importantly to allow running computationally intensive programs in the browser, but instead of being a subset of JavaScript, it is a binary instruction format for a stack-based virtual machine. WebAssembly is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications. Emscripten supports WebAssembly too, and changing the compilation target to WebAssembly instead of asm.js is straightforward.

Besides games and multimedia software, medical applications also benefited from modern-day web innovations. Although medical software systems, such as Health Information Systems (HIS), have been on the web for a long time, it was only after the introduction of the fifth major version of the HyperText Markup Language (HTML5) [6] the Web Graphics Library (WebGL) [7] and WebAssembly that computationally more involved medical programs, such as radiological software tools, started to appear on the web.

One such example is cornerstone.js [8], a framework that allows viewing Digital Imaging and Communications in Medicine (DICOM) [9] images inside the browser. It consists of a collection of libraries, each responsible for a specific task, like decoding DICOM images, connecting to a Picture Archiving and Communication System (PACS), or annotating medical images. For medical education-related use-case, Biodigital Human [10] is worth mentioning. It is an interactive 3D platform where one can explore, customize, and share 3D views of human anatomy, physiology, diseases, and treatments.

The neuroimaging community also started to adopt web technologies, an example of which is BrainBrowser [11], a web-based project that enables visualizing surface and volumetric data. The WebGL-based Surface Viewer is capable of rendering 3D surfaces in real-time, whereas the Volume Viewer uses HTML5 canvas to allow slice-by-slice traversal of volumetric data in Neuroimaging Informatics Technology Initiative (NIfTI) and various other neuroimaging formats.

Post-processing of neuroimaging data is often needed for accurate visualization. As an example, in the case of the simultaneous visualization of two MRI volumes of the same patient acquired at different times, overlaying the two on top of each other might not be sufficient because of the slightly different positions of the patient during the two image acquisitions. Linear registration of the two volumes is needed so that the same location on the two images refers to the same voxel (Fig. 1). This way, the two images can be overlaid on top of each other for more accurate evaluation. For linear registration to work robustly, another processing step has to be performed on the two input volumes. This is called brain extraction, which is the method of deleting non-brain tissue from an image of the whole head.

View Full Size

Linear registration – An SWI MRI volume (first image) is linearly registered to a reference T1-weighted MRI volume (second image). Since the patient position was different during the image acquisitions, the registration algorithm has to calculate a transformation matrix that aligns the input volume to the reference volume. The final image shows the corrected input volume superimposed on the reference volume

Citation: Imaging 15, 1; 10.1556/1647.2023.00111

• Download Figure
• Download figure as PowerPoint slide

The most widespread tools for performing the aforementioned image processing techniques on neuroimaging data are the Brain Extraction Tool (BET) [12] and the FMRIB Linear Image Registration Tool (FLIRT) [13, 14] of the FMRIB Software Library (FSL) [15]. Another widely used software to perform linear registration is Slicer 3D [16], which focuses more on a graphical user interface-based approach, whereas the components of the FSL library are mainly used as command-line tools. Slicer 3D is also capable of performing brain extraction, however it is available as an extension and is not part of the software by default.

To the best of our knowledge, there is no fully client-side web-based neuroimaging workflow that supports brain extraction and linear registration. Our goal is to create a web-based, cross-platform, extensible neuroimaging platform that works in any modern web browser and does not require a web server for the computations. We aim to combine the visualization capabilities of BrainBrowser VolumeViewer, and the image processing power of FSL by porting FSL BET and FLIRT to WebAssembly and integrating them into BrainBrowser through an extensible plugin system.

Methods

Software architecture

Our project consists of three main parts:

1. ●the porting of the brain extraction (BET) and linear registration (FLIRT) FSL tools to WebAssembly
2. ●the integration of the ported tools into BrainBrowser through a plugin system
3. ●creating a demo application to showcase multiple neuroimaging workflows using the ported tools

We created a version of FSL that can run on the web using a tool called Emscripten. Our approach involved customizing the software by selectively including only the components required for executing BET and FLIRT. These changes allowed us to generate WebAssembly programs that can be used to run BET and FLIRT in a web browser. To enhance user experience, we used web workers to run the tools, this way the browser window remains responsive while the program performs the computation. From a user perspective, the ported tools behave exactly like their native command-line versions, i.e., they can receive and parse the same command-line arguments, and they generate output files in the same manner.

We integrated our ported tools into BrainBrowser (Fig. 2), which allows visualization of NIfTI volumes through its VolumeViewer module.

View Full Size

Extending BrainBrowser - WebMRI defines a plugin system on top of BrainBrowser VolumeViewer, and adds two additional volume loaders: “blend” and “dicom”. The plugin system handles automatic GUI generation (“buildControlPanel”) and runs the web workers (“runPluginWorker”) that invoke the WebAssembly binaries

Citation: Imaging 15, 1; 10.1556/1647.2023.00111

• Download Figure
• Download figure as PowerPoint slide

Results

The entry point of our demo application showcasing WebMRI is the dashboard (Fig. 3). There is a navigation bar on the top with menu items “File”, “Tools”, and “About”. The “File” menu hosts actions such as opening DICOM series, a NIfTI volume, or creating a blended volume (“Create overlay”) out of two input volumes. The “Create overlay” functionality is an important last step in a linear registration workflow, as it simultaneously visualizes the two volumes registered to each other. The “Tools” menu shows the list of the plugins available in the application (BET and FLIRT), and through the “About” menu, users can access the developer documentation and user manual of WebMRI.

View Full Size

The WebMRI dashboard - The image shows the demo application screen with the navigation bar on top, the panel section on the left, and the rendering view on the right. The rendering view is provided by BrainBrowser VolumeViewer. The red crosshair shows the currently selected voxel, and the coordinates are shown in the “Volume controls” widget

Citation: Imaging 15, 1; 10.1556/1647.2023.00111

• Download Figure
• Download figure as PowerPoint slide

Under the navigation bar, we present the main screen, which is divided into the rendering section on the right, and the panel section on the left. The rendering section hosts the BrainBrowser VolumeViewer widget, which visualizes the loaded volume in sagittal, axial, and coronal planes. The panel section next to it hosts the “Workspace” and “Volume controls” widgets.

The “Workspace” widget (Fig. 4) shows the files currently opened in the application. These files can either be loaded by the user (input files) or generated by plugins (output files). The workspace has a root folder, which is a collection of files that are visible to plugins as input files and plugin-specific folders, into which plugins generate their output files. If the users want to further process output files, they first have to move the relevant files to the root folder by clicking on the blue up arrow next to the file names. This way, the selected files will be visible to the plugins. The “Volume controls” widget contains coordinate information, blending controls, and windowing sliders.

View Full Size

The workspace widget - It contains the main “root” folder and the plugin-specific folders. Volumes can be moved to the “root” folder and can be deleted. Only volumes in the “root” folder are visible to the plugins

Citation: Imaging 15, 1; 10.1556/1647.2023.00111

• Download Figure
• Download figure as PowerPoint slide

Our demo application showcases our two ported FSL tools. They can be accessed from the “Tools” menu. Figure 5 shows the automatically generated user interface for BET. The input volumes can be selected from a drop-down menu. The user is presented with a list of modifiable parameters, which will translate to command-line arguments by the plugin system when the “Run” button is clicked, and the underlying tool is invoked.

View Full Size

Plugins – Automatically generated plugin UI for the ported FSL BET brain extraction tool

Citation: Imaging 15, 1; 10.1556/1647.2023.00111

• Download Figure
• Download figure as PowerPoint slide

The most practical use case our system is capable of supporting in its current form is the multi-modal assessment of neuroimaging data. As an example, let us look at how cerebral microbleeds can be detected in an MRI volume. The most widespread way of analyzing microbleeds is to leverage susceptibility weighted imaging (SWI). Although SWI can help identify microbleeds with high sensitivity, their localization is challenging relying solely on this modality. T1-weighted MRI images preserve the anatomical features of the brain, so using the T1 modality in combination with SWI can help with localization. WebMRI can co-register an SWI to a T1 MRI volume thus enabling precise microbleed detection. To enhance registration result brain extraction can also be performed. Since our system can handle DICOM to NIfTI conversion, the only step the user has to perform before loading data into our system is to export the DICOM series from the PACS.

Discussion

With WebMRI, we built a cross-platform, extensible neuroimaging platform that supports brain extraction and linear registration and can be run in any modern web browser without using third-party browser plugins or a dedicated web server. For brain extraction, we used the BET, and for linear registration, the FLIRT tool of the popular neuroimaging library collection FSL. For the porting process, we used Emscripten to convert the native programs to WebAssembly binaries. We extended BrainBrowser with a plugin system to be able to load our ported tools into the system combining volumetric visualization with image processing. We built a demo application that incorporates the ported tools and demonstrates that a complete neuroimaging workflow from DICOM loading to linear registration can be performed inside a web browser.

Building on the foundations of BrainBrowser, we increased the scope of browser-based neuroimaging and showed that not only neuroimaging data visualization but computationally intensive processing algorithms could also be performed inside modern web browsers. In contrast to Slicer 3D (to which our system is similar in terms of modularity) or other not browser-based programs, our solution does not require an installation. By removing the need for a web server, and complex setup processes, we reduced the cost and improved the ease of use of certain neuroimaging workflows. These factors can improve the clinical adoption of new image-processing tools. Although we demonstrated the porting of two popular neuroimaging tools, there is no limit to bringing other algorithms into the WebMRI plugin system as long as they are written in a programming language that can be compiled to WebAssembly. As more and more programming languages used in computationally intensive application development (such as Rust) support compilation to WebAssembly, the number of libraries that can be ported to WebMRI will increase.

The limitation of our fully browser-based system is twofold. The memory usage of heavy workloads, such as processing large numbers of volumes in bulk, might exceed the resources available inside a browser tab. There is also a performance penalty for the porting due to the overhead of the WebAssembly runtime compared to fully native execution. The impact of these limitations varies between programs and browsers and can be mitigated using optimizations in the Emscripten toolchain or the original codebase. If these factors make it not possible to reliably run a given tool in the browser, it can still be run in a web server since Emscripten supports execution in a Node.js runtime.

In the future, we plan to increase our fleet of ported neuroimaging tools and make our plugin system more modular and extensible. We aim to make it easier to integrate our system into a clinical PACS environment, so there will be no need to export DICOM files. We also plan to support hybrid execution so that a given algorithm can be run on the client or server side, depending on the requirements of the given workload. We plan to improve the visualization capabilities of WebMRI by adding support for customizable volume viewer widgets and annotation of points of interest on the volumes.

Ethical statement

The study submitted to IMAGING have been conducted in accordance with the Declaration of Helsinki and according to requirements of all applicable local and international standards.

Authors' contribution

AH – Conceptualization, Software architecture design, Software development, Software testing, Drafting of the manuscript, Preparing the Figures. FK – Conceptualization, Software testing, Revision. SSz – Conceptualization, Software testing, Revision. TD – Conceptualization, Software testing, Revision. PB – Conceptualization, Software testing, Revision. AT – Conceptualization, Product design, Software testing, Revision. All authors reviewed the final version of the manuscript and agreed to submit it to IMAGING for publication.

Funding sources

No financial support was received for this study.

Conflict of interests

The authors have no conflict of interest to disclose.