Skip to main content

Future Directions

MRI / CT Registration

MRI/CT registration is an established area, so the initial approach should be to test existing algorithms and libraries rather than develop a registration method from scratch.

A reasonable workflow is:

  1. Start with rigid registration for translation and rotation.
  2. Optionally refine with affine or deformable registration.
  3. Use a multimodal similarity measure such as mutual information or shared anatomical features.
  4. Transform a CT-derived segmentation into MRI space.
  5. Use the transformed segmentation to define a region of interest in the MRI.

Existing tools such as ANTsPy are worth evaluating.

Sinus Symmetry / Anomaly Detection

Explore whether left-right sinus comparison can contribute to anomaly detection.

Initial research did not identify much work specifically using bilateral sinus symmetry as the anomaly signal.

Normal sinuses can also have substantial left/right asymmetry, so the contralateral side cannot simply be assumed to represent normal anatomy.

  • Normal sinus asymmetry and anatomical variation
  • Supervised classification of segmented maxillary-sinus volumes
  • Unsupervised anomaly detection using autoencoders trained on healthy sinus volumes
  • Self-supervised approaches using residual maps to highlight abnormalities
  • Automatic MRI sinus segmentation and extraction of volume, intensity, and opacification features

Collaborative Surgical Planning

Explore OperSight as a collaborative surgical-planning/review environment, including surgeon-radiologist collaboration and remote planning.

Skin Graft Modeling

Explore workflows for planning/modeling skin grafts.

This would likely benefit from more precise cutting geometry and higher spatial resolution around selected regions.

Higher-Precision Cutting

Explore using a finer voxel grid or higher-resolution representation within a selected region of interest.

The goal would be to support smaller, more precise, and flexible cut geometry than is practical with the current source scan resolution.

Tissue-Specific Transfer Functions

Explore additional MRI and CT transfer functions designed to emphasize specific tissues or structures for targeted visualization and cutting.

MedGemma / AI-Assisted Imaging

Explore integrating Google MedGemma as an AI layer within OperSight.

MedGemma 1.5 supports 3D CT/MRI interpretation and can work with DICOMweb-based imaging workflows, potentially making it a useful candidate for the existing OperSight and Orthanc workflow.

Possible Applications​

Potential uses within OperSight include:

  • Scan-aware Q&A and anatomical explanations
  • Resident training and interactive case review
  • Generating summaries or educational context from imaging and saved Plan data
  • Reviewing a user-created Surgical Pathway and surfacing relevant anatomical considerations

Initial Prototype​

The first experiment should be small and focused.

  1. Select a de-identified ENT CT.
  2. Make the scan available through the existing Orthanc/DICOMweb workflow.
  3. Test sending the imaging data to MedGemma.
  4. Ask scan-related questions.
  5. Evaluate the usefulness and relevance of the responses.
  6. Use the results to determine whether deeper OperSight UI integration is worth exploring.

Initial Role in OperSight​

MedGemma should initially be treated as an assistive and educational AI layer.

The initial integration should focus on:

  • Explanations
  • Educational context
  • Case review
  • Scan-aware assistance

It should not initially be used to:

  • Make surgical decisions
  • Produce authoritative surgical geometry
  • Produce authoritative segmentations

The goal of the first prototype is to evaluate whether MedGemma provides useful imaging context before investing in deeper integration with OperSight.

Last updated on by jacobgqc