Project: Talk2View - The medical viewer you can talk to

[andy9t7]

Draft Status

Ready - team will start page creating immediately

Category

Infrastructure

Key Investigators

• Andy Huynh (Talk2View, Australia)
• Ben Zwick (Talk2View, Australia)
• Kat Dunn (Talk2View, Australia)
• Wilson Wan (Talk2View, Australia)

Project Description

Talk2View helps users use complex medical software using natural language commands. Currently, Talk2View is available as a standalone application built on top of 3D Slicer and a 3D Slicer Extension module.

Objective

1. Objective A. Package and Distribute Talk2View 3D Slicer Extension
2. Objective B. Evaluate Connector Architecture for Community Development
3. Objective C. Web App Feasibility using Trame
4. Objective D. Ask Community for new Features and Connectors

Approach and Plan

Objective A. Package and Distribute Talk2View 3D Slicer Extension

1. Consult the 3D Slicer developer community on the best approach to package and distribute the Talk2View 3D Slicer extension.
2. Package and distribute Talk2View using the recommended approach.

Objective B. Evaluate Connector Architecture for Community Development

1. Consult the 3D Slicer developer community on best practices for designing a connector architecture that integrates with existing extensions with minimal friction.
2. Implement the connector architecture and publish clear documentation so community members can connect their own extensions for the AI agent to use as tools.

Objective C. Web App Feasibility using Trame

1. Speak with Thibault to evaluate Trame’s capabilities and confirm whether delivering Talk2View as a web application is feasible.

Objective D. Ask Community for new Features and Connectors

1. Engage the 3D Slicer community to identify high-value features or connectors that could be added to Talk2View.

Progress and Next Steps

Objective A. Package and Distribute Talk2View 3D Slicer Extension

• Talk2View 3D Slicer extension is currently available on our website.
• Ask Slicer Developers on best approach to package and distribute on the Extension Index.
• Package and distribute using recommended approach.

Objective B. Evaluate Connector Architecture for Community Development

• Community can develop connectors in Talk2View.
• Seek advice on making this process more frictionless.
• Implement into Talk2View 3D Slicer Extension.
• Document the process for community development.

Objective C. Web App Feasibility using Trame

• Set-up meeting with Thibault to talk about Trame's capabilities and if it's feasible to add Talk2View as a web app.
• Create an MVP to showcase and test if it is worth working towards.

Objective D. Ask Community for new Features and Connectors

• Showcase Talk2View to community.
• List features/connectors that would help improve Talk2View.
• Add features/connectors to Talk2View.

Illustrations

Talk2View4SlicerDemo1.mp4

Background and References

No response

[sjh26]

Let me know when this is ready to go

[andy9t7]

Let me know when this is ready to go

Hi Sam, it's ready!

Thanks

[github-actions]

Project Page Pull Request Creation

✅ COMPLETED: See #1656