Skip to content

Unable to reproduce CNN scores #2

Description

@ciaran-coleman

Hello,

I am attempting to reproduce your results but am seeing differences in the WSI CNN count results. Example WSI NA5001_2AB:

  • My results. Cored: 41, Diffuse: 288, CAA: 649
  • Your results. Cored: 7, Diffuse: 410, CAA: 595

All packages I have matched, e.g. version 3.4.1 of libopencv/opencv/py-opencv, version 0.3.0 of torch, pyvips 2.1.2 with libvips 8.2.2. I should say that I am on a Windows 10 machine so this could very well be the problem, but just wanted to check first that I haven't missed something else.

When running notebook 2.2) I get the same AUC-PRC scores and so it seems the model is working correctly and giving the correct confidence.

I have also checked the stain normalization is working as intended by comparing 256x256 patches from the training data set provided in the Zenodo link to the equivalent cropped images produced when processing the WSIs with notebooks 1.1)-1.3). So I believe the WSIs are correctly stain normalized.

Would it be at all possible to see your prediction confidence heatmap for the example above so that I can inspect? I suspect this is where the differences lie that lead to the discrepancy.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions