Open science is supported by the eScience Center. One of our eScience Coordinators (Adriënne Mendrik) was co-organizer and presenter at a tutorial on designing benchmarks and challenges for measuring algorithm performance in biomedical image analysis at the 20th International Conference on Medical Image Computing and Computer Assisted Intervention (September 14th, 2017 in Quebec City, Quebec, Canada - view the article in Computer Vision Magazine).
Challenges are open online competitions, for which data and evaluation metrics are made publicly available to get insight into algorithm performance for a specific problem.
Adriënne Mendrik and Stephen Aylward presented a theoretical framework to help guide challenge design and redefine the objective of grand challenges, to either gain insight (insight challenge) or solve a problem (deployment challenge).
There is to date no long-term solution that fully facilitates the sustainability of grand challenges in the biomedical image analysis field