– Written By CAMIS Project team, within the Data Visualisation & Open Source Technologies Working Group
The CAMIS (Comparing Analysis Method Implementations in Software) Working Group project is a cross-industry initiative run in collaboration with members from PHUSE, PSI and the ASA.
Several discrepancies have been discovered in statistical analysis results between programming languages, even in fully qualified statistical computing environments. The goal of CAMIS is to investigate and document the source of these discrepancies to reinstate confidence in reliability.
Novartis are embracing the use of open-source programming languages for clinical trial analysis and reporting. As part of this journey, we were internally troubleshooting and documenting discrepancies we observed when comparing methodological implementations. When we learned about CAMIS, we saw the opportunity to both contribute to and learn from a knowledge repository.
To facilitate the adoption of CAMIS, we focused on three main objectives:
- Raise awareness about the CAMIS project within our organisation.
- Contribute our findings insights to the CAMIS repository.
- Prevent tools such as git from becoming obstacles.
To meet these goals, we organised an Open-Source in Action Hackathon and invited the CAMIS chairs to collaborate with us. We timed the hackathon with our Summer Rejuvenation period, where meeting loads are reduced for two weeks. In preparation for the hackathon, we conducted training sessions on git and GitHub, which attracted over 100 participants.
We worked with CAMIS leads to provide guidance on high-priority contributions. During the hackathon, we held daily office hours to troubleshoot any issues, and we used a Kanban board to keep track of who was working on what and to encourage collaboration.
During the hackathon, a diverse group of participants from statistics, programming and software development collaborated to contribute to CAMIS. Participants had the opportunity to interact with others who shared their interests and work together to develop solutions in an inclusive and supportive atmosphere.
The hackathon facilitated 25+ contributions while also giving participants hands-on experience with open-source tooling with expert support. These contributions vary in complexity from single-language implementation of a statistical test to a multi-language comparison of more complex methodology such as covariate adjustment for logistic regression. We observed a reasonably even split between SAS and R contributions and a handful of Python contributions.
This hackathon has increased awareness of CAMIS at Novartis and further encouraged and enabled our participants to contribute their knowledge and experience to the open-source community.
For CAMIS, the Novartis Hackathon created a spike in contributions, with the volume and range of complexity surpassing our expectations. A few hackathon participants have since become CAMIS volunteers, continuing their passion for contributing to this industry collaborative project. It’s nice to see the CAMIS repository gaining traction amongst industry, societies and regulatory.
Congratulations to the hackathon participants, the Novartis leadership team for spearheading this, and the CAMIS team for their continued support.