Yue Zhu, Chen Wang, et al.
MASCOTS 2024
Writing thread safe code for concurrent processing requires experience and training, thus legacy research code are usually single threaded, which post a challenge when it comes to scaling. This challenge is harder to overcome in the digital health domain, where code changes might trigger a regulatory review process. In this work, we report a solution of leveraging container technology to convert single threaded legacy code into cloud native services, scaling out data processing throughput via data parallelism. We tested the setup with a batch data processing job on the EverythingALS dataset and obtained 8X speed up compared to single threaded processing. This solution is built as part of IBM Health Guardian, a digital health tool suite. It is generalizable and can be adapted to other projects. The work greatly improves the automation of adoption of legacy research code in the evolving digital health domain. It will attract more open domain research contributions.
Yue Zhu, Chen Wang, et al.
MASCOTS 2024
Haoran Qiu, Weichao Mao, et al.
USENIX ATC 2023
Apoorve Mohan, Matthew Sheard
NVIDIA GTC 2022
Marcelo Amaral, Tatsuhiro Chiba, et al.
CLOUD 2022