Globus Compute: Federated FaaS for Integrated Research Solutions
The ever-increasing amount of data produced by scientific experiments and simulations makes it essential to employ HPC resources to extract meaningful insights from the raw data. HPC enables researchers to perform simulations, modeling, and analysis, which are critical to predicting outcomes, guiding experiments, and developing new technologies [1]. However, the pervasive use of HPC presents new challenges as researchers must familiarize themselves with complex environments, batch schedulers, and authentication schemes.
Globus Compute [2] is a Function-as-a-Service platform designed to provide a scalable, secure, and simple interface to HPC resources. Globus Compute implements a federated model via which users may deploy endpoints on arbitrary remote computers, from the edge to high performance computing (HPC) cluster, and they may then invoke Python functions on those endpoints via a reliable cloud-hosted service. Unlike traditional cloud-hosted FaaS platforms (e.g., Amazon Lambda) and open-source platforms (e.g., OpenWhisk) [3], Globus Compute combines a single cloud-hosted web service with an ecosystem of edge endpoints. Recently, Globus Compute released a multi-user endpoint, designed to be deployed and managed by system administrators. These endpoints secure access to resources and provide fine-grained auditability, while trivializing access for users.
In this talk, we will discuss how Globus Compute can be used to simplify and federate access to heterogeneous HPC resources. We will discuss the design, implementation, and deployment of Globus Compute and discuss how multi-user endpoints can be used by administrators to securely manage system access. Additionally, we will showcase how Compute can be used to enable integrated research solutions that span institutions.
ABOUT THE AUTHOR
Ryan Chard joined Argonne National Laboratory in 2016 as a Maria Goeppert Mayer Fellow. He is currently a Consultant at Argonne focusing on the development of cyberinfrastructure to accelerate scientific research. In particular, he works on tools to streamline data analysis pipelines and develops integrated research solutions that incorporate HPC resources into scientific processes. He has a Ph.D. in computer science and an M.S. from Victoria University of Wellington, New Zealand.
-----
For more information about the eResearch NZ / eRangahau Aotearoa conference, visit:
https://eresearchnz.co.nz/
