Learning and adapting with OSG: Investigating the strong nuclear force

Connor Natzke’s journey with the OSG Consortium began in 2019 as a student of the OSG User School. Today, nearly three years later, Natzke has executed 600,000 simulations with the help of OSG staff and prior OSG programming. These simulations, each of them submitted as a job, logged over 135,000 core hours provided by the Open Science Pool (OSPool). Natzke’s history with the OSG Consortium reflects a pattern of learning, adapting, and improving that translates to the acceleration and expansion of scientific discovery. During the March OSG All-Hands Meeting 2022, Natzke was presented the David Swanson Memorial Award, which recognized him for his dedication and tenacity since joining the OSG Community.

Natzke is a Ph.D. student at the Colorado School of Mines and is currently located at TRIUMF, a particle physics laboratory in Vancouver, British Columbia. Natzke’s research focuses on the strong nuclear force, a fundamental force in nature that keeps protons and neutrons bound together in a cohesive unit at the center of an atom. This force exists at subatomic scales. Therefore, Natzke and his team require something quite large to observe it –– the GRIFFIN spectrometer. Standing at over ten feet tall, GRIFFIN can measure the angle between photons emitted from an unstable atomic nucleus located at the center of the instrument. This angle reveals important information about nuclear structure, but Natzke relies on numerous simulations to unveil the whole picture.

Because the gamma-ray detectors that make up GRIFFIN have limits to their measurement capabilities, Natzke and his team use a Monte Carlo simulation package called GEANT4 to reconstruct the angle between the emitted photons more precisely. This simulation involves mapping a large parameter space –– an energy surface –– of individual photon energies. Forty-one combinations of photon energies are needed to make one of these maps and three simulations are run for each of these combinations, with each requiring one billion simulated events. The resulting time required to make just one energy surface map is fifty thousand core hours or roughly five years and nine months if Natzke was relying simply on his laptop’s computational power.

“With standard computation, this quickly becomes an intractable project,” Natzke explains. “Luckily, I attended the OSG User School in 2019 and learned that Monte Carlo simulations are essentially the poster child for distributed high-throughput computing.”

With Monte Carlo simulations, one simulation of one billion events produces results equivalent to one million simulations of one thousand events. This unique quality transforms what would otherwise be a single lengthy and time-consuming job into many short and quick jobs that can be scaled out to run in a high-throughput environment. As Natzke sums it up, “It’s frankly beautiful how easily this works.”

With the help of OSG Research Computing Facilitation Lead Lauren Michael, Natzke used a personal meta-scheduler for HTCondor called DAGMan (Directed Acyclic Graph Manager) to automate his workflow. He wrote python scripts that created and submitted the DAG file to automate the process further. In total, this workflow took roughly 24 hours to produce one of 41 points on the energy surface map. Before using DAGMan, each point took one week.

But Natzke didn’t stop there. In 2021, he attended the OSG All-Hands Meeting and learned about Pegasus, an HTCondor-integrated workflow system that is offered by OSG’s Access Points. With support from OSG Facilitator and Pegasus developer, Mats Rynge, Natzke remodeled his workflow using Pegasus to improve file management, transfers, and error handling. The additional automation that Natzke had written around his DAGMan workflow was already provided by Pegasus, and it was enhanced. Natzke humbly jokes, “It’s written by computer scientists, rather than physicists masquerading as computer scientists.” His resulting workflow only takes three commands and finishes in merely four hours, a forty-fold increase compared to Natzke’s capabilities before OSG services.

With this new workflow, Natzke can expand upon what’s possible in terms of his research: “Every time I run this, I’m amazed at how much time and effort I’ve saved, and just the pure automation and capacity that I have access to with OSG. It’s just mind-blowing to me.”

…

The OSG David Swanson Award was established to honor our late colleague and chair of the OSG Consortium, David Swanson. David contributed to campus research across the country by advancing distributed high-throughput computing (dHTC) and the OSG. Learn more about David’s legacy and past recipients of his namesake award.

Watch a video recording of Connor Natzke’s presentation at the OSG All-Hands Meeting 2022, and browse his slides.