DST: Simplifying the HPC Cluster Conundrum

The heritage of the 15-year-old analytics firm, Data in Science Technologies (DST) is deeply entwined with providing data management solutions. However, in the past five years, DST has transitioned into a consulting company to equip their clients with the know-how of the technology world, enabling them to make informed decisions. DST is governed by a vision to simplify the complex areas of HPC clusters for its customers. This organizational motto stemmed from the lack of skilled professionals in the HPC ecosystem, which had handicapped the growth of many institutions. While the skill-sets to tread the HPC path are expensive and hard to find, DST sought to put an end to this significant crunch with its gamut of consulting solutions. The firm’s on-demand services allow organizations to hire an HPC-skilled professional, which is complemented by DST’s consulting services designed to deal with the intricacies of project development. Besides, the firm’s HPC Operational Management System (HOMS) assists organizations in monitoring and managing the convoluted HPC clusters. All in all, DST’s solutions portfolio is uniquely architected to augment the efficiencies of organizations treading the difficult terrains of HPC.

In spite of building state-of-the-art infrastructure, many organizations fail to resolve the complexities of an HPC cluster. For instance, a New York-based research institution that had moved to a new GPFS cluster, confronted a unique challenge of running few scripts on the new cluster. These scripts were written in an old programming language and refused to transition into the new environment. Even after brainstorming various solutions, which was corroborated by a full-blown research team, a definitive answer to the conundrum did not materialize. In the course of time, DST’s assistance was sought, and within two weeks the research organization was able to run all the workflows in the new GPFS cluster. “The reason behind our successful endeavors is the team that we employ. We have individuals who are veterans in the HPC ecosystem with experiences ranging from 15 to 20 years,” says Debbie Willbanks, Sr. Partner at DST. One of our clients stated to us “We always had people on staff here on premises that were running the system and were responsible for running the system. And that was a weakness for us because really our specialty was doing research, that’s where we add value.” We think that says it all and why DST exists, Willbanks adds.

Akin to the turnkey portfolio of solutions, DST deploys an organized onboarding framework to assimilate new clients seamlessly. To begin with, an intensive discovery and documentation phase of the HPC ecosystem takes place with key stakeholders, wherein the DST team documents each aspect of the current state—from software, hardware, and networking, applications and their dependencies, file systems, workflow, data archiving, and data retention. Frontline managers thoroughly study these aspects and are tasked to understand the requirements and nuances of new clients, whereas a remote monitoring team espouses the management of the organization’s cluster. DST has a three-pronged approach that encircles understanding, monitoring, and providing their customers with robust support options in order to ensure a smooth transition into DST’s solutions. Additionally, every cluster policy that DST formulates is properly documented, ensuring accountability of the entire process.