Powering and Cooling HiPerGator: The UF Data Center

HiPerGator, the University of Florida supercomputer, is housed in the UF Data Center (UFDC). While its power and ranking as the most powerful supercomputer in U.S. higher education is well known, not many people know about the components at the UFDC that help keep HiPerGator online and cooled.

Backup Batteries

HiPerGator and the other computers housed in the UFDC, along with the chilled water pumps and air handlers, are run by high-power batteries. These batteries ensure that the computers get clear power without spikes or brown-outs. There is enough power available in the UFDC to keep all systems operating for about 10 minutes after an external utilities power failure. During those 10 minutes, UFDC diesel generators begin providing continuing power. The diesel generator and the chillers cool their water to 55F to send to the air handlers, which then cool the air that is used to cool the computers.

Air Exchange

To get fresh air throughout the UFDC and avoid sick-building syndrome, 10% of the air inside the data hall is constantly replaced with outside air, which is cleaned by removing particles and living mold and spores.

UF Data Center Generators

The UFDC has two generators. One has a horse-power capacity of 2.25 MW and produces 1 MW of electricity if the utilities’ power becomes unavailable. A second, similar 4 MW diesel produces the remaining 2.2 MW of electricity to provide the full 3.2 MW that the UFDC is rated for.

Transparent Floor Tiles

The HiPerGator room has a raised floor of about three feet. This is because the mostly empty space is needed to allow cold air to be delivered to the front of the computers. The fans inside the computers blow the cold air past the hot CPUs, with the hot air being returned through the ceiling to the air handlers in hallways outside the 5000 sq. ft. HiPerGator room.

Air Handlers

Speaking of the air handlers, they blow hot air past the radiators that have 55F water flowing through them. All 125,000 cubic feet of air in the HiPerGator data hall must be replaced twice every minute to avoid HiPerGator overheating! The ideal temperature for the HiPerGator room? It is 60F.

Even with the cooling requirements for a supercomputer, HiPerGator is ranked high up on the worldwide green-500 computing list, and the UF Data Center is a certified LEED® building. Learn more about HiPerGator here.

Getting Started with HiPerGator

To assist researchers and instructors in getting started with HiPerGator, UFIT produced a series of videos that explain the processes for setting up a HiPerGator account, training and support for UF’s high-performance computing environment, and using HiPerGator in undergraduate courses:

Getting Started with HiPerGator

Teaching with HiPerGator

UFIT also has a video explaining what ResVault is. That system can be used for computing on highly regulated data like export controlled data. HiPerGator is also certified to allow working with PHI if the proper procedure is followed.

Our Research Computing staff look forward to meeting you and enabling your line of inquiry. You’ll find many additional resources on the https://rc.ufl.edu/ website to help you begin your journey in UF’s high-performance computing environment, and staff are available for in-person and online consultations as needed to fit your schedule. Please contact Senior Director Erik Deumens if you have any questions about getting started with HiPerGator and our campus’s research computing ecosystem.

Proposal Support Available for Researchers

UFIT is committed to doing everything possible to enable the research community. In addition to providing high performance computing consulting, our staff can help make your sponsored funding proposals more competitive with letters and templates related to the computing and infrastructure needs of your project. There are four components that UFIT’s Research Computing staff can assist with:

1. The budget form showing the cost of computing services or hardware acquisition.
2. Some funding agencies require a commitment letter from Research Computing. We will be happy to write a letter for your specific project.
3. A description of the facilities in support of your project is available to include in the proposal’s facilities section.
4. An explicit data management plan (DMP) is required by most funding agencies, including the National Science Foundation and the National Institutes of Health. Research Computing staff are happy to assist you in developing a DMP and its associated budget.

The Research Computing website also has examples of justification text for Hardware Acquisition and Consulting Services. All of the templates and sample text mentioned above are available on https://www.rc.ufl.edu/research/proposal-support/. Please contact Research Computing Director Erik Deumens if you need assistance with your proposal’s computing infrastructure documentation.

Launching UF’s AI Initiative

On July 21, the University of Florida announced its partnership with NVIDIA, the multi-national technology company that is the world leader in artificial intelligence (AI). This $70 million initiative will enable UF researchers to solve global challenges, provide unprecedented access to the most powerful AI tools, transform how UF educates its students, and help create the workforce for tomorrow.

The opportunities to profoundly impact the world and lead the way in research and discovery will be powered by HiPerGator and the supercomputing environment built and managed by UFIT. Along with recent enhancements to storage and facilities improvements, UFIT will complete the installation of the NVIDIA GPU system (NVIDIA DGX SuperPOD™ architecture) and design and deploy other required components to support the HiPerGator 3.0 go-live scheduled for January, 2021.

More information about this first-of-its-kind partnership for AI in higher education are available via the following resources:

https://ai.ufl.edu/
UF Press Release
NVIDIA Partnership Announcement

UF Data Center Grand Opening

On May 7, UF administrators, research faculty, Dell executives, and Gainesville community leaders gathered at the East Campus to celebrate the grand opening of the UF Data Center. The event marked the unveiling of one of the most powerful university-based supercomputers in the country: HiPerGator.

“The UF Data Center is home to the fastest supercomputer in Florida, enabling UF and the state to recruit the brightest scholars from around the world,” said Vice President and CIO Elias Eldayrie. “Our expanded computing capacity will reap global dividends scientifically and strengthen Florida as a hub for technology and innovation.”

HiPerGator will attract the best faculty and post-doc candidates because of the supercomputer’s ability to process ‘big data’. The computational speed of HiPerGator is an astounding 150 trillion calculations per second. What does this mean? Data is processed faster, allowing faculty to analyze research and move/share findings with top institutions around the world. Before HiPerGator, UF professor David Ostrov needed months to conduct drug tests. Now, computations can be done in just hours.

“HiPerGator can help drugs get from the computer to the clinic more quickly. We want to discover and deliver safe, effective therapies that protect or restore people’s health as soon as we can,” Ostrov said.

Check out the HiPerGator fact sheet to learn more about the first supercomputer in the state of Florida! Additional information is available on the Research Computing Web page or by contacting UF Research Computing.