Data Parallelism: How to Train Deep Learning Models on Multiple GPUs

The NVIDIA AI Technology Center at the University of Florida is offering an instructor-led, deep learning institute workshop in April: Data Parallelism: How to Train Deep Learning Models on Multiple GPUs.

Workshop Dates: April 11-12, 2024 (Thursday and Friday), from 1:00-5:00 p.m.

Registration Link:

The workshop will be held over two days (four hours each day) in Malachowsky Hall’s NVIDIA Auditorium. Its focus is on techniques for data-parallel deep learning training on multiple GPUs to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, attendees will learn how to decrease model training time by distributing data to multiple GPUs, while retaining the accuracy of training on a single GPU. The full course outline may be found on this NVIDIA website page.

The course is FREE and open to the university community, but pre-registration is required. Also required is experience with Python. Technologies used in the workshop are PyTorch, PyTorch Distributed Data Parallel, and NCCL.

If you have any questions about this workshop, please email the instructor, NVIDIA Data Scientist Yungchao Yang (


Supporting UF’s Campus Master Plan

UF’s campus master plan directs the use and strategy for university buildings and land resources. When new buildings are proposed or existing buildings are renovated, two teams from UFIT–Construction Management and A/V Design & Installation–work with UF’s Planning, Design, and Construction.

“Our department relies on UFIT’s expertise to drive the success of every project we have on campus,” says UF Construction Director Frank Javaheri. “With their involvement from the concept during the programming phases, UFIT provides the information necessary for budgetary requirements, procurement of telecom and A/V equipment, and coordination and installation. Partnering with UFIT, we can provide the best systems the university needs for our day-to-day activities, education, research, and other advancements.”

UF PD&C sends the plans for UFIT evaluation. UFIT’s Construction Management team evaluates the project’s telecommunications budget, confirms the plans have adequate space for telecommunication rooms, and evaluates the proposed location for conflicts with existing underground telecommunications assets. UFIT’s A/V Design & Installation team, integrated into campus construction projects for the last decade, works with PD&C on all projects with an audio and/or video technology component. Staff meet with stakeholders to determine their needs and ensure a high-quality product at the best cost for the university. A longer story of UFIT’s involvement and commitment to the campus construction process, with focus on the design and build of Malachowsky Hall, is online.