Date: Friday, June 20, 2025, 10:00-14:30, Cyprus time
Venue: This training event is held as a hybrid event. You are welcome to join us at the Andreas Mouskos Seminar Room, The Cyprus Institute. Otherwise please, connect to our live stream of the discussion, available on Zoom (Password: VsSCz1)
Language: English
Registration: Registration for this event is open until Wednesday, June 18, 2025. Registration form here.
Pre-requisites: Attendees should be familiar with Python. Some previous experience with PyTorch will also be useful. Hands on exercises are part of the training and will be provided in Python.
Requirements: Attendees should bring with them their own laptop (with Administrative privileges) to follow the hands-on practical. They should make sure the following software is available on their laptops:
- A web browser and PDF viewer and
- A command line interface or other client that supports SSH.
Attendees should be able to access the machine prior to the event. Instructions on how to generate the SSH key can be found at Accessing and Navigating Cyclone Tutorial. We recommend generating the SSH key via Git Bash for Windows (2.5.2. Section – Option 2 on the tutorial)
Agenda
09:30-10:00
Hands-on Setup (Optional)
Please use this session to ensure you can access the HPC system.
10:00-11:00
Prof. Constantine Dovrolis
Efficient Training, Fine-Tuning and Inference of Large-Scale ML Models
This tutorial explores model-centric techniques for improving the training and inference efficiency of generative AI models, particularly large language models (LLMs). We cover key methods such as quantization, pruning, knowledge distillation, low-rank approximations, and mixed-precision training. Techniques for efficient fine-tuning (e.g., LoRA, adapters, QLoRA) and inference (e.g., KV-cache, speculative decoding) are also presented. The tutorial addresses the robustness–accuracy trade-off through advanced data augmentation and hybrid models. We conclude with emerging directions including quantum computing applications and efficient architectures like Mixture-of-Experts.
11:00-12:00
Assoc. Prof. Mihalis Nicolaou
Towards Interpretable, Controllable, and Efficient AI: From SVD to Multilinear Mixture of Experts
Large-scale, often generative models (i.e., foundation models) have recently emerged as a dominant paradigm. Utilizing broad-data pre-training at scale, such models have proven more robust and generalizable to alternatives, albeit being considered more opaque due to their sheer scale and complexity. In this talk, we will discuss some of our recent works aiming to make large-scale deep networks more interpretable, controllable, and efficient – for example, by enabling unsupervised local editing in pre-trained networks, making fine-tuning of large models to new tasks efficient, scaling sub computations to achieve specialization with the multilinear mixture of experts, and grounding visual variability to concepts in vision-language models.
Break
12:30-14:30
Dr. Christodoulos Stylianou, Mr. Giorgos Kosta
Fine-tuning AI Models with Multi-GPU Parallelism
In this hands-on session, participants will apply the concepts and tools introduced previously to fine-tune a large language model on an HPC cluster using multiple GPUs. Working through real-world examples, attendees will gain experience using distributed training frameworks such as DDP, FSDP, and DeepSpeed’s ZeRO, and will implement parameter-efficient fine-tuning with techniques like LoRA. The session is designed to be practical and interactive, with step-by-step guidance to help participants get comfortable with the technical workflow, code integration, and resource management on a supercomputer.
About the speaker:
Mihalis A. Nicolaou is Associate Professor at the Computation-based Science and Technology Research Center at The Cyprus Institute. Previously, he has held positions at Imperial College London and the University of London. He received the B.Sc. degree from the University of Athens, Greece, and the M.Sc. and Ph.D. degrees from the Department of Computing, Imperial College London, U.K. His research is focused on developing machine/deep learning algorithms with favourable properties, such as efficiency, interpretability, and generalization, and is interested in impactful applications ranging from computer vision to climate science and health.
Prof. Constantine Dovrolis is the Director of the center for Computational Science and Technology (CaSToRC) at The Cyprus Institute (CyI). He is also a Professor at the School of Computer Science at the Georgia Institute of Technology. His research is highly inter-disciplinary, combining Network Theory, Data Mining and Machine Learning. More recently, his group has been focusing on neuro-inspired architectures for machine learning based on what is currently known about the structure and function of brain networks.
Christodoulos (Chris) Stylianou joined the Computation-based Science and Technology Research Center (CaSToRC) at The Cyprus Institute (CyI) as a Research Engineer for the EuroCC2 Project. He holds an undergraduate degree in Electrical & Electronic Engineering (MEng) from Imperial College London, a Masters Degree in High Performance Computing (HPC) with Data Science (MSc) and a PhD in HPC, Computational & Data Science and Software Engineering from EPCC at The University of Edinburgh.
Chris specializes in High Performance Computing and more specifically in Heterogeneous and Distributed Computing, Performance Portability, and the application of Artificial Intelligence in accelerating Sparse Linear Algebra for the next generation hardware architectures. His current research interests are around acceleration of training time via sparsification of Neural Networks. Chris is also skilled in software development, training, and fostering collaboration between academia, industry, and government with a growing interest in product management.
Giorgos is a Research Engineer at the Computation-based Science and Technology Research Center (CaSToRC) of The Cyprus Institute, working on the EuroCC2 Project. He earned his undergraduate degree (BSc) in Computer Science from the University of Leeds. As part of the EuroCC2 Engineering Team, Giorgos is actively involved in communication and dissemination activities to promote EuroCC2 and raise awareness in the Cypriot scientific and industrial communities about the transformative potential of High-Performance Computing (HPC). His current work focuses on leveraging HPC and High-Performance Data Analytics (HPDA) in applied domains such as Geospatial data analysis and Magnetic Resonance Imaging (MRI), with a particular emphasis on integrating Artificial Intelligence (AI) techniques to drive innovation in these areas.