Pre-Conference Talk by YANG Chengran and YANG Zhou

Talk #1: Answer Summarization for Technical Queries: Benchmark and New Approach
by YANG Chengran, PhD Candidate

We find that existing approaches to generate an answer summary for a given technical query in Software Question and Answer (SQA) sites are assessed solely through user studies. In this work, we manually construct a high-quality benchmark to enable automatic evaluation of answer summarization for technical queries for SQA sites. Using the benchmark, we comprehensively evaluate the performance of existing approaches and find that there is still a big room for improvements.

Motivated by the results, we propose a new approach TechSumBot with three key modules. We evaluate TechSumBot in both automatic (i.e., using our benchmark) and manual (i.e., via a user study) manners. The results from both evaluations consistently demonstrate that TechSumBot outperforms the best-performing baseline approaches from both SE and NLP domains by a large margin. Additionally, we also conducted an ablation study, which demonstrates that each module in TechSumBot contributes to boosting the overall performance of TechSumBot.

Talk #2: Compressing Pre-trained Models of Code into 3 MB
by YANG Zhou, PhD Candidate

Although large pre-trained models of code have delivered significant advancements, they consume hundreds of megabytes of memory and run slowly on personal devices, causing problems in deployment and greatly degrading the user experience.

It motivates us to propose Compressor, a novel approach that can compress the pre-trained models of code into extremely small models with negligible performance sacrifice. Compressor proposes a genetic algorithm (GA)-based strategy to guide the simplification process. We design a GA algorithm to maximize a model’s Giga floating-point operations (GFLOPs), under the constraint of the target model size. Then, we use knowledge distillation to train the small model. We evaluate Compressor with two state-of-the-art pre-trained models on two important tasks: vulnerability prediction and clone detection. Results show that Compressor can simplify pre-trained models to 3 MB (a size that is 160× smaller) with negligible compromise of prediction accuracy.

YANG Chengran is a Ph.D. candidate at SMU SCIS, supervised by Prof. David Lo. His research focuses on automatic summarization in software engineering.

YANG Zhou is a Ph.D. student and research engineer in the School of Computing and Information Systems, supervised by Prof. David LO. Zhou currently focuses on MLOps, trying to make ML systems robust, safe, ethical, and trustworthy. Zhou walks on the streets and freezes memorable moments with his Fujifilm X100 camera.