Compress and Compare : Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments

To deploy machine learning models on-device, practitioners use compression algorithms to shrink and speed up models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle chang...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - PP(2024) vom: 10. Sept.
1. Verfasser: Boggust, Angie (VerfasserIn)
Weitere Verfasser: Sivaraman, Venkatesh, Assogba, Yannick, Ren, Donghao, Moritz, Dominik, Hohman, Fred
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM377421774
003 DE-627
005 20240913232912.0
007 cr uuu---uuuuu
008 240911s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2024.3456371  |2 doi 
028 5 2 |a pubmed24n1532.xml 
035 |a (DE-627)NLM377421774 
035 |a (NLM)39255121 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Boggust, Angie  |e verfasserin  |4 aut 
245 1 0 |a Compress and Compare  |b Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 13.09.2024 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a To deploy machine learning models on-device, practitioners use compression algorithms to shrink and speed up models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle changes in model behavior, and negotiating complex accuracy-efficiency trade-offs. However, existing compression tools poorly support comparison, leading to tedious and, sometimes, incomplete analyses spread across disjoint tools. To support real-world comparative workflows, we develop an interactive visual system called COMPRESS AND COMPARE. Within a single interface, COMPRESS AND COMPARE surfaces promising compression strategies by visualizing provenance relationships between compressed models and reveals compression-induced behavior changes by comparing models' predictions, weights, and activations. We demonstrate how COMPRESS AND COMPARE supports common compression analysis tasks through two case studies, debugging failed compression on generative language models and identifying compression artifacts in image classification models. We further evaluate COMPRESS AND COMPARE in a user study with eight compression experts, illustrating its potential to provide structure to compression workflows, help practitioners build intuition about compression, and encourage thorough analysis of compression's effect on model behavior. Through these evaluations, we identify compression-specific challenges that future visual analytics tools should consider and COMPRESS AND COMPARE visualizations that may generalize to broader model comparison tasks 
650 4 |a Journal Article 
700 1 |a Sivaraman, Venkatesh  |e verfasserin  |4 aut 
700 1 |a Assogba, Yannick  |e verfasserin  |4 aut 
700 1 |a Ren, Donghao  |e verfasserin  |4 aut 
700 1 |a Moritz, Dominik  |e verfasserin  |4 aut 
700 1 |a Hohman, Fred  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g PP(2024) vom: 10. Sept.  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:PP  |g year:2024  |g day:10  |g month:09 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2024.3456371  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d PP  |j 2024  |b 10  |c 09