Pareto Multi-Task Learning

Xi LIN, Hui-Ling ZHEN, Zhenhua LI, Qingfu ZHANG, Sam KWONG

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

122 Citations (Scopus)

Abstract

Multi-task learning is a powerful method for solving multiple correlated tasks simultaneously. However, it is often impossible to find one single solution to optimize all the tasks, since different tasks might conflict with each other. Recently, a novel method is proposed to find one single Pareto optimal solution with good trade-off among different tasks by casting multi-task learning as multiobjective optimization. In this paper, we generalize this idea and propose a novel Pareto multi-task learning algorithm (Pareto MTL) to find a set of well-distributed Pareto solutions which can represent different trade-offs among different tasks. The proposed algorithm first formulates a multi-task learning problem as a multiobjective optimization problem, and then decomposes the multiobjective optimization problem into a set of constrained subproblems with different trade-off preferences. By solving these subproblems in parallel, Pareto MTL can find a set of well-representative Pareto optimal solutions with different trade-off among all tasks. Practitioners can easily select their preferred solution from these Pareto solutions, or use different trade-off solutions for different situations. Experimental results confirm that the proposed algorithm can generate well-representative solutions and outperform some state-of-the-art algorithms on many multi-task learning applications.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems, 32 : 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
EditorsHanna M. WALLACH, Hugo LAROCHELLE, Alina BEYGELZIMER, Florence D'ALCHÉ-BUC, Edward A. FOX, Roman GARNETT
PublisherNeural Information Processing Systems Foundation
Number of pages11
ISBN (Print)9781713807933
Publication statusPublished - Dec 2019
Externally publishedYes
Event33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 - Vancouver Convention Center, Vancouver, Canada
Duration: 8 Dec 201914 Dec 2019
https://nips.cc/Conferences/2019

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Conference

Conference33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019
Abbreviated titleNeurIPS 2019
Country/TerritoryCanada
CityVancouver
Period8/12/1914/12/19
Internet address

Bibliographical note

This work was supported by the Natural Science Foundation of China under Grant 61876163 and Grant 61672443, ANR/RGC Joint Research Scheme sponsored by the Research Grants Council of the Hong Kong Special Administrative Region, China and France National Research Agency (Project No. A-CityU101/16), and Hong Kong RGC General Research Funds under Grant 9042489 (CityU 11206317) and Grant 9042322 (CityU 11200116).

Fingerprint

Dive into the research topics of 'Pareto Multi-Task Learning'. Together they form a unique fingerprint.

Cite this