Potentials and Limitations of Active Learning

For the Reduction of Energy Consumption During Model Training

Abstract

This article investigates the potential and limitations of using Active Learning (AL) to reduce AI’s carbon footprint and increase the accessibility of machine learning to low-resource projects. First, this paper reviews the recent literature on sustainable AI. The core of the article concerns AL as an emissions reduction technique. Because AL reduces the required data for model training, one can hypothesize that energy consumption  and, accordingly, carbon emissions – also decreases. This paper tests this assumption. The leading questions concern whether AL is more efficient than traditional data sampling strategies and how we can optimize AL for sustainability. The experiments show that the benefit of AL strongly depends on its parameter settings and the data set size. Only in limited scenarios does the size reduction outweigh the computational costs for AL. For projects with more resources for annotations, AL is beneficial from an ecological perspective and should ideally be paired with model compression techniques. For smaller projects, however, AL can even have a negative impact on machine learning’s carbon footprint.