A performance versus computational cost of the submitted systems is shown in Figure 2. The best submission by Schmid et al. [1] (source code) had a log loss of 1.091 and an accuracy of 59.6% overtaking the baseline system (1.532 and 44.2%). This submission used a teacher-student approach where a classifier based on state-of-the-art pre-trained audio embedding (PaSST) was used as a teacher for a smaller student model based on RF-regularized CNN. The recording device generalization was enhanced during the system training by using the MixStyle data augmentation technique to mix frequency-wise statistics.

Low-complexity requirements were tackled in the submissions mainly by adapting existing state-of-the-art audio classification approaches (e.g., MobileNets or BC-ResNets) to meet the computational requirements. Another approach was to utilize very involved acoustic feature extraction solutions together with simplified neural network architectures. Generally, a lot of focus in the development was put on the training process and data augmentation strategies. Often used strategies were quantization-aware training and knowledge distillation with bigger pre-trained networks that were fine-tuned on the current task. Top performing systems in the challenge are using neural architectures with large receptive fields, transformer architectures, and coordinate attention to optimizing their performance.