Scalability, Generality and Temporal Aspects in Automatic Recognition of Predominant Musical Instruments in Polyphonic Music

In this paper we present an approach towards the classification of pitched and unpitched instruments in polyphonic audio. In particular, the presented study accounts for three aspects currently lacking in literature: model scalability to polyphonic data, model generalisation in respect to the number of instruments, and incorporation of perceptual in formation. Therefore, our goal is a unifying recognition framework which enables the extraction of the main in struments' information. The applied methodology consists of training classifiers with audio descriptors, using exten sive datasets to model the instruments sufficiently. All data consist of real world music, including categories of 11 pitched and 3 percussive instruments. We designed our descriptors by temporal integration of the raw feature val ues, which are directly extracted from the polyphonic data. Moreover, to evaluate the applicability of modelling tem poral aspects in polyphonic audio, we studied the perfor mance of different encodings of the temporal information. Along with accuracies of 63% and 78% for the pitched and percussive classification task, results show both the impor tance of temporal encoding as well as strong limitations of modelling it accurately.