Technology
mufin uses mathematical extraction of specific details from a music title to create an objective desciption of the title’s characteristics which is independent of human influences. Using this description, songs which are similar to one another can be filtered out of a large database (mufin – MusicFinder).
In addition, standardized information is generated for the music, and this information will enable a targeted search for musical characteristics for the next generation of search machines – similar to today’s full text search for text documents. Development of such technologies for media annotation and semantic netwoking of musical content drives mufin GmbH together with the Fraunhofer Institut and the German National Library.
In the scope of the Theseus research program initiated by the Federal Ministry for Economy and Technology (BMWi), mufin GmbH is developing technologies for the field of media annotation and semantic music networking.
What is the advantage compared to human annotation of details?
- Low time requirements
- Objectivity (free of personal bias or listening habits)
- Comparability (two people will never perceive the same piece of music exactly the same)
- Large database possible (not limited to the mainstream; music can also be found by artists which one hasn’t heard before)
- Online and offline / locally useable
- Low cost
How does the mufin system work?
First, mufin makes an analysis of the music piece based on the signal level; the temporal progress as well as the frequency range is observed. Details are derived from this section and used to map specific characteristics of the audio signal, and these can be directly or indirectly relevant for human perception.
The result of our research has been successful in finding those details and characteristics which are relevant for musical similarity and for forming a model with them. In this model, similarities can be calculated which in sum allow similarities between pieces to be determined.
What does parametrization mean?
Parametrization means that the user can decide which musical aspects they prefer and which are less important. The mufin system reacts to this by considering respective aspects more or less intensively during the similarity search. This method makes a personalized search possible. Parametrization can either occur explicitly via respective settings options, or implicitly via a self-teaching system.
How do holistic and aspect-based similarity searches differ?
The holistic search compares all details for the complete music piece with one another to provide a rigid benchmark of the similarity of two songs. The aspect-based approach allows the user to adjust the search to suit their requirements, i.e. by changing the weighting of intuitive parameters (e.g. intensity of the rhythm).
Which musical aspects are considered by the similarity search?
Among others, rhythmical characteristics (e.g. intensity of the rhythm, tempo, percussion in the piece), sound colour, harmonic and melodic qualities are weighted differently for the result.
Can the actual audio material recreated from the fingerprint itself?
No.
How large is a mufin fingerprint?
A fingerprint currently requires around 3 kB (independent of the length of the audio file).
How long does extraction of a music title take?
Extraction from an average song currently requires about 2-10 seconds (industry standard PC, including decoding of mp3 format).
What is the MPEG-7 standard?
MPEG-7 is an ISO standard which has been defined by the Moving Picture Experts Group (MPEG). It serves for the description of multimedia data with the help of meta information. MPEG-7 (in contrast to MPEG-1, MPEG-2 and MPEG-4) is not engaged with a compression standard for video or audio.
mufin GmbH and Fraunhofer IDMT work actively on the MPEG-7 standard, and have, among others, introduced their low-level1 features to the standard. The use of the technology by mufin guarantees (in contrast to proprietary, non-standardized processes) simple data exchange, e.g. for future search machines.