Brian McFee

Brian McFee brian.mcfee@nyu.edu

0000-0001-6261-9747

Associate Professor of Music Technology and Data Science at New York University

Music and Performing Arts Professions / MARL and Center for Data Science

I develop machine learning tools to analyze music and multimedia data.

For a full history, here's my curriculum vitæ .

Ph.D. Students

Qingyang (Tom) Xi (Music Technology)
Marios Glytsos (Music Technology)
Aramis Tanelus (Data Science)

Alumni:

Dr. Elena Georgieva , 2025 (Music Technology)
Dr. Chris Ick , 2025 (Data Science)
Dr. Morgan Buisson , 2024 (Télécom-Paris)

Books

Digital Signals Theory (2023) [print | online]

Publications

2025

Lose the Frames: Event-based Metrics for Efficient Music Structure Analysis Evaluations

Xi, Q. and McFee, B.

International society for music information retrieval (ISMIR) conference.

2025

Quantifying Regularity in Music Structure Analysis

McFee, B.

International society for music information retrieval (ISMIR) conference.

2025

A dataset of amateur karaoke singing and audiobook narration

Georgieva, E. , Ripollés, P. , and McFee, B.

Audio Engineering Society (AES) International Conference on Artificial Intelligence and Machine Learning

2025

Investigating the Sensitivity of Pre-trained Audio Embeddings to Common Effects

Deng, V. , Wang, C. , Richard, G. , and McFee, B.

International conference on acoustics, speech and signal processing (ICASSP).

2025

Hybrid Losses for Hierarchical Embedding Learning

Tian, H. , Lattner, S. , McFee, B. , and Saitis, C.

International conference on acoustics, speech and signal processing (ICASSP).

2024

Using Pairwise Link Prediction and Graph Attention Networks for Music Structure Analysis

Buisson, M. , McFee, B. , and Essid, S.

International society for music information retrieval (ISMIR) conference.

2024

The Changing Sound of Music: An Exploratory Corpus Study of Vocal Trends Over Time

Georgieva, E. , Ripollés, P. , and McFee, B.

International society for music information retrieval (ISMIR) conference.

2024

Spatial Scaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Rooms

Roman, I.R. , Ick, C. , Ding, S. , Roman, A.S. , McFee, B. , and Bello, J.P.

International conference on acoustics, speech and signal processing (ICASSP).

2024

Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation

Buisson, M. , McFee, B. , Essid, S. , and Crayencour, H.

IEEE Transactions on Audio, Speech and Language Processing

2023

A Repetition-based Triplet Mining Approach for Music Segmentation

Buisson, M. , McFee, B. , Essid, S. , and Crayencour, H.

International society for music information retrieval (ISMIR) conference.

2023

Transfer Learning and Bias Correction with Pre-trained Audio Embeddings

Wang, C. , Richard, G. , and McFee, B.

International society for music information retrieval (ISMIR) conference.

2023

Leveraging Geometrical Acoustic Simulations of Spatial Room Impulse Responses for Improved Sound Event Detection and Localization

Ick, C. and McFee, B.

Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)

2023

Efficient Evaluation Algorithms for Sound Event Detection

Lostanlen, V. and McFee, B.

Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)

2023

Foley Sound Synthesis at the DCASE 2023 Challenge

Choi, K. , Im, J. , Heller, L.M. , McFee, B. , Imoto, K. , Okamoto, Y. , Lagrange, M. , and Takamichi, S.

Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)

2023

Automatic recognition of cascaded guitar effects

Guo, J. and McFee, B.

International conference on digital audio effects (DAFx)

2022

Learning multi-level representations for hierarchical music structure analysis

Buisson, M. , McFee, B. , Essid, S. , and Crayencour, H.

International society for music information retrieval (ISMIR) conference.

2021

Automatic Hierarchy Expansion for Improved Structure and Chord Evaluation

Kinnaird, K. and McFee, B.

Transactions of the International Society for Music Information Retrieval

2021

Sound Event Detection in Urban Audio With Single and Multi-Rate PCEN

Ick, C. and McFee, B.

International conference on acoustics, speech and signal processing (ICASSP).

2021

Multi-Task Self-Supervised Pre-Training for Music Classification

Wu, H.H. , Kao, C.C. , Tang, Q. , Sun, M. , McFee, B. , Bello, J.P. , and Wang, C.

International conference on acoustics, speech and signal processing (ICASSP).

2021

Interactive Learning of Signal Processing Through Music: Making Fourier Analysis Concrete for Students

Müller, M. , McFee, B. , and Kinnaird, K.

IEEE Signal Processing Magazine

2020

Audio-Based Music Structure Analysis: Current Trends, Open Challenges, and Applications

Nieto, O. , Mysore, G.J. , Wang, C. , Smith, J.B.L. , Schlüter, J. , Grill, T. , and McFee, B.

Transactions of the International Society for Music Information Retrieval

2020

Multiple F0 estimation in vocal ensembles using convolutional neural networks

Cuesta, H. , McFee, B. , and Gómez, E.

International society for music information retrieval (ISMIR) conference.

2020

Entrofy your cohort: a transparent method for diverse cohort selection

Huppenkothen, D. , McFee, B. , and Norén, L.

PLOS ONE

2020

Learning the helix topology of musical pitch

Lostanlen, V. , Sridar, S. , McFee, B. , Farnsworth, A. , and Bello, J.P.

International conference on acoustics, speech and signal processing (ICASSP).

2019

Improving structure evaluation through automatic hierarchy expansion

McFee, B. and Kinnaird, K.

International society for music information retrieval (ISMIR) conference.

2019

Voice anonymization in urban sound recordings

Cohen-Hadria, A. , Cartwright, M. , McFee, B. , and Bello, J.P.

International workshop on machine learning for signal processing (MLSP).

2019

Enhanced hierarchical music structure annotations via feature level similarity fusion

Tralie, C.J. and McFee, B.

International conference on acoustics, speech and signal processing (ICASSP).

2019

A music structure informed downbeat tracking system using skip-chain conditional random fields and deep learning

Fuentes, M. , McFee, B. , Crayencour, H. , Essid, S. , and Bello, J.P.

International conference on acoustics, speech and signal processing (ICASSP).

2019

Open source practices for music signal processing research

McFee, B. , Kim, J.W. , Cartwright, M. , Salamon, J. , Bittner, R.M. , and Bello, J.P.

IEEE Signal Processing Magazine

2019

Per-channel energy normalization: why and how

Lostanlen, V. , Salamon, J. , Cartwright, M. , McFee, B. , Farnsworth, A. , Kelling, S. , and Bello, J.P.

IEEE Signal Processing Letters

2018

Adaptive pooling operators for weakly labeled sound event detection

McFee, B. , Salamon, J. , and Bello, J.P.

IEEE Transactions on Audio, Speech and Language Processing

2018

OpenMIC-2018: An open dataset for multiple instrument recognition

Humphrey, E. , Durand, S. , and McFee, B.

19th International Society for Music Information Retrieval (ISMIR) conference

2018

Analysis of common design choices in deep learning systems for downbeat tracking

Fuentes, M. , McFee, B. , Crayencour, H. , Essid, S. , and Bello, J.P.

19th International Society for Music Information Retrieval (ISMIR) conference

2018

Bubble cooperative networks for identifying important speech cues

Trinh, V.A. , McFee, B. , and Mandel, M.

InterSpeech

2017

Evaluating hierarchical structure in music annotations

McFee, B. , Nieto, O. , Farbood, M. , and Bello, J.P.

Frontiers in Psychology

2017

Structured training for large-vocabulary chord recognition

McFee, B. and Bello, J.P.

18th International Society for Music Information Retrieval (ISMIR) conference

2017

Deep salience representations for F0 estimation in polyphonic music

Best student paper award

Bittner, R.M. , McFee, B. , Salamon, J. , Li, P. , and Bello, J.P.

18th International Society for Music Information Retrieval (ISMIR) conference

2017

Statistical methods for scene and event classification

McFee, B.

Computational Analysis of Sound Scenes and Events

2016

resampy: efficient sample rate conversion in python

McFee, B.

The Journal of Open Source Software

2016

A plan for sustainable MIR evaluation

McFee, B. , Humphrey, E. , and Urbano, J.

17th International Society for Music Information Retrieval (ISMIR) conference

2015

A software framework for musical data augmentation

ISMIR 2025 10-year test of time runner up

McFee, B. , Humphrey, E. , and Bello, J.P.

16th International Society for Music Information Retrieval (ISMIR) conference

2015

Hierarchical evaluation of segment boundary detection

McFee, B. , Nieto, O. , and Bello, J.P.

16th International Society for Music Information Retrieval (ISMIR) conference

2015

librosa: Audio and Music Signal Analysis in Python

McFee, B. , Raffel, C. , Liang, D. , Ellis, D.P.W. , McVicar, M. , Battenberg, E. , and Nieto, O.

14th annual Scientific Computing with Python conference (SciPy)

2014

Analyzing song structure with spectral clustering

Best oral presentation award

McFee, B. and Ellis, D.P.W.

15th International Society for Music Information Retrieval (ISMIR) conference

2014

mir_eval: a transparent implementation of common MIR metrics

Best poster presentation award

Raffel, C. , McFee, B. , Salamon, J. , Humphrey, E. , Nieto, O. , Liang, D. , and Ellis, D.P.W.

15th International Society for Music Information Retrieval (ISMIR) conference

2014

Codebook-based audio feature representation for music information retrieval

Vaizman, Y. , McFee, B. , and Lanckriet, G.R.G.

IEEE Transactions on Audio, Speech and Language Processing

2014

Speech enhancement by low-rank and convolutive dictionary spectrogram decomposition

Chen, Z. , McFee, B. , and Ellis, D.P.W.

Interspeech

2014

Learning to segment songs with ordinal linear discriminant analysis

McFee, B. and Ellis, D.P.W.

International conference on acoustics, speech and signal processing (ICASSP)

2014

Better beat tracking through robust onset aggregation

McFee, B. and Ellis, D.P.W.

International conference on acoustics, speech and signal processing (ICASSP)

2013

Iterative category discovery via multiple kernel metric learning

Galleguillos, C. , McFee, B. , and Lanckriet, G.R.G.

International Journal of Computer Vision

2013

Robust structural metric learning

Lim, D.K.H. , McFee, B. , and Lanckriet, G.R.G.

30th International Conference on Machine Learning (ICML)

2012

Hypergraph models of playlist dialects

McFee, B. and Lanckriet, G.R.G.

13th International Society for Music Information Retrieval (ISMIR) conference

2012

How significant is statistically significant? The case of audio music similarity and retrieval

Urbano, J. , Downie, J.S. , McFee, B. , and Schedl, M.

13th International Society for Music Information Retrieval (ISMIR) conference

2012

The Million Song Dataset Challenge

McFee, B. , Bertin-Mahieux. T. , Ellis, D.P.W. , and Lanckriet, G.R.G.

4th International Workshop on Advances in Music Information Research (AdMIRe)

2012

Learning content similarity for music recommendation

McFee, B. , Barrington, L. , and Lanckriet, G.R.G.

IEEE Transactions on Audio, Speech and Language Processing

2011

The natural language of playlists

Best poster presentation award

McFee, B. and Lanckriet, G.R.G.

12th International Society for Music Information Retrieval (ISMIR) conference

2011

Large-scale music similarity search with spatial trees

McFee, B. and Lanckriet, G.R.G.

12th International Society for Music Information Retrieval (ISMIR) conference

June, 2011

From region similarity to category discovery

Galleguillos, C. , McFee, B. , Belongie, S. , and Lanckriet, G.R.G.

IEEE conference on Computer Vision and Pattern Recognition (CVPR)

February, 2011

Learning multi-modal similarity

McFee, B. and Lanckriet, G.R.G.

Journal of Machine Learning Research (JMLR)

February, 2011

Contextual object localization with multiple kernel nearest neighbor

McFee, B. , Galleguillos, C. , and Lanckriet, G.R.G.

IEEE Transactions on Image Processing (TIP)

2010

Learning similarity from collaborative filters

McFee, B. , Barrington, L. , and Lanckriet, G.R.G.

11th International Society for Music Information Retrieval (ISMIR) conference

2010

Collaborative filtering based on P2P networks

Koenigstein, N. , Lanckriet, G.R.G. , McFee, B. , and Shavitt, Y.

11th International Society for Music Information Retrieval (ISMIR) conference

2010

Metric learning to rank

McFee, B. and Lanckriet, G.R.G.

Twenty-seventh International Conference on Machine Learning (ICML)

2010

Multi-class object localization by combining local contextual interactions

Galleguillos, C. , McFee, B. , Belongie, S. , and Lanckriet, G.R.G.

IEEE conference on Computer Vision and Pattern Recognition (CVPR)

2009

Heterogeneous embedding for subjective artist similarity

Best presentation award

McFee, B. and Lanckriet, G.R.G.

10th International Society for Music Information Retrieval (ISMIR) conference

2009

Partial order embedding with multiple kernels

McFee, B. and Lanckriet, G.R.G.

Twenty-sixth International Conference on Machine Learning (ICML)

Software

autopool: automatic pooling for multiple instance learning
crema: convolutional and recurrent estimators for music analysis
pumpp: practically universal music pre-processor
pescador: stream sampling for iterative learning algorithms
amen: algorithmic music remixing.
resampy: efficient audio resampling in Python.
muda: Musical Data Augmentation.
JAMS: a JSON Annotated Music Specification. v0.2 technical report
Ordinal LDA: Python (sklearn) implementation of ordinal linear discriminant analysis.
librosa: A python package for music and audio signal analysis.
MLR: MATLAB implementation of metric learning to rank.
Hypergraph playlists: Python implementation of the model from this paper.
Spatial trees: Python implementation of spatial trees for approximate nearest neighbor search, as used in this paper.

Data

Open-MIC 2018: Audio and partial instrument annotations for 20,000 10-second clips
MSD Challenge: Large-scale music recommendation on the Million Song Dataset. See also the year 1 test set.
AotM-2011: Annotated playlists from Art of the Mix, indexed to the Million Song Dataset.
AotM-2003: An earlier collection of playlists from Art of the Mix, also indexed to the Million Song Dataset.
aset400 kernels: Kernel matrices for aset400 artist similarity experiments
eHarmony: Matchings and anonymized features for several hundred thousand eHarmony users.

Teaching

F23, F24: (NYU) DS-GA 1006: Capstone Project in Data Science
F20: (NYU) DS-GA 3001: Search and Discovery
S19, S20, S21, S22, S23: (NYU) DS-GA 1004: Big Data
F18, F19, F20, S21, F21, F22, F23, F24: (NYU) MPATE-GE 2599: Fundamentals of Digital Signals Theory I
S14: (Columbia) EECS E6891: Reproducing computational results