[me]

Brian McFee brian.mcfee@nyu.edu

ORCID iD icon 0000-0001-6261-9747

Assistant Professor of Music Technology and Data Science at New York University

Music and Performing Arts Professions / MARL and Center for Data Science

I develop machine learning tools to analyze music and multimedia data.

For a full history, here's my curriculum vitæ .

Ph.D. Students

Alumni:

Publications

2025
Investigating the Sensitivity of Pre-trained Audio Embeddings to Common Effects
Deng, V. , Wang, C. , Richard, G. , and McFee, B.
International conference on acoustics, speech and signal processing (ICASSP).
2025
Hybrid Losses for Hierarchical Embedding Learning
Tian, H. , Lattner, S. , McFee, B. , and Saitis, C.
International conference on acoustics, speech and signal processing (ICASSP).
2024
Using Pairwise Link Prediction and Graph Attention Networks for Music Structure Analysis
Buisson, M. , McFee, B. , and Essid, S.
International society for music information retrieval (ISMIR) conference.
2024
The Changing Sound of Music: An Exploratory Corpus Study of Vocal Trends Over Time
Georgieva, E. , Ripollés, P. , and McFee, B.
International society for music information retrieval (ISMIR) conference.
2024
Spatial Scaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Rooms
Roman, I.R. , Ick, C. , Ding, S. , Roman, A.S. , McFee, B. , and Bello, J.P.
International conference on acoustics, speech and signal processing (ICASSP).
2024
Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation
Buisson, M. , McFee, B. , Essid, S. , and Crayencour, H.
IEEE Transactions on Audio, Speech and Language Processing
2023
A Repetition-based Triplet Mining Approach for Music Segmentation
Buisson, M. , McFee, B. , Essid, S. , and Crayencour, H.
International society for music information retrieval (ISMIR) conference.
2023
Transfer Learning and Bias Correction with Pre-trained Audio Embeddings
Wang, C. , Richard, G. , and McFee, B.
International society for music information retrieval (ISMIR) conference.
2023
Leveraging Geometrical Acoustic Simulations of Spatial Room Impulse Responses for Improved Sound Event Detection and Localization
Ick, C. and McFee, B.
Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)
2023
Efficient Evaluation Algorithms for Sound Event Detection
Lostanlen, V. and McFee, B.
Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)
2023
Foley Sound Synthesis at the DCASE 2023 Challenge
Choi, K. , Im, J. , Heller, L.M. , McFee, B. , Imoto, K. , Okamoto, Y. , Lagrange, M. , and Takamichi, S.
Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)
2023
Automatic recognition of cascaded guitar effects
Guo, J. and McFee, B.
International conference on digital audio effects (DAFx)
2022
Learning multi-level representations for hierarchical music structure analysis
Buisson, M. , McFee, B. , Essid, S. , and Crayencour, H.
International society for music information retrieval (ISMIR) conference.
2021
Automatic Hierarchy Expansion for Improved Structure and Chord Evaluation
Kinnaird, K. and McFee, B.
Transactions of the International Society for Music Information Retrieval
2021
Sound Event Detection in Urban Audio With Single and Multi-Rate PCEN
Ick, C. and McFee, B.
International conference on acoustics, speech and signal processing (ICASSP).
2021
Multi-Task Self-Supervised Pre-Training for Music Classification
Wu, H.H. , Kao, C.C. , Tang, Q. , Sun, M. , McFee, B. , Bello, J.P. , and Wang, C.
International conference on acoustics, speech and signal processing (ICASSP).
2021
Interactive Learning of Signal Processing Through Music: Making Fourier Analysis Concrete for Students
Müller, M. , McFee, B. , and Kinnaird, K.
IEEE Signal Processing Magazine
2020
Audio-Based Music Structure Analysis: Current Trends, Open Challenges, and Applications
Nieto, O. , Mysore, G.J. , Wang, C. , Smith, J.B.L. , Schlüter, J. , Grill, T. , and McFee, B.
Transactions of the International Society for Music Information Retrieval
2020
Multiple F0 estimation in vocal ensembles using convolutional neural networks
Cuesta, H. , McFee, B. , and Gómez, E.
International society for music information retrieval (ISMIR) conference.
2020
Entrofy your cohort: a transparent method for diverse cohort selection
Huppenkothen, D. , McFee, B. , and Norén, L.
PLOS ONE
2020
Learning the helix topology of musical pitch
Lostanlen, V. , Sridar, S. , McFee, B. , Farnsworth, A. , and Bello, J.P.
International conference on acoustics, speech and signal processing (ICASSP).
2019
Improving structure evaluation through automatic hierarchy expansion
McFee, B. and Kinnaird, K.
International society for music information retrieval (ISMIR) conference.
2019
Voice anonymization in urban sound recordings
Cohen-Hadria, A. , Cartwright, M. , McFee, B. , and Bello, J.P.
International workshop on machine learning for signal processing (MLSP).
2019
Enhanced hierarchical music structure annotations via feature level similarity fusion
Tralie, C.J. and McFee, B.
International conference on acoustics, speech and signal processing (ICASSP).
2019
A music structure informed downbeat tracking system using skip-chain conditional random fields and deep learning
Fuentes, M. , McFee, B. , Crayencour, H. , Essid, S. , and Bello, J.P.
International conference on acoustics, speech and signal processing (ICASSP).
2019
Open source practices for music signal processing research
McFee, B. , Kim, J.W. , Cartwright, M. , Salamon, J. , Bittner, R.M. , and Bello, J.P.
IEEE Signal Processing Magazine
2019
Per-channel energy normalization: why and how
Lostanlen, V. , Salamon, J. , Cartwright, M. , McFee, B. , Farnsworth, A. , Kelling, S. , and Bello, J.P.
IEEE Signal Processing Letters
2018
Adaptive pooling operators for weakly labeled sound event detection
McFee, B. , Salamon, J. , and Bello, J.P.
IEEE Transactions on Audio, Speech and Language Processing
2018
OpenMIC-2018: An open dataset for multiple instrument recognition
Humphrey, E. , Durand, S. , and McFee, B.
19th International Society for Music Information Retrieval (ISMIR) conference
2018
Analysis of common design choices in deep learning systems for downbeat tracking
Fuentes, M. , McFee, B. , Crayencour, H. , Essid, S. , and Bello, J.P.
19th International Society for Music Information Retrieval (ISMIR) conference
2018
Bubble cooperative networks for identifying important speech cues
Trinh, V.A. , McFee, B. , and Mandel, M.
InterSpeech
2017
Evaluating hierarchical structure in music annotations
McFee, B. , Nieto, O. , Farbood, M. , and Bello, J.P.
Frontiers in Psychology
2017
Structured training for large-vocabulary chord recognition
McFee, B. and Bello, J.P.
18th International Society for Music Information Retrieval (ISMIR) conference
2017
Deep salience representations for F0 estimation in polyphonic music
Best student paper award
Bittner, R.M. , McFee, B. , Salamon, J. , Li, P. , and Bello, J.P.
18th International Society for Music Information Retrieval (ISMIR) conference
2017
Statistical methods for scene and event classification
McFee, B.
Computational Analysis of Sound Scenes and Events
2016
resampy: efficient sample rate conversion in python
McFee, B.
The Journal of Open Source Software
2016
A plan for sustainable MIR evaluation
McFee, B. , Humphrey, E. , and Urbano, J.
17th International Society for Music Information Retrieval (ISMIR) conference
2015
A software framework for musical data augmentation
McFee, B. , Humphrey, E. , and Bello, J.P.
16th International Society for Music Information Retrieval (ISMIR) conference
2015
Hierarchical evaluation of segment boundary detection
McFee, B. , Nieto, O. , and Bello, J.P.
16th International Society for Music Information Retrieval (ISMIR) conference
2015
librosa: Audio and Music Signal Analysis in Python
McFee, B. , Raffel, C. , Liang, D. , Ellis, D.P.W. , McVicar, M. , Battenberg, E. , and Nieto, O.
14th annual Scientific Computing with Python conference (SciPy)
2014
Analyzing song structure with spectral clustering
Best oral presentation award
McFee, B. and Ellis, D.P.W.
15th International Society for Music Information Retrieval (ISMIR) conference
2014
mir_eval: a transparent implementation of common MIR metrics
Best poster presentation award
Raffel, C. , McFee, B. , Salamon, J. , Humphrey, E. , Nieto, O. , Liang, D. , and Ellis, D.P.W.
15th International Society for Music Information Retrieval (ISMIR) conference
2014
Codebook-based audio feature representation for music information retrieval
Vaizman, Y. , McFee, B. , and Lanckriet, G.R.G.
IEEE Transactions on Audio, Speech and Language Processing
2014
Speech enhancement by low-rank and convolutive dictionary spectrogram decomposition
Chen, Z. , McFee, B. , and Ellis, D.P.W.
Interspeech
2014
Learning to segment songs with ordinal linear discriminant analysis
McFee, B. and Ellis, D.P.W.
International conference on acoustics, speech and signal processing (ICASSP)
2014
Better beat tracking through robust onset aggregation
McFee, B. and Ellis, D.P.W.
International conference on acoustics, speech and signal processing (ICASSP)
2013
Iterative category discovery via multiple kernel metric learning
Galleguillos, C. , McFee, B. , and Lanckriet, G.R.G.
International Journal of Computer Vision
2013
Robust structural metric learning
Lim, D.K.H. , McFee, B. , and Lanckriet, G.R.G.
30th International Conference on Machine Learning (ICML)
2012
Hypergraph models of playlist dialects
McFee, B. and Lanckriet, G.R.G.
13th International Society for Music Information Retrieval (ISMIR) conference
2012
How significant is statistically significant? The case of audio music similarity and retrieval
Urbano, J. , Downie, J.S. , McFee, B. , and Schedl, M.
13th International Society for Music Information Retrieval (ISMIR) conference
2012
The Million Song Dataset Challenge
McFee, B. , Bertin-Mahieux. T. , Ellis, D.P.W. , and Lanckriet, G.R.G.
4th International Workshop on Advances in Music Information Research (AdMIRe)
2012
Learning content similarity for music recommendation
McFee, B. , Barrington, L. , and Lanckriet, G.R.G.
IEEE Transactions on Audio, Speech and Language Processing
2011
The natural language of playlists
Best poster presentation award
McFee, B. and Lanckriet, G.R.G.
12th International Society for Music Information Retrieval (ISMIR) conference
2011
Large-scale music similarity search with spatial trees
McFee, B. and Lanckriet, G.R.G.
12th International Society for Music Information Retrieval (ISMIR) conference
June, 2011
From region similarity to category discovery
Galleguillos, C. , McFee, B. , Belongie, S. , and Lanckriet, G.R.G.
IEEE conference on Computer Vision and Pattern Recognition (CVPR)
February, 2011
Learning multi-modal similarity
McFee, B. and Lanckriet, G.R.G.
Journal of Machine Learning Research (JMLR)
February, 2011
Contextual object localization with multiple kernel nearest neighbor
McFee, B. , Galleguillos, C. , and Lanckriet, G.R.G.
IEEE Transactions on Image Processing (TIP)
2010
Learning similarity from collaborative filters
McFee, B. , Barrington, L. , and Lanckriet, G.R.G.
11th International Society for Music Information Retrieval (ISMIR) conference
2010
Collaborative filtering based on P2P networks
Koenigstein, N. , Lanckriet, G.R.G. , McFee, B. , and Shavitt, Y.
11th International Society for Music Information Retrieval (ISMIR) conference
2010
Metric learning to rank
McFee, B. and Lanckriet, G.R.G.
Twenty-seventh International Conference on Machine Learning (ICML)
2010
Multi-class object localization by combining local contextual interactions
Galleguillos, C. , McFee, B. , Belongie, S. , and Lanckriet, G.R.G.
IEEE conference on Computer Vision and Pattern Recognition (CVPR)
2009
Heterogeneous embedding for subjective artist similarity
Best presentation award
McFee, B. and Lanckriet, G.R.G.
10th International Society for Music Information Retrieval (ISMIR) conference
2009
Partial order embedding with multiple kernels
McFee, B. and Lanckriet, G.R.G.
Twenty-sixth International Conference on Machine Learning (ICML)

Software

autopool
automatic pooling for multiple instance learning
crema
convolutional and recurrent estimators for music analysis
pumpp
practically universal music pre-processor
pescador
stream sampling for iterative learning algorithms
amen
algorithmic music remixing.
resampy
efficient audio resampling in Python.
muda
Musical Data Augmentation.
JAMS
a JSON Annotated Music Specification. v0.2 technical report
Ordinal LDA
Python (sklearn) implementation of ordinal linear discriminant analysis.
librosa
A python package for music and audio signal analysis.
MLR
MATLAB implementation of metric learning to rank.
Hypergraph playlists
Python implementation of the model from this paper.
Spatial trees
Python implementation of spatial trees for approximate nearest neighbor search, as used in this paper.

Data

Open-MIC 2018
Audio and partial instrument annotations for 20,000 10-second clips
MSD Challenge
Large-scale music recommendation on the Million Song Dataset. See also the year 1 test set.
AotM-2011
Annotated playlists from Art of the Mix, indexed to the Million Song Dataset.
AotM-2003
An earlier collection of playlists from Art of the Mix, also indexed to the Million Song Dataset.
aset400 kernels
Kernel matrices for aset400 artist similarity experiments
eHarmony
Matchings and anonymized features for several hundred thousand eHarmony users.

Teaching

F23, F24
(NYU) DS-GA 1006: Capstone Project in Data Science
F20
(NYU) DS-GA 3001: Search and Discovery
S19, S20, S21, S22, S23
(NYU) DS-GA 1004: Big Data
F18, F19, F20, S21, F21, F22, F23, F24
(NYU) MPATE-GE 2599: Fundamentals of Digital Signals Theory I
S14
(Columbia) EECS E6891: Reproducing computational results