![[me]](i/bmcfee14.jpg)
Brian McFee brian.mcfee@nyu.edu
https://orcid.org/0000-0001-6261-9747Assistant Professor of Music Technology and Data Science at New York University
Music and Performing Arts Professions/Music and Audio Research Lab (MARL) and Center for Data Science
I develop machine learning tools to analyze music and multimedia data. This includes recommender systems, image and audio analysis, similarity learning, cross-modal feature integration, and automatic annotation.
For a full history, here's my curriculum vitæ.
Ph.D. Students
- Qingyang (Tom) Xi (Music Technology)
- Chris Ick (Data Science)
- Elena Georgieva (Music Technology)
Publications
Spatial Scaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Rooms
International conference on acoustics, speech and signal processing (ICASSP).
Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation
Buisson, M., McFee, B., Essid, S., and Crayencour, H.
IEEE Transactions on Audio, Speech and Language Processing (to appear)
A Repetition-based Triplet Mining Approach for Music Segmentation
Buisson, M., McFee, B., Essid, S., and Crayencour, H.
International society for music information retrieval (ISMIR) conference.
Transfer Learning and Bias Correction with Pre-trained Audio Embeddings
Wang, C., Richard, G., and McFee, B.
International society for music information retrieval (ISMIR) conference.
Leveraging Geometrical Acoustic Simulations of Spatial Room Impulse Responses for Improved Sound Event Detection and Localization
Ick, C. and McFee, B.
Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)
Efficient Evaluation Algorithms for Sound Event Detection
Lostanlen, V. and McFee, B.
Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)
Foley Sound Synthesis at the DCASE 2023 Challenge
Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)
Automatic recognition of cascaded guitar effects
Guo, J. and McFee, B.
International conference on digital audio effects (DAFx)
Learning multi-level representations for hierarchical music structure analysis
Buisson, M., McFee, B., Essid, S., and Crayencour, H.
International society for music information retrieval (ISMIR) conference.
Automatic Hierarchy Expansion for Improved Structure and Chord Evaluation
Kinnaird, K. and McFee, B.
Transactions of the International Society for Music Information Retrieval
Sound Event Detection in Urban Audio With Single and Multi-Rate PCEN
Ick, C. and McFee, B.
International conference on acoustics, speech and signal processing (ICASSP).
Multi-Task Self-Supervised Pre-Training for Music Classification
Wu, H.H., Kao, C.C., Tang, Q., Sun, M., McFee, B., Bello, J.P., and Wang, C.
International conference on acoustics, speech and signal processing (ICASSP).
Interactive Learning of Signal Processing Through Music: Making Fourier Analysis Concrete for Students
Müller, M., McFee, B., and Kinnaird, K.
IEEE Signal Processing Magazine
Audio-Based Music Structure Analysis: Current Trends, Open Challenges, and Applications
Transactions of the International Society for Music Information Retrieval
Multiple F0 estimation in vocal ensembles using convolutional neural networks
Cuesta, H., McFee, B., and Gómez, E.
International society for music information retrieval (ISMIR) conference.
Entrofy your cohort: a transparent method for diverse cohort selection
Huppenkothen, D., McFee, B., and Norén, L.
PLOS ONE
Learning the helix topology of musical pitch
International conference on acoustics, speech and signal processing (ICASSP).
Improving structure evaluation through automatic hierarchy expansion
McFee, B. and Kinnaird, K.
International society for music information retrieval (ISMIR) conference.
Voice anonymization in urban sound recordings
International workshop on machine learning for signal processing (MLSP).
Enhanced hierarchical music structure annotations via feature level similarity fusion
Tralie, C.J. and McFee, B.
International conference on acoustics, speech and signal processing (ICASSP).
A music structure informed downbeat tracking system using skip-chain conditional random fields and deep learning
International conference on acoustics, speech and signal processing (ICASSP).
Open source practices for music signal processing research
IEEE Signal Processing Magazine
Per-channel energy normalization: why and how
IEEE Signal Processing Letters
Adaptive pooling operators for weakly labeled sound event detection
McFee, B., Salamon, J., and Bello, J.P.
IEEE Transactions on Audio, Speech and Language Processing
OpenMIC-2018: An open dataset for multiple instrument recognition
Humphrey, E., Durand, S., and McFee, B.
19th International Society for Music Information Retrieval (ISMIR) conference
Analysis of common design choices in deep learning systems for downbeat tracking
19th International Society for Music Information Retrieval (ISMIR) conference
Bubble cooperative networks for identifying important speech cues
Trinh, V.A., McFee, B., and Mandel, M.
InterSpeech
Evaluating hierarchical structure in music annotations
Frontiers in Psychology
Structured training for large-vocabulary chord recognition
McFee, B. and Bello, J.P.
18th International Society for Music Information Retrieval (ISMIR) conference
Deep salience representations for F0 estimation in polyphonic music
18th International Society for Music Information Retrieval (ISMIR) conference
Statistical methods for scene and event classification
McFee, B.
Computational Analysis of Sound Scenes and Events
resampy: efficient sample rate conversion in python
McFee, B.
The Journal of Open Source Software
A plan for sustainable MIR evaluation
McFee, B., Humphrey, E., and Urbano, J.
17th International Society for Music Information Retrieval (ISMIR) conference
A software framework for musical data augmentation
McFee, B., Humphrey, E., and Bello, J.P.
16th International Society for Music Information Retrieval (ISMIR) conference
Hierarchical evaluation of segment boundary detection
McFee, B., Nieto, O., and Bello, J.P.
16th International Society for Music Information Retrieval (ISMIR) conference
librosa: Audio and Music Signal Analysis in Python
14th annual Scientific Computing with Python conference (SciPy)
Analyzing song structure with spectral clustering
McFee, B. and Ellis, D.P.W.
15th International Society for Music Information Retrieval (ISMIR) conference
mir_eval: a transparent implementation of common MIR metrics
15th International Society for Music Information Retrieval (ISMIR) conference
Codebook-based audio feature representation for music information retrieval
Vaizman, Y., McFee, B., and Lanckriet, G.R.G.
IEEE Transactions on Audio, Speech and Language Processing
Speech enhancement by low-rank and convolutive dictionary spectrogram decomposition
Chen, Z., McFee, B., and Ellis, D.P.W.
Interspeech
Learning to segment songs with ordinal linear discriminant analysis
McFee, B. and Ellis, D.P.W.
International conference on acoustics, speech and signal processing (ICASSP)
Better beat tracking through robust onset aggregation
McFee, B. and Ellis, D.P.W.
International conference on acoustics, speech and signal processing (ICASSP)
Iterative category discovery via multiple kernel metric learning
Galleguillos, C., McFee, B., and Lanckriet, G.R.G.
International Journal of Computer Vision
Robust structural metric learning
Lim, D.K.H., McFee, B., and Lanckriet, G.R.G.
30th International Conference on Machine Learning (ICML)
Hypergraph models of playlist dialects
McFee, B. and Lanckriet, G.R.G.
13th International Society for Music Information Retrieval (ISMIR) conference
How significant is statistically significant? The case of audio music similarity and retrieval
13th International Society for Music Information Retrieval (ISMIR) conference
The Million Song Dataset Challenge
4th International Workshop on Advances in Music Information Research (AdMIRe)
Learning content similarity for music recommendation
McFee, B., Barrington, L., and Lanckriet, G.R.G.
IEEE Transactions on Audio, Speech and Language Processing
The natural language of playlists
McFee, B. and Lanckriet, G.R.G.
12th International Society for Music Information Retrieval (ISMIR) conference
Large-scale music similarity search with spatial trees
McFee, B. and Lanckriet, G.R.G.
12th International Society for Music Information Retrieval (ISMIR) conference
From region similarity to category discovery
IEEE conference on Computer Vision and Pattern Recognition (CVPR)
Learning multi-modal similarity
McFee, B. and Lanckriet, G.R.G.
Journal of Machine Learning Research (JMLR)
Contextual object localization with multiple kernel nearest neighbor
McFee, B., Galleguillos, C., and Lanckriet, G.R.G.
IEEE Transactions on Image Processing (TIP)
Learning similarity from collaborative filters
McFee, B., Barrington, L., and Lanckriet, G.R.G.
11th International Society for Music Information Retrieval (ISMIR) conference
Collaborative filtering based on P2P networks
11th International Society for Music Information Retrieval (ISMIR) conference
Metric learning to rank
McFee, B. and Lanckriet, G.R.G.
Twenty-seventh International Conference on Machine Learning (ICML)
Multi-class object localization by combining local contextual interactions
IEEE conference on Computer Vision and Pattern Recognition (CVPR)
Heterogeneous embedding for subjective artist similarity
McFee, B. and Lanckriet, G.R.G.
10th International Society for Music Information Retrieval (ISMIR) conference
Partial order embedding with multiple kernels
McFee, B. and Lanckriet, G.R.G.
Twenty-sixth International Conference on Machine Learning (ICML)
Software
- autopool
- automatic pooling for multiple instance learning
- crema
- convolutional and recurrent estimators for music analysis
- pumpp
- practically universal music pre-processor
- pescador
- stream sampling for iterative learning algorithms
- amen
- algorithmic music remixing.
- resampy
- efficient audio resampling in Python.
- muda
- Musical Data Augmentation.
- JAMS
- a JSON Annotated Music Specification. v0.2 technical report
- Ordinal LDA
- Python (sklearn) implementation of ordinal linear discriminant analysis.
- librosa
- A python package for music and audio signal analysis.
- MLR
- MATLAB implementation of metric learning to rank.
- Hypergraph playlists
- Python implementation of the model from this paper.
- Spatial trees
- Python implementation of spatial trees for approximate nearest neighbor search, as used in this paper.
More projects can be found on my GitHub profile.
Data
- Open-MIC 2018
- Audio and partial instrument annotations for 20,000 10-second clips
- MSD Challenge
- Large-scale music recommendation on the Million Song Dataset. See also the year 1 test set.
- AotM-2011
- Annotated playlists from Art of the Mix, indexed to the Million Song Dataset.
- AotM-2003
- An earlier collection of playlists from Art of the Mix, also indexed to the Million Song Dataset.
- aset400 kernels
- Kernel matrices for aset400 artist similarity experiments
- eHarmony
- Matchings and anonymized features for several hundred thousand eHarmony users.
Teaching
- Fall 2020
- (NYU) DS-GA 3001: Search and Discovery
- Spring 2019, 2020, 2021
- (NYU) DS-GA 1004: Big Data
- Fall 2018, 2019, 2020
- (NYU) MPATE-GE 2599: Fundamentals of Digital Signals Theory I
- Spring 2014
- (Columbia) EECS E6891: Reproducing computational results
Hack-day projects
- The Keplerphone
- Music synthesis from the Kepler satellite. Shredding exoplanet rock.
- Yankomatic
- Automatically generate song title parodies using rhyming food words.
- Handmadedness
- Calculate how hand-made a song sounds by measuring repetition patterns and tempo stability.
- Frankenmasher 2000
- Isolate and recombine vocals and instruments from different songs to create mash-ups of hypothetical super-groups.
- Mend-a-break
- Resynthesize any audio clip using sparse convolutions of the Amen Break.
- Auto-chip-tune
- Generate Nintendo chip-tune programs from audio analysis.
See my hack-day blog for more projects.