Can I directly download audio using youtube-dl?

Here is a recent article in webupd8.org that explains how to use youtube-dl to directly download audio instead of first downloading video+audio and then extracting audio using -x. Unfortunately, the search facility that Maythux asked for here isn't offered. But it is still worth a look:
Video Downloader youtube-dl Gets Support For Separate Audio And Video YouTube DASH Streams

Basically, download the latest version of youtube-dl from source or from the ppa offered in the link above but not from the Software Center.

Then, if you already have the video link ...

Run, as example:

youtube-dl -F 'http://www.youtube.com/watch?v=HRIF4_WzU1w'

This will list the various download formats available for this url (audio and video).

$ youtube-dl -F 'http://www.youtube.com/watch?v=HRIF4_WzU1w'
[youtube] Setting language
[youtube] HRIF4_WzU1w: Downloading webpage
[youtube] HRIF4_WzU1w: Downloading video info webpage
[youtube] HRIF4_WzU1w: Extracting video information
[info] Available formats for HRIF4_WzU1w:
format code extension resolution  note 
171         webm      audio only  DASH webm audio , audio@ 48k (worst)
140         m4a       audio only  DASH audio , audio@128k
160         mp4       192p        DASH video 
133         mp4       240p        DASH video 
134         mp4       360p        DASH video 
135         mp4       480p        DASH video 
17          3gp       176x144     
36          3gp       320x240     
5           flv       400x240     
43          webm      640x360     
18          mp4       640x360     (best)

Now, choose desired audio format. I went for 140

Run:

youtube-dl -f 140 'http://www.youtube.com/watch?v=HRIF4_WzU1w'

$ youtube-dl -f 140 'http://www.youtube.com/watch?v=HRIF4_WzU1w'
[youtube] Setting language
[youtube] HRIF4_WzU1w: Downloading webpage
[youtube] HRIF4_WzU1w: Downloading video info webpage
[youtube] HRIF4_WzU1w: Extracting video information
[download] Destination: Martin Luther King - I Have a Dream on August 28, 1963 [Sous-titres & Subtitles] [FULL SPEECH]-HRIF4_WzU1w.m4a
[download] 100% of 15.19MiB in 00:04

That's it.

From @xiota's comment, to download the best audio in m4a format:

$ youtube-dl -f 'bestaudio[ext=m4a]' 'http://youtu.be/hTvJoYnpeRQ'

^{I've missed it on the first reading, so I've decided to post the comment as an answer to make it more visible.}

This alias let me easily download any audio with better filenames:

$ alias | grep audio

alias youtube-dl-audio='youtube-dl --ignore-errors --output "%(title)s.%(ext)s" --extract-audio --audio-format mp3'

Audio quality: From the documentation: "By default youtube-dl tries to download the best available quality, i.e. if you want the best quality you don't need to pass any special options, youtube-dl will guess it for you by default.". So there's no need for --audio-quality 0 or -f bestaudio.

Updates: Download last version of youtube-dl. Avoid repo or PPA version so you can upgrade with youtube-dl -U (necessary quite often).

To install it right away for all UNIX users (Linux, OS X, etc.), type:

sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl

Albums (or compilations, mashups, etc.): You can split the file into chunks/tracks with mp3split. To generate the playlist timestamps (like the ones found on YouTube comments) and track count:

ffmpeg -i input.mp3 -af silencedetect=n=-40dB:d=2.7 -f null - |& awk '/silence_end/ {print $4,$5}' | awk '{S=$2;printf "%d:%02d:%02d\n",S/(60*60),S%(60*60)/60,S%60}' | tee >(wc -l)  # Play with tolerance (n/noise threshold level) and duration on silencedetect. More: `ffmpeg -h filter=silencedetect`.

Please leave a comment if you found a better approach, maybe something using MusicBrainz.

Common problems

• Extremely slow? Switch to yt-dlp. Includes things like --split-chapters!
• Error 403? Add --rm-cache-dir.
• Stuck at "Downloading webpage"? Add --force-ipv4.

Alternatives and other notes

• ytmdl: A simple script to get songs from YouTube in mp3 format with all tags from iTunes.

• To download in batch from a plain text, here.

• MP3 Tagging: I’m still trying to find a reliable CLI tool to fix and tag all the generated mp3:

  • Beets (maybe with tracker cheatsheet and gnome-music?)
  • lltag
  • foobar2000