evtest and dbus

Here is a version involving a small script running in the background. A few easy steps need to be taken:

• Install the evtest program: sudo apt install evtest. This program is able to decode and print raw events from the /dev/input/ hierarchy of devices.

• Add your user account to the group input. Logout and login again. (This is needed because we are going to read from some device under /dev/input/, which normally can only be done by root or users in group input. We could run the script as root, but then we would have to find a way to tell it which user dbus instance it needs to talk to.)

• Disable the capslock key like you did or use the corresponding Mate preference. It would also work if you map the capslock key to ctrl, like many people do.

• Store the following script somewhere, e. g. $HOME/bin/capshack and make it executable:

  #!/bin/bash
  
  KEYBOARD=/dev/input/by-path/platform-i8042-serio-0-event-kbd
  EVENT="type 1 (EV_KEY), code 58 (KEY_CAPSLOCK), value 1"
  
  toggleVLC () {
          dbus-send \
              --type=method_call \
              --dest=org.mpris.MediaPlayer2.vlc \
              /org/mpris/MediaPlayer2 \
              org.mpris.MediaPlayer2.Player.PlayPause
  }
  
  evtest "$KEYBOARD" | \
  while read line
  do
      if [[ "$line" == *"$EVENT" ]]
      then
          toggleVLC
      fi
  done
  

The above script will listen to your keyboard events and execute a command when the capslock key is pressed. The command that is run is using dbus to send a play/pause event to a running VLC instance. You might have to tweak the KEYBOARD variable to point to your actual keyboard, but it is likely the above value will work.

If you prefer the toggleVLC() function to be called when you release the capslock key, you can change the EVENT variable in the script such that it ends with value 0 instead of value 1.

Also, it is possible that for you the keycode of the capslock key is different from 58. You can check this by running evtest, look at the output when you press capslock, and adjust the EVENT variable accordingly. You do not even have to match by the keycode and set EVENT simply to (KEY_CAPSLOCK), value 1.

Instead of using evtest and above script you could use thd from the triggerhappy package (something like thd --dump /dev/input/* would give similar output) but I found that a little bit less educational.

Now we create a user systemd unit to start the script automatically in the background:

• Create the user systemd hierarchy: mkdir -p $HOME/.config/systemd/user

• Add the following text to the file $HOME/.config/systemd/user/capshack.service

  [Unit]
  Description=Abuse capslock key
  
  [Service]
  ExecStart=%h/bin/capshack
  
  [Install]
  WantedBy=default.target
  

• Enable the service unit you just made: systemctl --user enable capshack

• Start the service: systemctl --user start capshack

From now on you should be able to toggle play/pause in VLC using the capslock key. The background script should be automatically started when you log in.

triggerhappy

As mentioned already you could also use the triggerhappy package to do this. This is the preferred method for people who want to fiddle less with the system and rather like a more solid solution.

• Add user to the input group like above

• Install the package: sudo apt install triggerhappy

• Add a configuration for triggerhappy to ~/.config/thd/capslock.conf:

  KEY_CAPSLOCK    1   dbus-send --type=method_call --dest=org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause
  

• Run thd like this: thd --triggers ~/.config/thd/capslock.conf /dev/input/event*

• Make thd run for your user in a similar way like described for capshack already.

• Make sure you have only one of the mentioned methods (capshack or thd) running, to not double toggle which each key press.

I tested all this on 18.04 under the standard session and 16.04 mate. But it should theoretically also work under wayland or with other desktop environments.

In Ubuntu 16.04 after spending hours googling & fiddling with dconf-editor and xbindkeys I found an extremely easy solution:

• Very quick setup in 2 minutes
• Only xdotool needs to be installed
• No scripts necessary
• Caps lock is still recognized but doesn't toggle letter case
• Works with standard Ubuntu Keyboard Custom Shortcut Settings

If you need Gnome XF86AudioPlay support you need xdotool (not needed for VLC as OP stipulates):

sudo apt install xdotool

Now here's the tricky part. Every time you press the Caps LocK key to toggle the music between Play and Pause, capital letters also toggle off and on.

Use this method to turn off Caps Lock toggling while still keeping key active:

setxkbmap -option caps:none

Music players that adhere to Gnome

Then in Ubuntu 16.04 go into Settings >> Keyboard >> Shortcuts >> Custom Shortcuts:

• Set the name to Audio Play
• Set the command to xdotool key XF86AudioPlay
• Then click the Apply button

After adding the new option the assigned shortcut key will read Disabled. Click on Disabled and when prompted for key to assign press the Caps LocK key.

Music players like VLC

Follow the steps for Gnome in the previous section but for the command use this instead:

qdbus org.mpris.MediaPlayer2.vlc /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause

VLC won't recognize media keys: How to make the keyboard media keys to work with VLC globally? even when built into the keyboard. Therefor the custom keyboard shortcut to XF86AudioPlay is useless with xdotool.

Setting custom keyboard shortcuts from command line

This Q&A discusses how to set custom keyboard shortcuts from the command line: How to set custom keyboard shortcuts from terminal?

Deactivating caps lock permanently

Next step is to make this setxkbmap -option caps:none persistent across reboots. Googling results in this answer by Terdon: Save setxkbdmap settings:

An easy way would be to add the command to your $HOME/.profile file (you can create it if it doesn't exist):

setxkbmap -option caps:none

That should make it run every time you log in.

Note that you should use $HOME/.profile rather than $HOME/.bash_profile, $HOME/.bashrc or some other similar file. This is because this setting should be read by your login manager.

This answer has been tested on Ubuntu MATE 16.04.4 LTS in a VM (but the keyboard was a USB keyboard on passthrough, so that should not make a difference). It also requires no extra software, and can be done on a vanilla install without installing any packages, and it is persistent between boots.

Remap Caps Lock

1. Edit the file /usr/share/X11/xkb/symbols/pc as root. Be careful not to make a mistake here (like forgetting the comma) because you might not be able to use your keyboard anymore if you do. (Guess what I learned the hard way? ;)

2. Comment out the line

       key <CAPS> {    [ Caps_Lock     ]   };
   

   by placing two forward slashes (//) at the beginning of it.

3. Add beneath it the following line:

       key <CAPS> {    [ XF86AudioPlay, XF86AudioPause ]   };
   

4. Comment out the line

       modifier_map Lock   { Caps_Lock };
   

   in the same manner as above.

5. Reboot.

Note: (credit to danzel for pointing this out) Updates to the package xkb-data can overwrite this file. A reasonably safe method for preventing this is diverting the path using dpkg-divert. (I say "reasonably safe" because the upstream source for this file hasn't been modified in three years.)

Alternative Method

Manually disable your Caps Lock key in your keyboard layout settings, then use the approach in this answer to add your own key re-mappings.

Problems?

Your caps lock key no longer locks caps, and it has been remapped to Play/Pause at this point. You should be able to play and pause music in VLC now. If it works at this point, you may skip the rest of these instructions.

However, when I tried it, VLC did not respond to the Play/Pause key.

Make sure the keyboard shortcut is set properly

6. Open Keyboard Shortcuts (System → Preferences → Hardware → Keyboard Shortcuts)

7. Ensure that "Play (or Play/Pause)" is set to "XF86AudioPlay"

8. If it is not, click on the "Shortcut" section and press your new Play/Pause key to set the shortcut.

9. If it still doesn't work, reboot a couple of times for good measure. (That's actually what fixed it for me.)

Install xbindkeys, xcape and playerctl. Make sure that xcape (mapping CapsLock to a spare key like Alt+F12 by xcape -e 'Caps_Lock=Alt_L|F12) and playerctl are autostarted on log-in, and add to the file ~/.xbindkeysrc the two lines

"playerctl play-pause"
  Alt + F12

Make your capslock key a play/pause key

This solution will remap your capslock key's scancode to a different keycode in the kernel input driver, in this case play/pause.

Advantages

• No need to modify existing system files (only one new file has to be created)
• Applications won't know that it is actually a capslock key but think a real play/pause key has been pressed
• No need to disable capslock, no possibility to mess up xkb configuration

Disadvantages

• Has to be done for each keyboard individually (which, depending on the situation, can also be an advantage)

Prerequisites

• evtest (can be installed via sudo apt-get install evtest)

Procedure

In a terminal, run sudo evtest and select your keyboard. If there is no output when you press a key, press Ctrl+C and try a different device. Remember the device event number you selected, we will need it afterwards. Press the capslock key. Some lines like the following will appear:

Event: time 1529406021.187148, -------------- SYN_REPORT ------------
Event: time 1529406021.218427, type 4 (EV_MSC), code 4 (MSC_SCAN), value 70039
Event: time 1529406021.218427, type 1 (EV_KEY), code 58 (KEY_CAPSLOCK), value 0

We need the value from the MSC_SCAN line preceding the KEY_CAPSLOCK line. In the example above, this would be 70039, but the scancode on your keyboard is probably different.

Now run the following command, but replace the X with the device event number you used for the evtest command:

grep "" /sys/class/input/eventX/device/id/*

...this will output bustype, product, vendor and version of the keyboard.

Create a file named 65-keyboard-custom.hwdb in /etc/udev/hwdb.d/ (root permissions required) with the following content:

evdev:input:b[bustype]v[vendor]p[product]e[version]*
 KEYBOARD_KEY_[scancode]=playpause

...where

• [bus], [vendor], [product] and [version] have exactly 4 characters and letters need to be uppercase
• [scancode] has to be lowercase
• the evdev:... line has no preceding space
• the KEYBOARD_KEY... line has exactly one preceding space

For example:

evdev:input:b0003v046DpC228e0110*
 KEYBOARD_KEY_70039=playpause

Finally, compile the new configuration to the hardware database:

sudo systemd-hwdb update

If you want to apply the changes immediately, inform udev:

sudo udevadm trigger

If you want to revert the changes, just remove the file you created and reboot.

You can find more detailed information in my original answer on this topic.

Maybe using xbindkeys or XTE you could do that. But first be sure of what is the Keycodes, you can see the code for the capslock key using command xevin the terminal and pressing it (you can see keycode of any keu or mouse button, including Gaming mouses). With capslock and play buttons' keycode in hand you can edit xbindkeys confi file and put there your changes. use touch ~/.xbindkeysrc to create the config file (if system does not create it automatically) and nano ~/.xbindkeysrc to edit it. I've used it last month and here its an examble. take a look at these links that might be very helpfull for you.

https://help.ubuntu.com/community/MultimediaKeys

https://gist.github.com/jbonney/5743509

Remap a button from my gaming mouse to a keyboard key

https://matmunn.me/post/binding-volume-control-mouse-buttons-ubuntu/