Home / server / Questions / 391396
Accepted
Cerber
Asked: 2012-05-23 03:30:18 +0800 CST

How to split a PEM file

  • 772

Note : This is not really a question because I already found the answer but since I didn't find it easily here I will post it so that it can benefit others.

Question : How to read a concatenated PEM file as the one used by apache/mod_ssl directive SSLCACertificateFile ?

Answer (original) (source) : 

cat $file|awk 'split_after==1{n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > "cert" n ".pem"}'

This can leave an empty file if there's a blank line at the end, such as with openssl pkcs7 -outform PEM -in my-chain-file -print_certs. To prevent that, check the length of the line before printing:

cat $file|awk 'split_after==1{n++;split_after=0}
   /-----END CERTIFICATE-----/ {split_after=1}
   {if(length($0) > 0) print > "cert" n ".pem"}'

Answer 29/03/2016 :

Following @slugchewer answer, csplit might be a clearer option with :

csplit -f cert- $file '/-----BEGIN CERTIFICATE-----/' '{*}'
shell awk openssl x509

8 Answers

  1. The awk snippet works for extracting the different parts, but you still need to know which section is the key / cert / chain. I needed to extract a specific section, and found this on the OpenSSL mailinglist: http://openssl.6102.n7.nabble.com/Convert-pem-to-crt-and-key-files-tp47681p47697.html

    # Extract key
openssl pkey -in foo.pem -out foo-key.pem

# Extract all the certs
openssl crl2pkcs7 -nocrl -certfile foo.pem |
  openssl pkcs7 -print_certs -out foo-certs.pem

# Extract the textually first cert as DER
openssl x509 -in foo.pem -outform DER -out first-cert.der
    • 36

  2. The split command is available on most systems, and its invocation is likely easier to remember.

    If you have a file collection.pem that you want to split into individual-* files, use:

    split -p "-----BEGIN CERTIFICATE-----" collection.pem individual-

    If you don't have split, you could try csplit:

    csplit -s -z -f individual- collection.pem '/-----BEGIN CERTIFICATE-----/' '{*}'

    -s skips printing output of file sizes

    -z does not create empty files

    • 27
  3. Best Answer

    This was previously answered on StackOverflow : 

    awk '
  split_after == 1 {n++;split_after=0}
  /-----END CERTIFICATE-----/ {split_after=1}
  {print > "cert" n ".pem"}' < $file

    Edit 29/03/2016 : See @slugchewer answer

    • 19

  4. If you want to get a single certificate out of a multi-certificate PEM bundle, try:

    $ openssl crl2pkcs7 -nocrl -certfile INPUT.PEM | \
    openssl pkcs7 -print_certs | \
    awk '/subject.*CN=host.domain.com/,/END CERTIFICATE/'
    • The first two openssl commands will process a PEM file and and spit it back out with pre-pended "subject:" and "issuer:" lines before each cert. If your PEM is already formatted this way, all you need is the final awk command.
    • The awk command will spit out the individual PEM matching the CN (common name) string.

    source1 , source2

    • 9

  5. If you are handling full chain certificates (i.e. the ones generated by letsencrypt / certbot etc), which are a concatenation of the certificate and the certificate authority chain, you can use bash string manipulation.

    For example:

    # content of /path/to/fullchain.pem
-----BEGIN CERTIFICATE-----
some long base64 string containing
the certificate
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
another base64 string
containing the first certificate
in the authority chain
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
another base64 string
containing the second certificate
in the authority chain
(there might be more...)
-----END CERTIFICATE-----

    To extract the certificate and the certificate authority chain into variables:

    # load the certificate into a variable
FULLCHAIN=$(</path/to/fullchain.pem)
CERTIFICATE="${FULLCHAIN%%-----END CERTIFICATE-----*}-----END CERTIFICATE-----"
CHAIN=$(echo -e "${FULLCHAIN#*-----END CERTIFICATE-----}" | sed '/./,$!d')

    Explanation:

    Instead of using awk or openssl (which are powerful tools but not always available, i.e. in Docker Alpine images), you can use bash string manipulation.

    "${FULLCHAIN%%-----END CERTIFICATE-----*}-----END CERTIFICATE-----": from end of the content of FULLCHAIN, return the longest substring match, then concat -----END CERTIFICATE----- as it get stripped away. The * matches all the characters after -----END CERTIFICATE-----.

    $(echo -e "${FULLCHAIN#*-----END CERTIFICATE-----}" | sed '/./,$!d'): from the beginning of the content of FULLCHAIN, return the shortest substring match, then strip leading new lines. Likewise, the * matches all the characters before -----END CERTIFICATE-----.

    For a quick reference (while you can find more about string manipulation in bash here):

    ${VAR#substring}= the shortest substring from the beginning of the content of VAR

    ${VAR%substring}= the shortest substring from the end of the content of VAR

    ${VAR##substring}= the longest substring from the beginning of the content of VAR

    ${VAR%%substring}= the longest substring from the end of the content of VAR

    • 5

  6. Also worth noting that PEM files are just a collection of keys/certificates inside BEGIN/END blocks, so it's pretty easy to just cut/paste if it's just a single file with one or two interesting entities...

    • 3

  7. Hmmm...almost same way I prepared the solution ( as suggested y @Cerber ) without realizing that this situation seems many people have. My solution follow almost same logic but use some more basic commands:

    My all certs are in file: certin.pem

    c=0
while read line
  do
    if echo $line | grep END; then
    echo $line >> certout$c.pem
    c=`expr $c + 1`
    else
     echo $line
     echo $line >> certout$c.pem
    fi
done < /tmp/certin.pem

    This basically keep writing in a file until encounters "END" and then start writing to another file in incremented way. This way you will have "N" number of output files (certout0.pem, certout1.pem and so on..) files depending on how many certificates are there in your input pem file ( certin.pem ).

    • 0

  8. You can use awk as follows:

    awk '/-----BEGIN CERTIFICATE-----/ {f=1} /-----END CERTIFICATE-----/ {print; f=0} f' cert_and_key.pem

    awk '/-----BEGIN PRIVATE KEY-----/ {f=1} /-----END PRIVATE KEY-----/ {print; f=0} f' cert_and_key.pem

    The command awk '/hello/ {f=1} /world/ {print; f=0} f' some_file will print all lines in some_file that between the lines hello and world, inclusive.

    • 0