I have a problem with pandoc not generating a table of content correclty.
Here is my workflow:
- I create a Markdown file with the content of a web page.
- I version my project with Git.
- I
git pushto my self-hosted GitLab server. - Using CI/CD, my server runs 3 scripts in Ubuntu/bionic LXC containers.
- The first one extracts metadata.
- The second one calls pandoc to generate an HTML page with header, footer, table of contents and CSS.
- The third one pushes the new HTML page to my production server.
Everything works perfectly except a small detail: when pandoc generates the HTML, the table of contents is missing. What's wierd is that it works when I run it on my local machine (Fedora 34).
Here is the script I am running in the second container (which has nothing installed beforehand):
apt-get update
apt-get install -y python3 pandoc pandoc-citeproc
mkdir -p build
python3 .deploy/convert.py
You can find the result of the script down below.
NOTE: in an LXC container, the user is root, no need for sudo.
Content of .deploy/convert.py:
import sys
import os
import re
from pathlib import Path
if __name__ == '__main__':
with open('data/type', 'r') as f:
type = f.readline()
with open('data/number', 'r') as f:
number = f.readline()
with open('data/name', 'r') as f:
name = f.readline()
result = list(Path(".").rglob("*.md"))
for file in result:
parts = str(file).split('.')
parts.pop()
filename = '.'.join(parts)
if filename != 'README':
parts = str(filename).split('/')
parts.pop()
directory = '/'.join(parts)
os.system('mkdir -p build/' + directory)
# This is the important line!
os.system('pandoc --toc --metadata title=' + type + '-' + number +
' --metadata type=' + type +
' --metadata number=' + number +
' --metadata name="' + name +
'" -s --template .deploy/template.html ' +
str(file) +
' -o build/' + filename + '.html'
)
with open('build/' + filename + '.html', 'r') as f:
filedata = f.read()
filedata = re.sub('href="(.*)\.md"', 'href="\\1.html"', filedata)
with open('build/' + filename + '.html', 'w') as f:
f.write(filedata)
The 3 containers are called sequentially, leaving what is called artifacts in between: they are files that can be passed to the next container. I checked the artifacts before the convert container and after it, the only missing part is the table of content.
Did I miss something?
Here is the result I get from the script. Again, even if you see $ instead of #, know that I am root!
$ apt-get update
Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:5 https://packagecloud.io/github/git-lfs/ubuntu bionic InRelease
Reading package lists...
$ apt-get install -y python3 pandoc
Reading package lists...
Building dependency tree...
Reading state information...
python3 is already the newest version (3.6.7-1~18.04).
python3 set to manually installed.
The following package was automatically installed and is no longer required:
libfreetype6
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
liblua5.1-0 libluajit-5.1-2 libluajit-5.1-common pandoc-data
Suggested packages:
texlive-latex-recommended texlive-xetex texlive-luatex pandoc-citeproc
texlive-latex-extra context wkhtmltopdf
The following NEW packages will be installed:
liblua5.1-0 libluajit-5.1-2 libluajit-5.1-common pandoc pandoc-data
0 upgraded, 5 newly installed, 0 to remove and 13 not upgraded.
Need to get 7103 kB of archives.
After this operation, 53.7 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblua5.1-0 amd64 5.1.5-8.1build2 [100 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libluajit-5.1-common all 2.1.0~beta3+dfsg-5.1 [44.3 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libluajit-5.1-2 amd64 2.1.0~beta3+dfsg-5.1 [227 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 pandoc-data all 1.19.2.4~dfsg-1build4 [40.1 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 pandoc amd64 1.19.2.4~dfsg-1build4 [6692 kB]
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Fetched 7103 kB in 1s (5904 kB/s)
dpkg-preconfigure: unable to re-open stdin:
Selecting previously unselected package liblua5.1-0:amd64.
(Reading database ... 29084 files and directories currently installed.)
Preparing to unpack .../liblua5.1-0_5.1.5-8.1build2_amd64.deb ...
Unpacking liblua5.1-0:amd64 (5.1.5-8.1build2) ...
Selecting previously unselected package libluajit-5.1-common.
Preparing to unpack .../libluajit-5.1-common_2.1.0~beta3+dfsg-5.1_all.deb ...
Unpacking libluajit-5.1-common (2.1.0~beta3+dfsg-5.1) ...
Selecting previously unselected package libluajit-5.1-2:amd64.
Preparing to unpack .../libluajit-5.1-2_2.1.0~beta3+dfsg-5.1_amd64.deb ...
Unpacking libluajit-5.1-2:amd64 (2.1.0~beta3+dfsg-5.1) ...
Selecting previously unselected package pandoc-data.
Preparing to unpack .../pandoc-data_1.19.2.4~dfsg-1build4_all.deb ...
Unpacking pandoc-data (1.19.2.4~dfsg-1build4) ...
Selecting previously unselected package pandoc.
Preparing to unpack .../pandoc_1.19.2.4~dfsg-1build4_amd64.deb ...
Unpacking pandoc (1.19.2.4~dfsg-1build4) ...
Setting up libluajit-5.1-common (2.1.0~beta3+dfsg-5.1) ...
Setting up pandoc-data (1.19.2.4~dfsg-1build4) ...
Setting up libluajit-5.1-2:amd64 (2.1.0~beta3+dfsg-5.1) ...
Setting up liblua5.1-0:amd64 (5.1.5-8.1build2) ...
Setting up pandoc (1.19.2.4~dfsg-1build4) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.5) ...
$ mkdir -p build
$ python3 .deploy/convert.py
Uploading artifacts for successful job 00:01
Uploading artifacts...
Runtime platform arch=amd64 os=linux pid=1620 revision=bd40e3da version=14.9.1
build: found 2 matching files and directories
untracked: found 4 files
Uploading artifacts as "archive" to coordinator... 201 Created id=3380 responseStatus=201 Created token=WHTmbFcM
Cleaning up file based variables 00:00
Job succeeded
With pandoc 1, the placeholder for the table of contents was
$toc$, but with pandoc 2, it became$table-of-contents$.Somehow, the container installs pandoc 1, which isn't compatible with the latest version of the script.