Home / server / Questions / 1080518
Accepted
basin
Asked: 2021-10-15 01:20:22 +0800 CST

RewriteRule is applied to all files during directory listing

  • 772

With Options Indexes and LogLevel rewrite:trace6 I see that my rewrite rule is applied to all files in the directory being listed. It even goes inside sub-directories and looks for index.html. This is bad for performance. Is this really necessary? How to disable it? In the actual configuration I have a "prg" RewriteMap and this program receives all listed files on its STDIN.

The problem is reproducible with this minimum of configuration:

Environment:

  • OpenSUSE Tumbleweed
  • apache2-2.4.51-1.1.x86_64

Configs:

  • /etc/apache2/conf.d/test.conf (see below the full httpd.conf)

    LoadModule rewrite_module /usr/lib64/apache2/mod_rewrite.so
LogLevel debug rewrite:trace6
<Directory "/srv/www/htdocs/test">
    Options Indexes FollowSymLinks
    RewriteEngine on
    RewriteRule badrule neverhappen
</Directory>

Commands:

mkdir /srv/www/htdocs/test
touch /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzz{a..z}
systemctl restart apache2
tail -F /var/log/apache2/error_log &
curl -s -o /dev/null http://localhost/test/

Output:

[rid#556dab9a2060/initial] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/ ->
[rid#556dab9a2060/initial] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri ''
[rid#556dab9a2060/initial] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/index.html -> index.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'index.html'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/index.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/index.html.var -> index.html.var
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'index.html.var'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/index.html.var
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/HEADER.html -> HEADER.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'HEADER.html'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/HEADER.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzza -> zzzzzzzzzzzzzzzzzzzzza
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'zzzzzzzzzzzzzzzzzzzzza'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzza
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzb -> zzzzzzzzzzzzzzzzzzzzzb
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'zzzzzzzzzzzzzzzzzzzzzb'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzb
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzc -> zzzzzzzzzzzzzzzzzzzzzc
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'zzzzzzzzzzzzzzzzzzzzzc'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzc
...

Full httpd.conf:

# sudo -u nobody /sbin/httpd -d /etc/httpd-test -f /etc/httpd-test/httpd.conf -DFOREGROUND
#
# The following is not needed on SUSE
LoadModule mpm_event_module modules/mod_mpm_event.so
#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
#LoadModule mpm_worker_module modules/mod_mpm_worker.so

# The following is not needed on SUSE
LoadModule unixd_module modules/mod_unixd.so

LoadModule authz_core_module modules/mod_authz_core.so
LoadModule autoindex_module  modules/mod_autoindex.so
LoadModule rewrite_module    modules/mod_rewrite.so

ErrorLog "|/bin/cat"
LogLevel debug rewrite:trace6
PidFile /dev/shm/httpd-test.pid

ServerName localhost

Listen 8080

DocumentRoot "/srv/www/htdocs"

<Directory "/srv/www/htdocs/test">
    Require all granted
    Options Indexes FollowSymLinks
    RewriteEngine on
    RewriteRule badrule neverhappen
</Directory>
mod-rewrite apache-2.4 directory-listing

1 Answers

  1. Best Answer

    With a directory listing generated by mod_autoindex an internal subrequest is triggered for all listed items in the directory. This is indicated in the log with subreq (as opposed to initial).

    You can prevent the RewriteRule directive being processed for subrequests by using the NS (nosubreq) flag. For example:

    RewriteRule badrule neverhappen [NS]

    Alternatively, you can move the RewriteRule directive out of the <Directory> container and have it directly under the <VirtualHost> instead. Note that the directive might need to be changed slightly since it's now working in a different context (virtualhost as opposed to directory) and processed earlier. This naturally prevents the directive being processed for each item in the directory listing (without having to include the NS flag). However, it can still be called for other subrequests, such as calls for HeaderName and ReadmeName files associated with the directory listing (if these directives are set, regardless of whether the files exist or not), so you may still choose to include the NS flag on these directives if this is an issue.

    It even goes inside sub-directories and looks for index.html

    This is caused by mod_dir testing for DirectoryIndex documents. If this is not required then it can be disabled:

    DirectoryIndex disabled

    Reference:

    • 0