Are there situations were activedefrag should be kept disabled in Redis 5 (with Jemalloc allocator)?

Redis 4 added active memory defragmentation (source: release notes):

Active memory defragmentation. Redis is able to defragment the memory while online if the Jemalloc allocator is used (the default on Linux). Useful for workloads where the allocator cannot keep the fragmentation low enough, so the only possibility is for Redis and the allocator to collaborate in order to defragment the memory.

With Redis 5, the feature (now refered to as version 2) has been improved:

Source 1: tweet from Salvatore Sanfilippo, the Redis main developer

Active defragmentation version 2. Defragmenting the memory of a running server is black magic, but Oran Agra improved his past effort and now it works better than before. Very useful for long running workloads that tend to fragment Jemalloc.

Source 2: AWS announcement of Redis 5

One of the highlights of the previous release was the fact that Redis gained the capability to defragment the memory while online. The way it works is very clever: Redis scans the keyspace and, for each pointer, asks the allocator if moving it to a new address would help to reduce the fragmentation. This release ships with what can be called active defrag 2: It's faster, smarter, and has lower latency. This feature is especially useful for workloads where the allocator cannot keep the fragmentation low enough, so the strategy is for both Redis and the allocator to cooperate. For this to work, the Jemalloc allocator has to be used. Luckily, it's the default allocator on Linux.

Question: Assuming you are already using Jemalloc, is there any reason not to always set activedefrag yes?

Given that the alternative is to restart the instance to deal with fragmentation (which is highly problematic), and given that the overhead of activedefrag seems quite low from what I saw so far, the option seems to be too useful to be disabled by default.

Or are there any situations where it will harm performance?