puppetserver gem doesn't work with IPv6

Puppet Server Version: 6.11.0
OS Name/Version: Debian GNU/Linux 10

Desired Behavior:
/opt/puppetlabs/bin/puppetserver gem install --no-user-install --verbose r10k should install the gem on an IPv6-only system.

Actual Behavior:

ERROR: Could not find a valid gem 'r10k' (>= 0), here is why:
Unable to download data from https://rubygems.org/ - Errno::ENETUNREACH: Network is unreachable - Failed to open TCP connection to rubygems.org:443 (Network is unreachable - Network is unreachable) (https://rubygems.org/specs.4.8.gz)

This appears to be a problem with Puppetserver's Java, because installing gems using the regular system gem works fine from this IPv6-only machine. This is on a completely default, fresh Debian and puppetserver installation. The machine has no legacy IP address (other than 127.0.0.1). Doing the same thing on a VM that does have legacy IP access, puppetserver gem install works fine.