Computer Configuration > Administrative Templates > Microsoft Edge > Network settings
Both
HKLM\Software\Policies\Microsoft\Edge
This policy enables an override of the IPv6 reachability check. When overridden, the system will always query AAAA records when resolving host names. It applies to all users and interfaces on the device. If you enable this policy, the IPv6 reachability check will be overridden. If you disable or don't configure this policy, the IPv6 reachability check will not be overridden. The system only queries AAAA records when it is reachable to a global IPv6 host.