Python“请求”库-定义特定的DNS?

2024-03-28 22:16:33 发布

您现在位置:Python中文网/ 问答频道 /正文

在我的项目中,我使用python^{} library处理所有HTTP请求。

现在,我需要使用特定的DNS查询http服务器-有两个环境,每个环境都使用自己的DNS,并且更改是独立进行的。

所以,当代码运行时,它应该使用特定于环境的DNS,而不是在我的internet连接中指定的DNS。

有人用python请求尝试过吗?我只找到了urllib2的解决方案:
https://stackoverflow.com/questions/4623090/python-set-custom-dns-server-for-urllib-requests


Tags: 项目https服务器comhttp环境dnslibrary
3条回答

您应该查看TransportAdapters,包括源代码。关于它们的文档不是很好,但是它们提供了对RFC 2818RFC 6125中描述的许多功能的低级访问。特别是,这些文件鼓励(要求?)客户端代码,用于支持特定于应用程序的DNS,以检查证书的CommonName和SubjectAltName。这些调用中需要的关键字参数是“assert_hostname”。以下是如何使用请求库设置它:

from requests import Session, HTTPError
from requests.adapters import HTTPAdapter, DEFAULT_POOLSIZE, DEFAULT_RETRIES, DEFAULT_POOLBLOCK


class DNSResolverHTTPSAdapter(HTTPAdapter):
    def __init__(self, common_name, host, pool_connections=DEFAULT_POOLSIZE, pool_maxsize=DEFAULT_POOLSIZE,
        max_retries=DEFAULT_RETRIES, pool_block=DEFAULT_POOLBLOCK):
        self.__common_name = common_name
        self.__host = host
        super(DNSResolverHTTPSAdapter, self).__init__(pool_connections=pool_connections, pool_maxsize=pool_maxsize,
            max_retries=max_retries, pool_block=pool_block)

    def get_connection(self, url, proxies=None):
        redirected_url = url.replace(self.__common_name, self.__host)
        return super(DNSResolverHTTPSAdapter, self).get_connection(redirected_url, proxies=proxies)

    def init_poolmanager(self, connections, maxsize, block=DEFAULT_POOLBLOCK, **pool_kwargs):
        pool_kwargs['assert_hostname'] = self.__common_name
        super(DNSResolverHTTPSAdapter, self).init_poolmanager(connections, maxsize, block=block, **pool_kwargs)

common_name = 'SuperSecretSarahServer'
host = '192.168.33.51'
port = 666
base_url = 'https://{}:{}/api/'.format(common_name, port)
my_session = Session()
my_session.mount(self.base_url.lower(), DNSResolverHTTPSAdapter(common_name, host))
user_name = 'sarah'
url = '{}users/{}'.format(self.base_url, user_name)
default_response_kwargs = {
    'auth': (NAME, PASSWORD),
    'headers': {'Content-Type': 'application/json'},
    'verify': SSL_OPTIONS['ca_certs'],
    'cert': (SSL_OPTIONS['certfile'], SSL_OPTIONS['keyfile'])
}
response = my_session.get(url, **default_response_kwargs)

我使用common_name表示证书上预期的名称,以及代码将如何引用所需的计算机。我使用host作为外部世界识别的名称-FQDN、IP、DNS条目。。。当然,SSL_选项字典(在我的示例中)必须在您的计算机上列出适当的证书/密钥文件名。(另外,名称和密码应该解析为正确的字符串。)

一个定制的HTTPAdapter可以做到这一点。

别忘了将server_hostname设置为启用SNI

import requests


class HostHeaderSSLAdapter(requests.adapters.HTTPAdapter):
    def resolve(self, hostname):
        # a dummy DNS resolver
        import random
        ips = [
            '104.16.89.20',  # CloudFlare
            '151.101.2.109',  # Fastly
        ]
        resolutions = {
            'cdn.jsdelivr.net': random.choice(ips),
        }
        return resolutions.get(hostname)

    def send(self, request, **kwargs):
        from urllib.parse import urlparse

        connection_pool_kwargs = self.poolmanager.connection_pool_kw

        result = urlparse(request.url)
        resolved_ip = self.resolve(result.hostname)

        if result.scheme == 'https' and resolved_ip:
            request.url = request.url.replace(
                'https://' + result.hostname,
                'https://' + resolved_ip,
            )
            connection_pool_kwargs['server_hostname'] = result.hostname  # SNI
            connection_pool_kwargs['assert_hostname'] = result.hostname

            # overwrite the host header
            request.headers['Host'] = result.hostname
        else:
            # theses headers from a previous request may have been left
            connection_pool_kwargs.pop('server_hostname', None)
            connection_pool_kwargs.pop('assert_hostname', None)

        return super(HostHeaderSSLAdapter, self).send(request, **kwargs)


url = 'https://cdn.jsdelivr.net/npm/bootstrap/LICENSE'

session = requests.Session()
session.mount('https://', HostHeaderSSLAdapter())

r = session.get(url)
print(r.headers)

r = session.get(url)
print(r.headers)

requests使用了urllib3,最终也使用了httplib.HTTPConnection,因此在一定程度上,来自^{s>https://stackoverflow.com/questions/4623090/python-set-custom-dns-server-for-urllib-requests的技术(现在已经删除,它仅仅链接到Tell urllib2 to use custom DNS)仍然适用。

同名的urllib3.connection模块子类httplib.HTTPConnection,用调用self._new_conn的方法替换了.connect()方法。反过来,这个委托给urllib3.util.connection.create_connection()。可能最容易修补功能:

from urllib3.util import connection


_orig_create_connection = connection.create_connection


def patched_create_connection(address, *args, **kwargs):
    """Wrap urllib3's create_connection to resolve the name elsewhere"""
    # resolve hostname to an ip address; use your own
    # resolver here, as otherwise the system resolver will be used.
    host, port = address
    hostname = your_dns_resolver(host)

    return _orig_create_connection((hostname, port), *args, **kwargs)


connection.create_connection = patched_create_connection

您可以提供自己的代码来将地址的host部分解析为一个ip地址,而不是依赖connection.create_connection()调用(它包装socket.create_connection())来为您解析主机名。

像所有的monkeypatching一样,要注意代码在以后的版本中没有明显的变化;这里的补丁是针对urllib3版本1.21.1创建的。但最早应该在1.9版本中使用。


请注意,重新编写此答案是为了与较新的urllib3版本一起使用,这些版本添加了更方便的修补位置。请参阅适用于<;1.9版的旧方法的编辑历史记录,该方法是对所提供的urllib3版本的修补程序,而不是独立安装。

相关问题 更多 >