Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Graceful upgrade path for 0.28. #3394

Merged
merged 8 commits into from
Nov 12, 2024
Merged

Graceful upgrade path for 0.28. #3394

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d26a838
Graceful upgrade path for 0.28
tomchristie Nov 7, 2024
273c0da
Maintain but deprecate URL.raw
tomchristie Nov 7, 2024
beabd88
Graceful upgrade path for 0.28
tomchristie Nov 7, 2024
3823615
Tweak
tomchristie Nov 7, 2024
977b0d2
Linting
tomchristie Nov 7, 2024
0506adb
Linting
tomchristie Nov 7, 2024
1be413d
Sharpening up SSL docs
tomchristie Nov 7, 2024
0f9cd3f
Sharpening up SSL docs
tomchristie Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

This release introduces an `httpx.SSLContext()` class and `ssl_context` parameter.
The 0.28 release includes a limited set of backwards incompatible changes.

* Added `httpx.SSLContext` class and `ssl_context` parameter. (#3022, #3335)
* The `verify` and `cert` arguments have been deprecated and will now raise warnings. (#3022, #3335)
**Backwards incompatible changes**:

SSL configuration has been significantly simplified.

* The `verify` argument no longer accepts string arguments.
* The `cert` argument has now been removed.
* The `SSL_CERT_FILE` and `SSL_CERT_DIR` environment variables are no longer automatically used.

For users of the standard `verify=True` or `verify=False` cases this should require no changes.

For information on configuring more complex SSL cases, please see the [SSL documentation](docs/advanced/ssl.md).

**The following changes are also included**:

* The undocumented `URL.raw` property has now been deprecated, and will raise warnings.
* The deprecated `proxies` argument has now been removed.
* The deprecated `app` argument has now been removed.
* The `URL.raw` property has now been removed.
* Ensure JSON request bodies are compact. (#3363)
* Review URL percent escape sets, based on WHATWG spec. (#3371, #3373)
* Ensure `certifi` and `httpcore` are only imported if required. (#3377)
Expand Down
182 changes: 42 additions & 140 deletions docs/advanced/ssl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,191 +9,93 @@ By default httpx will verify HTTPS connections, and raise an error for invalid S
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
```

Verification is configured through [the SSL Context API](https://docs.python.org/3/library/ssl.html#ssl-contexts).
You can disable SSL verification completely and allow insecure requests...

```pycon
>>> context = httpx.SSLContext()
>>> context
<SSLContext(verify=True)>
>>> httpx.get("https://www.example.com", ssl_context=context)
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
```

You can use this to disable verification completely and allow insecure requests...

```pycon
>>> context = httpx.SSLContext(verify=False)
>>> context
<SSLContext(verify=False)>
>>> httpx.get("https://expired.badssl.com/", ssl_context=context)
>>> httpx.get("https://expired.badssl.com/", verify=False)
<Response [200 OK]>
```

### Configuring client instances

If you're using a `Client()` instance you should pass any SSL context when instantiating the client.

```pycon
>>> context = httpx.SSLContext()
>>> client = httpx.Client(ssl_context=context)
```

The `client.get(...)` method and other request methods on a `Client` instance *do not* support changing the SSL settings on a per-request basis.

If you need different SSL settings in different cases you should use more than one client instance, with different settings on each. Each client will then be using an isolated connection pool with a specific fixed SSL configuration on all connections within that pool.

### Configuring certificate stores

By default, HTTPX uses the CA bundle provided by [Certifi](https://pypi.org/project/certifi/).

You can load additional certificate verification using the [`.load_verify_locations()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locations) API:

```pycon
>>> context = httpx.SSLContext()
>>> context.load_verify_locations(cafile="path/to/certs.pem")
>>> client = httpx.Client(ssl_context=context)
>>> client.get("https://www.example.com")
<Response [200 OK]>
```

Or by providing an certificate directory:

```pycon
>>> context = httpx.SSLContext()
>>> context.load_verify_locations(capath="path/to/certs")
>>> client = httpx.Client(ssl_context=context)
>>> client.get("https://www.example.com")
<Response [200 OK]>
```
If you're using a `Client()` instance you should pass any `verify=<...>` configuration when instantiating the client.

### Client side certificates
By default the [certifi CA bundle](https://certifiio.readthedocs.io/en/latest/) is used for SSL verification.

You can also specify a local cert to use as a client-side certificate, using the [`.load_cert_chain()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_cert_chain) API:
For more complex configurations you can pass an [SSL Context](https://docs.python.org/3/library/ssl.html) instance...

```pycon
>>> context = httpx.SSLContext()
>>> context.load_cert_chain(certfile="path/to/client.pem")
>>> httpx.get("https://example.org", ssl_context=ssl_context)
<Response [200 OK]>
```

Or including a keyfile...

```pycon
>>> context = httpx.SSLContext()
>>> context.load_cert_chain(
certfile="path/to/client.pem",
keyfile="path/to/client.key"
)
>>> httpx.get("https://example.org", ssl_context=context)
<Response [200 OK]>
```

Or including a keyfile and password...
```python
import certifi
import httpx
import ssl

```pycon
>>> context = httpx.SSLContext(cert=cert)
>>> context = httpx.SSLContext()
>>> context.load_cert_chain(
certfile="path/to/client.pem",
keyfile="path/to/client.key"
password="password"
)
>>> httpx.get("https://example.org", ssl_context=context)
<Response [200 OK]>
# This SSL context is equivelent to the default `verify=True`.
ctx = ssl.create_default_context(cafile=certifi.where())
client = httpx.Client(verify=ctx)
```

### Using alternate SSL contexts

You can also use an alternate `ssl.SSLContext` instances.

For example, [using the `truststore` package](https://truststore.readthedocs.io/)...
Using [the `truststore` package](https://truststore.readthedocs.io/) to support system certificate stores...

```python
import ssl
import truststore
import httpx

ssl_context = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
client = httpx.Client(ssl_context=ssl_context)
# Use system certificate stores.
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
client = httpx.Client(verify=ctx)
```

Or working [directly with Python's standard library](https://docs.python.org/3/library/ssl.html)...
Loding an alternative certificate verification store using [the standard SSL context API](https://docs.python.org/3/library/ssl.html)...

```python
import ssl
import httpx
import ssl

ssl_context = ssl.create_default_context()
client = httpx.Client(ssl_context=ssl_context)
# Use an explicitly configured certificate store.
ctx = ssl.create_default_context(cafile="path/to/certs.pem") # Either cafile or capath.
client = httpx.Client(verify=ctx)
```

### Working with `SSL_CERT_FILE` and `SSL_CERT_DIR`
### Client side certificates

Unlike `requests`, the `httpx` package does not automatically pull in [the environment variables `SSL_CERT_FILE` or `SSL_CERT_DIR`](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_default_verify_paths.html). If you want to use these they need to be enabled explicitly.
Client side certificates allow a remote server to verify the client. They tend to be used within private organizations to authenticate requests to remote servers.

For example...
You can specify client-side certificates, using the [`.load_cert_chain()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_cert_chain) API...

```python
context = httpx.SSLContext()

# Use `SSL_CERT_FILE` or `SSL_CERT_DIR` if configured.
if os.environ.get("SSL_CERT_FILE") or os.environ.get("SSL_CERT_DIR"):
context.load_verify_locations(
cafile=os.environ.get("SSL_CERT_FILE"),
capath=os.environ.get("SSL_CERT_DIR"),
)
ctx = ssl.create_default_context()
ctx.load_cert_chain(certfile="path/to/client.pem") # Optionally also keyfile or password.
client = httpx.Client(verify=ctx)
```

## `SSLKEYLOGFILE`

Valid values: a filename

If this environment variable is set, TLS keys will be appended to the specified file, creating it if it doesn't exist, whenever key material is generated or received. The keylog file is designed for debugging purposes only.
### Working with `SSL_CERT_FILE` and `SSL_CERT_DIR`

Support for `SSLKEYLOGFILE` requires Python 3.8 and OpenSSL 1.1.1 or newer.
Unlike `requests`, the `httpx` package does not automatically pull in [the environment variables `SSL_CERT_FILE` or `SSL_CERT_DIR`](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_default_verify_paths.html). If you want to use these they need to be enabled explicitly.

Example:
For example...

```python
# test_script.py
import httpx

with httpx.Client() as client:
r = client.get("https://google.com")
```

```console
SSLKEYLOGFILE=test.log python test_script.py
cat test.log
# TLS secrets log file, generated by OpenSSL / Python
SERVER_HANDSHAKE_TRAFFIC_SECRET XXXX
EXPORTER_SECRET XXXX
SERVER_TRAFFIC_SECRET_0 XXXX
CLIENT_HANDSHAKE_TRAFFIC_SECRET XXXX
CLIENT_TRAFFIC_SECRET_0 XXXX
SERVER_HANDSHAKE_TRAFFIC_SECRET XXXX
EXPORTER_SECRET XXXX
SERVER_TRAFFIC_SECRET_0 XXXX
CLIENT_HANDSHAKE_TRAFFIC_SECRET XXXX
CLIENT_TRAFFIC_SECRET_0 XXXX
# Use `SSL_CERT_FILE` or `SSL_CERT_DIR` if configured.
# Otherwise default to certifi.
ctx = ssl.create_default_context(
cafile=os.environ.get("SSL_CERT_FILE", certifi.where()),
capath=os.environ.get("SSL_CERT_DIR"),
)
client = httpx.Client(verify=ctx)
```

### Making HTTPS requests to a local server

When making requests to local servers, such as a development server running on `localhost`, you will typically be using unencrypted HTTP connections.

If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it:
If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it...

1. Use [trustme](https://github.com/python-trio/trustme) to generate a pair of server key/cert files, and a client cert file.
2. Pass the server key/cert files when starting your local server. (This depends on the particular web server you're using. For example, [Uvicorn](https://www.uvicorn.org) provides the `--ssl-keyfile` and `--ssl-certfile` options.)
3. Tell HTTPX to use the certificates stored in `client.pem`:
3. Configure `httpx` to use the certificates stored in `client.pem`.

```pycon
>>> import httpx
>>> context = httpx.SSLContext()
>>> context.load_verify_locations(cafile="/tmp/client.pem")
>>> r = httpx.get("https://localhost:8000", ssl_context=context)
>>> r
Response <200 OK>
```python
ctx = ssl.create_default_context(cafile="client.pem")
client = httpx.Client(verify=ctx)
```
9 changes: 3 additions & 6 deletions docs/contributing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,12 +210,9 @@ configure HTTPX as described in the
the [SSL certificates section](https://www.python-httpx.org/advanced/ssl/),
this is where our previously generated `client.pem` comes in:

```
import httpx

with httpx.Client(proxy="http://127.0.0.1:8080/", verify="/path/to/client.pem") as client:
response = client.get("https://example.org")
print(response.status_code) # should print 200
```python
ctx = ssl.create_default_context(cafile="/path/to/client.pem")
client = httpx.Client(proxy="http://127.0.0.1:8080/", verify=ctx)
```

Note, however, that HTTPS requests will only succeed to the host specified
Expand Down
1 change: 0 additions & 1 deletion httpx/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ def main() -> None: # type: ignore
"RequestNotRead",
"Response",
"ResponseNotRead",
"SSLContext",
"stream",
"StreamClosed",
"StreamConsumed",
Expand Down
Loading