Skip to main content

Dangerous Requests

The Dangerous Requests package protects applications against SSRF (Server-Side Request Forgery) vulnerabilities when using the requests library to make HTTP requests. The package protects against SSRF by providing means to blacklist dangerous requests and whitelist safe ones, and then patching the requests library so that the rules are adhered to.

Under the hood, Dangerous Requests leverages on the advocate library to deliver its functionality.

info

Server-side request forgery, or SSRF for short, is a type of vulnerability that allows an attacker to trick a server-side application into making network requests to an unintended location. This could be an internal service within the organization's network or an external service to which the application is authenticated.

SSRF can lead to the leakage of privileged information such as authorization credentials, which can allow an attacker to escalate their privileges and gain access to more systems/data.

Installation​

The Dangerous Requests package can be installed using the command below:

pip install govtech-csg-xcg-dangerousrequests

Source code for the package can be found at https://github.com/GovTech-CSG/govtech-csg-xcg-dangerousrequests.

Setup​

In order to enable Dangerous Requests, you will need to add the govtech_csg_xcg.dangerousrequests app to the INSTALLED_APPS list in your settings.py file.

./<PROJECT_NAME>/settings.py
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    ...,
    'govtech_csg_xcg.dangerousrequests',
]

Logging​

The package uses its own logger with the name dangerousrequests. By default, this logger uses a stream handler and emits messages with level INFO and above. The log message format looks something like this:

2023-06-22 09:50:43,348 [INFO][XCG][dangerousrequests]: This is an informational message.

To safely override the default logging configuration, you can write code in settings.py to import the logger from the package and modify it directly. The example below sets the logger's level to DEBUG:

settings.py
from govtech_csg_xcg.dangerousrequests import logger
import logging

logger.setLevel(logging.DEBUG)

For more information on the Python logging library, see the official documentation.

Usage​

Basics​

Once Dangerous Requests is set up, you can use the requests library as you usually would (e.g. response = requests.get('https://httpbin.org/anything')). By default, dangerousrequests will prevent any request made to localhost and private IP addresses (or domains that resolve to a private IP address). To whitelist or blacklist any other URLs or IP addresses, see the detailed configuration guide below. Any attempts to make a request for a disallowed URL/IP address will raise advocate.exceptions.UnacceptableAddressException.

Dangerous Requests covers the following requests methods:

requests.get
requests.delete
requests.head
requests.post
requests.put
requests.session
requests.options
requests.patch
requests.request
Use of requests.Session

dangerousrequests should work regardless of whether you invoke the methods via the requests module object, or via a custom requests.Session object that you create in your app.

Configuration​

You can customize the behaviour of dangerousrequests by providing a Python dictionary named XCG_SECURITY with the key "requests" in settings.py. Both "blacklist" and "whitelist" configuration approaches are supported. A basic example is shown below:

./<PROJECT_NAME>/settings.py
import ipaddress

XCG_SECURITY = {
    'requests': {
        'ip_blacklist': {ipaddress.ip_network("37.19.221.152")},
        'ip_whitelist': {ipaddress.ip_network("127.0.0.1")},
        'port_whitelist': {9001, 80 ,443},
        'hostname_blacklist': {'google.com'},
        'request_blacklist': {('example.com', 'GET', '/foo'), 'httpbin.org'},
        'request_whitelist': {'httpbin.org', 'POST', '/post'},
        'allow_ipv6': True,
    }
}

The above configuration:

  • Blocks requests to 37.19.221.152
  • Allows requests to 127.0.0.1
  • Allows requests made to ports 9001, 80, and 443 only
  • Blocks requests made to google.com
  • Blocks requests made to example.com/foo using the HTTP GET method
  • Blocks all requests made to httpbin.org except for a POST request made to httpbin.org/post
  • Allows requests made to IPv6 addresses (IPv6 is blocked by default)

The full set of supported configuration options are given in the table below:

KeyValueDefault
ip_blacklistset of ipaddress.ip_network objectsEmpty set
ip_whitelistset of ipaddress.ip_network objectsEmpty set
port_whitelistset of int{80, 443, 8443, 8000}
port_blacklistset of intEmpty set
hostname_blacklistset of strEmpty set
request_blacklistset of str and/or tupleEmpty set
request_whitelistset of str and/or tupleEmpty set

Notes on IP address white/blacklisting

  • ip_whitelist has priority over ip_blacklist - this allows you to blacklist a broad IP range but whitelisting specific subsets of that range.

Notes on port white/blacklisting

  • All ports (excluding default whitelisted ports) are disallowed unless explicitly whitelisted.
  • port_blacklist has priority over port_whitelist.

Notes on request white/blacklisting

  • Each element in the request_blacklist and request_whitelist sets can be either:
    1. A str, which should indicate a hostname (e.g. secretsmanager.ap-southeast-1.amazonaws.com).
    2. A tuple in the form (HOSTNAME, HTTP_METHOD, PATH) (e.g. ('api.cloudflare.com', 'POST', '/anything')).
  • request_blacklist and request_whitelist hostnames support both wildcards (*) and regex, but the PATH value in tuple form only supports regex.
  • request_whitelist has priority over request_blacklist - this allows you to blacklist generic requests to specific hostnames, but whitelist specific methods/paths.

Notes on hostname white/blacklisting

  • hostname_blacklist has priority over request_whitelist and request_blacklist - e.g. if google.com is included in hostname_blacklist, requests to it will be blocked even if request_whitelist allows it.