Whole file

python-validators/validators

The author described this change as “fix: Valid URLs failing validation - query and fragment parts (#297)”. It counts as a record because the check below fails on the code as it stood at 71b40bd54 and passes on 960b48b8a, with nothing else changed between the two runs.

Projectpython-validators/validators

Fix saved2023-09-17

Sharing licenceMIT · LICENSE.txt

Change size+9 −11

What the code was meant to do, written into the code itself as a save note

fix: Valid URLs failing validation - query and fragment parts (#297)

The change

3	3		# standard
4	4		from functools import lru_cache
5	5		import re
6		-	from urllib.parse import unquote, urlsplit
	6	+	from urllib.parse import parse_qs, unquote, urlsplit
7	7
8	8		# local
9	9		from .hostname import hostname
⋯
34	34		)
35	35
36	36
37		-	@lru_cache
38		-	def _query_regex():
39		-	return re.compile(r"&?(\w+=?[^\s&]*)", re.IGNORECASE)
40		-
41		-
42	37		def _validate_scheme(value: str):
43	38		"""Validate scheme."""
44	39		# More schemes will be considered later.
⋯
108	103		) and _validate_auth_segment(basic_auth)
109	104
110	105
111		-	def _validate_optionals(path: str, query: str, fragment: str):
	106	+	def _validate_optionals(path: str, query: str, fragment: str, strict_query: bool):
112	107		"""Validate path query and fragments."""
113	108		optional_segments = True
114	109		if path:
115	110		optional_segments &= bool(_path_regex().match(path))
116		-	if query:
117		-	optional_segments &= bool(_query_regex().match(query))
	111	+	if query and parse_qs(query, strict_parsing=strict_query):
	112	+	optional_segments &= True
118	113		if fragment:
119	114		fragment = fragment.lstrip("/") if fragment.startswith("/") else fragment
120		-	optional_segments &= all(char_to_avoid not in fragment for char_to_avoid in ("/", "?"))
	115	+	optional_segments &= all(char_to_avoid not in fragment for char_to_avoid in ("?",))
121	116		return optional_segments
122	117
123	118
⋯
130	125		skip_ipv4_addr: bool = False,
131	126		may_have_port: bool = True,
132	127		simple_host: bool = False,
	128	+	strict_query: bool = True,
133	129		rfc_1034: bool = False,
134	130		rfc_2782: bool = False,
135	131		):
⋯
167	163		URL string may contain port number.
168	164		simple_host:
169	165		URL string maybe only hyphens and alpha-numerals.
	166	+	strict_query:
	167	+	Fail validation on query string parsing error.
170	168		rfc_1034:
171	169		Allow trailing dot in domain/host name.
172	170		Ref: [RFC 1034](https://www.rfc-editor.org/rfc/rfc1034).
⋯
214	212		rfc_1034,
215	213		rfc_2782,
216	214		)
217		-	and _validate_optionals(path, query, fragment)
	215	+	and _validate_optionals(path, query, fragment, strict_query)
218	216		)
219	217

The check that tells the two apart

fail→pass·tests/test_url.py::test_returns_true_on_valid_url[https://www.foo.com/bar#/baz/test]

Check file tests/test_url.py, taken without changes from the fix and copied onto the older code, so the exact same check runs against both versions.

Origin and history

The fix960b48b8a548fa5b104f1999ef0c18923a232c52

The code before it71b40bd54ecf88de792b15acc831801903cc6bc0

Broken version dated2023-09-02

Modulevalidators.url

Units changed_validate_optionals, url

Fingerprint44f504539f1a9e4a

Checked2026-08-18 by goldset/0.1

Every field above is generated by our program. None of it is written by hand.

Other bugs found in python-validators/validators

2025-03-28email
2024-05-09fix(ip_address): properly handle private is false
2024-04-18domain