bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest Auth (GH-18338)


* bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest authentication

 - The 'qop' value in the 'WWW-Authenticate' header is optional. The
   presence of 'qop' in the header should be checked before its value
   is parsed with 'split'.

Signed-off-by: Stephen Balousek <stephen@balousek.net>

* bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest authentication

 - Add NEWS item

Signed-off-by: Stephen Balousek <stephen@balousek.net>

* Update Misc/NEWS.d/next/Library/2020-02-06-05-33-52.bpo-39548.DF4FFe.rst

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 5e260e0fde211829fcb67060cfd602f4b679f802)

Co-authored-by: Stephen Balousek <sbalousek@users.noreply.github.com>
diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
index 6f6577b..fd91b9d 100644
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -1146,7 +1146,9 @@
                         req.selector)
         # NOTE: As per  RFC 2617, when server sends "auth,auth-int", the client could use either `auth`
         #     or `auth-int` to the response back. we use `auth` to send the response back.
-        if 'auth' in qop.split(','):
+        if qop is None:
+            respdig = KD(H(A1), "%s:%s" % (nonce, H(A2)))
+        elif 'auth' in qop.split(','):
             if nonce == self.last_nonce:
                 self.nonce_count += 1
             else:
@@ -1156,8 +1158,6 @@
             cnonce = self.get_cnonce(nonce)
             noncebit = "%s:%s:%s:%s:%s" % (nonce, ncvalue, cnonce, 'auth', H(A2))
             respdig = KD(H(A1), noncebit)
-        elif qop is None:
-            respdig = KD(H(A1), "%s:%s" % (nonce, H(A2)))
         else:
             # XXX handle auth-int.
             raise URLError("qop '%s' is not supported." % qop)
diff --git a/Misc/NEWS.d/next/Library/2020-02-06-05-33-52.bpo-39548.DF4FFe.rst b/Misc/NEWS.d/next/Library/2020-02-06-05-33-52.bpo-39548.DF4FFe.rst
new file mode 100644
index 0000000..4cf3248
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2020-02-06-05-33-52.bpo-39548.DF4FFe.rst
@@ -0,0 +1,2 @@
+Fix handling of header in :class:`urllib.request.AbstractDigestAuthHandler` when the optional ``qop`` parameter
+is not present.