Fix potential DoS and undefined behavior in KRB5KDF due to zero-length constant

The n_fold() function in krb5kdf.c computes LCM(blocksize, constant_len)
and then iterates from lcm - 1 down to 0. If constant_len is zero,
lcm becomes zero, leading to lcm - 1 underflowing to UINT_MAX.
Since the loop variable 'l' is signed (int), this causes undefined
behavior during integer conversion, and even if handled, would result
in an extremely long loop (DoS).

Although the KDF requires a non-empty protocol-derived constant per
RFC 3961, Section 5.1, the implementation did not explicitly reject
constant_len == 0. An empty octet string passed via OSSL_KDF_PARAM_CONSTANT
could bypass the NULL check and trigger this condition.

Add an explicit check in KRB5KDF() to reject constant_len == 0 with
PROV_R_INVALID_CONSTANT_LENGTH, alongside the existing upper-bound check.

This fixes a potential security vulnerability (DoS) and eliminates
undefined behavior.

Signed-off-by: Anton Moryakov <[email protected]>
Reviewed-by: Paul Dale <[email protected]>
Reviewed-by: Andrew Dinh <[email protected]>
Merge-date: Thu Sep 17 16:52:53 2026
Merged-from: https://github.com/openssl/openssl/pull/29743

0 files changed, 0 insertions, 0 deletions