CVE-2024-12797

A flaw was found in OpenSSL's RFC7250 Raw Public Key (RPK) authentication. This vulnerability allows man-in-the-middle (MITM) attacks via failure to abort TLS/DTLS handshakes when the server's RPK does not match the expected key despite the SSL_VERIFY_PEER verification mode being set.

Issue summary: Clients using RFC7250 Raw Public Keys (RPKs) to authenticate a server may fail to notice that the server was not authenticated, because handshakes don't abort as expected when the SSL_VERIFY_PEER verification mode is set. Impact summary: TLS and DTLS connections using raw public keys may be vulnerable to man-in-middle attacks when server authentication failure is not detected by clients. RPKs are disabled by default in both TLS clients and TLS servers. The issue only arises when TLS clients explicitly enable RPK use by the server, and the server, likewise, enables sending of an RPK instead of an X.509 certificate chain. The affected clients are those that then rely on the handshake to fail when the server's RPK fails to match one of the expected public keys, by setting the verification mode to SSL_VERIFY_PEER. Clients that enable server-side raw public keys can still find out that raw public key verification failed by calling SSL_get_verify_result(), and those that do, and take appropriate action, are not affected. This issue was introduced in the initial implementation of RPK support in OpenSSL 3.2. The FIPS modules in 3.4, 3.3, 3.2, 3.1 and 3.0 are not affected by this issue.

It was discovered that OpenSSL clients incorrectly handled authenticating servers using RFC7250 Raw Public Keys. In certain cases, the connection will not abort as expected, possibly causing the communication to be intercepted. George Pantelakis and Alicja Kario discovered that OpenSSL had a timing side-channel when performing ECDSA signature computations. A remote attacker could possibly use this issue to recover private data.