OpenSSL compatibility for libodbc

[kojo1]

Description

OpenSSL compatibility for libodbc.

• VERIFY_NONE by default when WOLFSSL_VERIFY_NONE_DEFAULT is enabled
• Added constant and macro functions

Fixes zd#21884

Testing

Integration test with:

[ODBC application]
↓ unixODBC (libodbc.2)
├─→ psqlodbcw.so ─→ libpq.5.dylib ─┐
└─→ libtdsodbc.so ─────────┴─→ libwolfssl.44.dylib

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m0plus

• FLASH: .text +16 B (+0.0%, 64,107 B / 262,144 B, total: 24% used)

gcc-arm-cortex-m3

• FLASH: .text +40 B (+0.0%, 122,221 B / 262,144 B, total: 47% used)

gcc-arm-cortex-m4-baremetal

• FLASH: .text +64 B (+0.1%, 66,763 B / 262,144 B, total: 25% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .rodata +192 B, .text +64 B (+0.0%, 769,748 B / 1,048,576 B, total: 73% used)

gcc-arm-cortex-m4-pkcs7

• FLASH: .text +64 B (+0.0%, 212,206 B / 262,144 B, total: 81% used)

gcc-arm-cortex-m4-pq

• FLASH: .text +64 B (+0.0%, 279,048 B / 1,048,576 B, total: 27% used)

gcc-arm-cortex-m4-rsa-only

• FLASH: .text +64 B (+0.0%, 324,648 B / 1,048,576 B, total: 31% used)

gcc-arm-cortex-m7

• FLASH: .text +64 B (+0.0%, 199,902 B / 262,144 B, total: 76% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +64 B (+0.0%, 279,624 B / 1,048,576 B, total: 27% used)

gcc-arm-cortex-m7-tls13

• FLASH: .text +64 B (+0.0%, 235,792 B / 262,144 B, total: 90% used)

linuxkm-standard

• Data: __patchable_function_entries -8 B (-0.0%, 47,944 B)
  No memory changes detected for:
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• linuxkm-pie
• stm32-sim-stm32h753

[kojo1]

Retest please.

[dgarske]

Skoll Code Review

Scan type: reviewOverall recommendation: REQUEST_CHANGES
Findings: 7 total — 7 posted, 0 skipped
5 finding(s) posted as inline comments (see file-level comments below)
2 finding(s) not tied to a diff line (full detail below)

Posted findings

• [High] OBJ_find_sigid_algs stub hardcodes SHA-256/RSA and ignores sigid — wolfssl/openssl/objects.h:81-88
• [Medium] No test coverage for WOLFSSL_VERIFY_NONE_DEFAULT security-relevant default change — src/internal.c:2645-2649
• [Low] WOLFSSL_ERR_R_BUF_LIB defined as 0 (conflicts with 'no error' and OpenSSL's value) — wolfssl/openssl/err.h:46
• [Low] Garbled comment on OBJ_find_sigid_algs literal NIDs — wolfssl/openssl/objects.h:77-79
• [Low] TLS_ST_OK hardcoded to 16, coupled to internal enum position — wolfssl/openssl/ssl.h:1572

Findings not tied to a diff line

SSL_R_UNKNOWN_PROTOCOL / WRONG_VERSION_NUMBER / UNSUPPORTED_PROTOCOL numeric value changed

File: wolfssl/openssl/ssl.h:1748-1753, wolfssl/openssl/err.h:47-49
Function: N/A (macros)
Severity: Medium

These three pre-existing public compat macros were changed from expanding to VERSION_ERROR (a negative wolfSSL internal error enum) to OpenSSL numeric reason codes (WOLFSSL_SSL_R_UNKNOWN_PROTOCOL=252, WOLFSSL_SSL_R_WRONG_VERSION_NUMBER=267, WOLFSSL_SSL_R_UNSUPPORTED_PROTOCOL=258). This is more OpenSSL-accurate, but it is a behavior change: any existing application that compared these macros against wolfSSL's internally reported error code (VERSION_ERROR, e.g. from wolfSSL_get_error/SSL_get_error) will no longer match, because wolfSSL still reports VERSION_ERROR internally, not 252/267/258. I confirmed there are no internal src/*.c or tests/*.c users of these specific macros (tests compare against VERSION_ERROR directly), so the build and current tests are unaffected, but downstream consumers relying on the old mapping would silently break.

Recommendation: Confirm this value change is intended and note it as a compatibility change in the PR/changelog. If wolfSSL's error surface (ERR_get_error/ERR_GET_REASON) does not translate internal VERSION_ERROR into these OpenSSL reason codes, apps checking reason == SSL_R_WRONG_VERSION_NUMBER will never match - verify the intended consumer (libodbc/libtds) actually reads these as raw constants and not via wolfSSL's error queue.

Referenced code: wolfssl/openssl/ssl.h:1748-1753, wolfssl/openssl/err.h:47-50 (4 lines)

---

BIO compat stubs have latent multi-use hazards (fixed index, app_data aliasing)

File: wolfssl/openssl/bio.h:196, wolfssl/openssl/ssl.h:1771-1774
Function: wolfSSL_BIO_get_new_index
Severity: Medium

Two BIO compat shims work for a single custom BIO but are unsafe if used more than once: (1) wolfSSL_BIO_get_new_index() always returns the constant 1000 - in OpenSSL each call returns a distinct index, so an app that registers two custom BIO types would get identical type values and collide. (2) BIO_get_app_data/BIO_set_app_data are mapped onto wolfSSL_BIO_get_data/wolfSSL_BIO_set_data, which is the SAME storage slot that a custom BIO method already uses for its method-private data. In OpenSSL these are two independent slots; collapsing them means an app that uses both BIO_set_data (method state) and BIO_set_app_data (app state) on the same BIO will overwrite one with the other. The wolfSSL_BIO_meth_get_* getters returning NULL and set_callback_ctrl being a no-op are acceptable documented stubs.

Recommendation: Keep for the narrow libodbc/libtds use, but document the single-custom-BIO limitation in the header comment. If feasible, back get_new_index with a static incrementing counter, and keep app_data in a separate slot from method data to avoid silent corruption when both are used.

Referenced code: wolfssl/openssl/bio.h:196, wolfssl/openssl/ssl.h:1771-1774 (4 lines)

---

Review generated by Skoll

[dgarske]

#10813 (review)

[dgarske]

@kojo1 can you rebase to latest master? That might resolve the CI issues