OdbDesign
OdbDesign ODB++ Parser - C++ library, gRPC/REST server, CLI, tests
 
Loading...
Searching...
No Matches
Odb::Lib::Text Namespace Reference

Functions

bool IsValidUtf8 (const char *data, std::size_t size) noexcept
 Validates that a byte sequence is valid UTF-8 per RFC 3629.
 
bool IsValidUtf8 (std::string_view s) noexcept
 Validates that a string_view contains valid UTF-8.
 
std::string ToUtf8 (std::string_view input)
 Converts input to valid UTF-8.
 
void SanitizeToUtf8 (std::string &s)
 In-place convenience overload.
 
void AssertAllStringFieldsAreValidUtf8 (const google::protobuf::Message &msg, std::string_view msgName)
 Debug-only assertion to verify all string fields in a message are valid UTF-8.
 

Function Documentation

◆ AssertAllStringFieldsAreValidUtf8()

void Odb::Lib::Text::AssertAllStringFieldsAreValidUtf8 ( const google::protobuf::Message &  msg,
std::string_view  msgName 
)

Debug-only assertion to verify all string fields in a message are valid UTF-8.

Recursively walks the protobuf message via the reflection API and asserts that every string field contains valid UTF-8: singular strings, repeated strings, and strings nested inside sub-messages and map entries (map keys and values). bytes fields are intentionally skipped — they may hold arbitrary binary data. It is only active in debug builds (#ifndef NDEBUG); in release builds it compiles to a no-op.

On failure the offending field path and a hex-escaped sample of the value are written to stderr and the process aborts.

Parameters
msgThe protobuf message to validate
msgNameA descriptive name for the message type (used in assertion messages)

Definition at line 403 of file Utf8Sanitizer.cpp.

Here is the caller graph for this function:

◆ IsValidUtf8() [1/2]

bool Odb::Lib::Text::IsValidUtf8 ( const char *  data,
std::size_t  size 
)
noexcept

Validates that a byte sequence is valid UTF-8 per RFC 3629.

Checks for:

  • Correct continuation byte ranges
  • No overlong encodings
  • No surrogate halves (U+D800..U+DFFF)
  • No code points above U+10FFFF
Parameters
dataPointer to the byte sequence
sizeLength in bytes
Returns
true if valid UTF-8, false otherwise

Definition at line 332 of file Utf8Sanitizer.cpp.

Here is the caller graph for this function:

◆ IsValidUtf8() [2/2]

bool Odb::Lib::Text::IsValidUtf8 ( std::string_view  s)
noexcept

Validates that a string_view contains valid UTF-8.

Parameters
sThe string to validate
Returns
true if valid UTF-8, false otherwise

Definition at line 349 of file Utf8Sanitizer.cpp.

Here is the call graph for this function:

◆ SanitizeToUtf8()

void Odb::Lib::Text::SanitizeToUtf8 ( std::string &  s)

In-place convenience overload.

Replaces the contents of s with ToUtf8(s) only if validation fails. Avoids allocation when already valid UTF-8.

May throw std::bad_alloc when transcoding is required (see ToUtf8).

Parameters
sThe string to sanitize

Definition at line 395 of file Utf8Sanitizer.cpp.

Here is the call graph for this function:

◆ ToUtf8()

std::string Odb::Lib::Text::ToUtf8 ( std::string_view  input)

Converts input to valid UTF-8.

  • If input is already valid UTF-8, returns a copy unchanged.
  • Otherwise, repairs it: every maximal valid UTF-8 sequence is kept as-is and each remaining invalid byte is decoded as Windows-1252 and re-encoded as UTF-8. Transcoding the whole string as CP1252 would corrupt the already-valid parts of mixed input into mojibake.

CP1252 differs from ISO-8859-1 only in the 0x80–0x9F range. The five undefined CP1252 slots (0x81, 0x8D, 0x8F, 0x90, 0x9D) are mapped to U+FFFD (REPLACEMENT CHARACTER).

Never returns invalid UTF-8. May throw std::bad_alloc on allocation failure; callers in the serialization path handle it like any other allocation failure (the per-RPC try/catch translates it to INTERNAL).

Parameters
inputThe raw byte sequence from ODB++ files
Returns
A valid UTF-8 string

Definition at line 354 of file Utf8Sanitizer.cpp.

Here is the call graph for this function:
Here is the caller graph for this function: