OdbDesign
OdbDesign ODB++ Parser - C++ library, gRPC/REST server, CLI, tests
 
Loading...
Searching...
No Matches
Utf8Sanitizer.cpp File Reference

Implementation of UTF-8 validation and CP1252→UTF-8 transcoding. More...

#include "Utf8Sanitizer.h"
#include <google/protobuf/descriptor.h>
#include <google/protobuf/message.h>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <string>
Include dependency graph for Utf8Sanitizer.cpp:

Go to the source code of this file.

Namespaces

namespace  Odb
 
namespace  Odb::Lib
 
namespace  Odb::Lib::Text
 

Functions

bool Odb::Lib::Text::IsValidUtf8 (const char *data, std::size_t size) noexcept
 Validates that a byte sequence is valid UTF-8 per RFC 3629.
 
bool Odb::Lib::Text::IsValidUtf8 (std::string_view s) noexcept
 Validates that a string_view contains valid UTF-8.
 
std::string Odb::Lib::Text::ToUtf8 (std::string_view input)
 Converts input to valid UTF-8.
 
void Odb::Lib::Text::SanitizeToUtf8 (std::string &s)
 In-place convenience overload.
 
void Odb::Lib::Text::AssertAllStringFieldsAreValidUtf8 (const google::protobuf::Message &msg, std::string_view msgName)
 Debug-only assertion to verify all string fields in a message are valid UTF-8.
 

Detailed Description

Implementation of UTF-8 validation and CP1252→UTF-8 transcoding.

The CP1252→Unicode mapping table is derived from the official Microsoft mapping: https://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT

Undefined slots at 0x81, 0x8D, 0x8F, 0x90, 0x9D map to U+FFFD.

Definition in file Utf8Sanitizer.cpp.