OdbDesign
OdbDesign ODB++ Parser - C++ library, gRPC/REST server, CLI, tests
 
Loading...
Searching...
No Matches
Utf8Sanitizer.h File Reference

UTF-8 validation and Windows-1252 to UTF-8 transcoding utilities. More...

#include <cstddef>
#include <string>
#include <string_view>
#include <google/protobuf/message.h>
Include dependency graph for Utf8Sanitizer.h:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces

namespace  Odb
 
namespace  Odb::Lib
 
namespace  Odb::Lib::Text
 

Functions

bool Odb::Lib::Text::IsValidUtf8 (const char *data, std::size_t size) noexcept
 Validates that a byte sequence is valid UTF-8 per RFC 3629.
 
bool Odb::Lib::Text::IsValidUtf8 (std::string_view s) noexcept
 Validates that a string_view contains valid UTF-8.
 
std::string Odb::Lib::Text::ToUtf8 (std::string_view input)
 Converts input to valid UTF-8.
 
void Odb::Lib::Text::SanitizeToUtf8 (std::string &s)
 In-place convenience overload.
 
void Odb::Lib::Text::AssertAllStringFieldsAreValidUtf8 (const google::protobuf::Message &msg, std::string_view msgName)
 Debug-only assertion to verify all string fields in a message are valid UTF-8.
 

Detailed Description

UTF-8 validation and Windows-1252 to UTF-8 transcoding utilities.

@why This exists because ODB++ design files exported from European EDA tools often contain legacy 8-bit text encoded in Windows-1252 (CP1252) or ISO-8859-1. When these raw bytes are copied verbatim into Protobuf string fields, they produce invalid UTF-8 on the wire. Strict Protobuf clients (including .NET's Google.Protobuf) refuse to parse such messages and throw InvalidProtocolBufferException: "String is invalid UTF-8".

This module provides:

  • RFC 3629 compliant UTF-8 validation
  • CP1252 → UTF-8 transcoding for invalid sequences
  • Zero-allocation fast path for already-valid UTF-8

Definition in file Utf8Sanitizer.h.