Unicode mapping issues when saving to a Database

Introduction

This article explains how to prevent character mapping issues when sending data to a database.

The most common mapping issue is “losing” character accents. You can also have issues with other extended character sets like Hebrew/Japanese/Chinese or other symbolic characters being mapped to some arbitrary/meaningless characters.

Most modern databases use Unicode for communication — which means they expect to receive Unicode encoded data. However Iguana database connections do not use Unicode by default, therefore you will need to enable Unicode if the default settings are not working and you have mapping issues (like “losing” character accents).

Examples of “losing” accents:

  • “åàá” > is converted to > “aaa”
  • “êëé” > is converted to > “eee”
  • etc.

Issue [top]

You have mapping issues with Unicode extended character sets:

  • The most common issue is when accented characters are “lost” (converted to their un-accented  equivalents) when saving data to a database
  • Other extended character sets like Hebrew/Japanese/Chinese or symbolic characters can also be mapped to some arbitrary/meaningless characters

Solution [top]

You need to enable Unicode communication by setting use_unicode=true when you connect to a modern Database, like Microsoft SQL Server, or Oracle etc.

This is an example connection for Microsoft SQL Server:

db.connect{api=db.SQL_SERVER, name=”NAME”, user=”MyUser”, password=”MyPassword”, use_unicode=true}

These are example connections for Oracle:

db.connect{api=db.ORACLE_OCI, name=”NAME”, user=”MyUser”, password=”MyPassword”, use_unicode=true}
db.connect{api=db.ORACLE_ODBC, name=”NAME”, user=”MyUser”, password=”MyPassword”, use_unicode=true}

Tagged: