Unicode on Downlevel Windows (IUC 18) Unicode Across Windows Michael S. Kaplan Trigeminal Software, Inc. Cathy A. Wissink Microsoft

Unicode on Downlevel Windows (IUC 18) Why Unicode in your application? World wide EXE World wide EXE Decrease localization costs Decrease localization costs Superior multilingual support Superior multilingual support Better integration with the NT platform Better integration with the NT platform Support for new languages on Windows 2000, Windows XP, and beyond Support for new languages on Windows 2000, Windows XP, and beyond

Unicode on Downlevel Windows (IUC 18) Why not Unicode in your apps? (note that none of these are true any more; some were never true!) One code page at a time is enough, right? One code page at a time is enough, right? Every major language has at least one version of Windows which supports it Every major language has at least one version of Windows which supports it Visual Studio does not support Unicode Visual Studio does not support Unicode There is no support for Unicode when you are on Win95/98/ME There is no support for Unicode when you are on Win95/98/ME

Unicode on Downlevel Windows (IUC 18) The old workarounds Build two versions of the application Build two versions of the application Multiple code bases or build types to maintain Multiple code bases or build types to maintain Larger downloads/setups Larger downloads/setups Harder to isolate problems in code Harder to isolate problems in code Always do ANSI applications Always do ANSI applications No multilingual support No multilingual support Poorer performance on NT platforms Poorer performance on NT platforms No support for Unicode only languages No support for Unicode only languages Create your own layer (based on the 4/99 MSJ article from Microsoft) Create your own layer (based on the 4/99 MSJ article from Microsoft) A lot of work required! A lot of work required!

Unicode on Downlevel Windows (IUC 18) Bumps in the road Microsoft assumed that the migration to NT would happen more quickly and that ISVs would make that happen Microsoft assumed that the migration to NT would happen more quickly and that ISVs would make that happen Developers needed to support the consumer applications Developers needed to support the consumer applications Microsoft did not have a consistent Unicode story for all platforms (and needed one) Microsoft did not have a consistent Unicode story for all platforms (and needed one) Something had to give.... Something had to give....

Unicode on Downlevel Windows (IUC 18) The Microsoft ® Layer for Unicode on Windows ® 95/98/ME Systems

Unicode on Downlevel Windows (IUC 18) What is MSLU? The MICROSOFT Layer for Unicode on Windows 95/98/ME Systems The MICROSOFT Layer for Unicode on Windows 95/98/ME Systems file name: UnicoWS.dll file name: UnicoWS.dll Allows you to write a single Unicode application for all platforms Allows you to write a single Unicode application for all platforms

Unicode on Downlevel Windows (IUC 18) Design goals for MSLU Cannot slow down Unicode applications on WinNT/Win2K/WinXP Cannot slow down Unicode applications on WinNT/Win2K/WinXP When possible, provide a consistent, identical Unicode subset of the Win32 API across all platforms When possible, provide a consistent, identical Unicode subset of the Win32 API across all platforms Based on the Millennium API set Based on the Millennium API set Supports Win95/98 also Supports Win95/98 also Allow developers to take advantage of the new languages on Windows 2000 and Windows XP without abandoning older platforms Allow developers to take advantage of the new languages on Windows 2000 and Windows XP without abandoning older platforms

Unicode on Downlevel Windows (IUC 18) Other design goals As small as possible: As small as possible: Dll is ~160kb Dll is ~160kb Has over 440 API wrappers Has over 440 API wrappers Has more than 40 "stub" wrappers Has more than 40 "stub" wrappers Frugal with resources Frugal with resources No registration required No registration required No dependencies No dependencies No special file location requirement No special file location requirement No inherent "DLL hell" issues No inherent "DLL hell" issues Cannot be put in the system directory! Cannot be put in the system directory!

Unicode on Downlevel Windows (IUC 18) Performance No slowdown at all on WinNT/2K/XP No slowdown at all on WinNT/2K/XP No extra DLL load No extra DLL load Native APIs called directly Native APIs called directly Minimal slowdown on Win95/98/ME Minimal slowdown on Win95/98/ME Need time to convert strings to ANSI Need time to convert strings to ANSI For functions with out params, need to convert strings to Unicode For functions with out params, need to convert strings to Unicode

Unicode on Downlevel Windows (IUC 18) MSLU's custom loader No Virginia, you cannot do a true static link No Virginia, you cannot do a true static link Your application statically links to the loader Your application statically links to the loader The loader works in a similar way to the /DELAYLOAD functionality in VC++ The loader works in a similar way to the /DELAYLOAD functionality in VC++ Should work with other C++ compilers, too Should work with other C++ compilers, too Supports the ability to override any API but get all the benefits of the loader! Supports the ability to override any API but get all the benefits of the loader! No fear of failure: No fear of failure: DLL not present DLL not present Low memory scenarios Low memory scenarios

Unicode on Downlevel Windows (IUC 18) Agenda What is Unicode? What is Unicode? Why Unicode in your application? Why Unicode in your application? Introducing the MS Layer for Unicode Introducing the MS Layer for Unicode Integration of the layer Integration of the layer Covered APIs and functionality Covered APIs and functionality What the layer is not What the layer is not What about Visual Studio.Net? What about Visual Studio.Net? Where to get MSLU Where to get MSLU

Unicode on Downlevel Windows (IUC 18) Integration of the layer Just add UnicoWS.lib to the link list, and you are done! Just add UnicoWS.lib to the link list, and you are done! 1) Add it after static libraries like the CRT 1) Add it after static libraries like the CRT 2) Add it before all of the libraries that it uses (such as kernel32.lib and shell32.lib) 2) Add it before all of the libraries that it uses (such as kernel32.lib and shell32.lib) Making sense of the rules: Making sense of the rules: The linker resolves references Left to Right The linker resolves references Left to Right When it reaches the end, it wraps around When it reaches the end, it wraps around uafxcw.lib libcmt.lib unicows.lib kernel32.lib gdi32.lib user32.lib oleaut32.lib oledlg.lib shell32.lib comdlg32.lib advapi32.lib winspool.lib

Unicode on Downlevel Windows (IUC 18) How the DLL is loaded Setting an override: Setting an override: Add a function that you use to load the DLL Add a function that you use to load the DLL "Set the hook" by adding this line: "Set the hook" by adding this line: extern "C" HMODULE (__stdcall *_PfnLoadUnicows) (void) = &LoadUnicows; Good place to handle failure Good place to handle failure Default behavior: LoadLibrary! Default behavior: LoadLibrary! Fallback to the shared location if you must (not preferred at all) Fallback to the shared location if you must (not preferred at all) When all else fails, fail APIs gracefully! When all else fails, fail APIs gracefully!

Unicode on Downlevel Windows (IUC 18) Overriding individual APIs Add your function (using the same signature as the Win32 header files define) Add your function (using the same signature as the Win32 header files define) "Set the hook" with this line of code: "Set the hook" with this line of code: extern "C" FARPROC Unicows_ W = (FARPROC)& ; Overriding "stubbed out" APIs Overriding "stubbed out" APIs The new PSDK will have notes for every API -- including any special issues, when needed The new PSDK will have notes for every API -- including any special issues, when needed

Unicode on Downlevel Windows (IUC 18) Detecting when APIs are wrapped properly Current technologies do not support MSLU: Current technologies do not support MSLU: Dependency Walker (DEPENDS.EXE) Dependency Walker (DEPENDS.EXE) DUMPBIN.EXE /IMPORTS DUMPBIN.EXE /IMPORTS Using these methods anyway to determine what is linked and what is not Using these methods anyway to determine what is linked and what is not

Unicode on Downlevel Windows (IUC 18) Loading your own "unicows.dll" How LoadLibrary works for loaded DLLs How LoadLibrary works for loaded DLLs If DLL of the same name is already loaded, LoadLibrary uses that DLL If DLL of the same name is already loaded, LoadLibrary uses that DLL How to stay independent of others How to stay independent of others Rename unicows.dll Rename unicows.dll Use the loader override to load the DLL Use the loader override to load the DLL

Unicode on Downlevel Windows (IUC 18) Integration samples Taking a few samples from the PSDK Taking a few samples from the PSDK Showing off a customer integration or two Showing off a customer integration or two

Unicode on Downlevel Windows (IUC 18) Agenda What is Unicode? What is Unicode? Why Unicode in your application? Why Unicode in your application? Introducing the MS Layer for Unicode Introducing the MS Layer for Unicode Integration of the layer Integration of the layer Covered APIs and functionality Covered APIs and functionality What the layer is not What the layer is not What about Visual Studio.Net? What about Visual Studio.Net? Where to get MSLU Where to get MSLU

Unicode on Downlevel Windows (IUC 18) Covered APIs and functionality - 1 Wraps almost every "W" Win32 API Wraps almost every "W" Win32 API The few that are not wrapped are ones that already support Unicode, such as the IME APIs The few that are not wrapped are ones that already support Unicode, such as the IME APIs Stubs you can override are present for APIs not explicitly supported Stubs you can override are present for APIs not explicitly supported Wraps several non "W" APIs when needed for Unicode support Wraps several non "W" APIs when needed for Unicode support Unicode support (e.g., IsWindowUnicode) Unicode support (e.g., IsWindowUnicode) Consistent user messaging (e.g., CallWindowProcA) Consistent user messaging (e.g., CallWindowProcA) Common, known issues (e.g., ExtTextOutW) Common, known issues (e.g., ExtTextOutW)

Unicode on Downlevel Windows (IUC 18) Covered APIs and functionality - 2 User messaging support User messaging support Similar to NT's user messaging support for ANSI applications Similar to NT's user messaging support for ANSI applications Threadsafe Threadsafe Can handle multithreaded applications properly Can handle multithreaded applications properly

Unicode on Downlevel Windows (IUC 18) Covered APIs and functionality - 3 (Playing well with the other children) Works properly with the Unicode versions of MFC, ATL, and the CRT in both VS 6.0 and VS 7.0 Works properly with the Unicode versions of MFC, ATL, and the CRT in both VS 6.0 and VS 7.0 Interoperability with other MSLU clients Interoperability with other MSLU clients in process in process out of process out of process Interoperability with non-Unicode applications Interoperability with non-Unicode applications

Unicode on Downlevel Windows (IUC 18) Agenda What is Unicode? What is Unicode? Why Unicode in your application? Why Unicode in your application? Introducing the MS Layer for Unicode Introducing the MS Layer for Unicode Integration of the layer Integration of the layer Covered APIs and functionality Covered APIs and functionality What the layer is not What the layer is not What about Visual Studio.Net? What about Visual Studio.Net? Where to get MSLU Where to get MSLU

Unicode on Downlevel Windows (IUC 18) What the layer is not Not a rewrite of the Win9x platform Not a rewrite of the Win9x platform No NT-specific functionality added No NT-specific functionality added No "Unicode-only" language support No "Unicode-only" language support No supplementary character support No supplementary character support No new international support No new international support No new functionality added, at all No new functionality added, at all Not an "NT emulator" for Windows 95! Not an "NT emulator" for Windows 95! Not a layer over components that are still shipping new versions Not a layer over components that are still shipping new versions

Unicode on Downlevel Windows (IUC 18) Other components, other solutions RichEdit RichEdit 2.0: Unicode support 2.0: Unicode support 3.0: Bidirectional support 3.0: Bidirectional support 4.0: Supplementary character support 4.0: Supplementary character support Windows Common Controls (comctl32.dll) Windows Common Controls (comctl32.dll) 5.80: full Unicode support 5.80: full Unicode support Uniscribe/GDI+ Uniscribe/GDI+ MLang MLang Common Language Runtime (CLR) Common Language Runtime (CLR) System.Globalization namespace System.Globalization namespace System.Text namespace's encoding support System.Text namespace's encoding support

Unicode on Downlevel Windows (IUC 18) Agenda What is Unicode? What is Unicode? Why Unicode in your application? Why Unicode in your application? Introducing the MS Layer for Unicode Introducing the MS Layer for Unicode Integration of the layer Integration of the layer Covered APIs and functionality Covered APIs and functionality What the layer is not What the layer is not What about Visual Studio.Net? What about Visual Studio.Net? Where to get MSLU Where to get MSLU

Unicode on Downlevel Windows (IUC 18) What about Visual Studio.Net? VS.NET should include the newest Platform SDK; unicows.lib is in there! VS.NET should include the newest Platform SDK; unicows.lib is in there! CLR languages like VB.NET, C#, and MC++ have their own delayload solution CLR languages like VB.NET, C#, and MC++ have their own delayload solution PInvoke does not support calling different libraries on different platforms PInvoke does not support calling different libraries on different platforms Do not call MSLU on NT platforms! Do not call MSLU on NT platforms! You will have the same runtime issue with PInvoke in your MC++ code You will have the same runtime issue with PInvoke in your MC++ code

Unicode on Downlevel Windows (IUC 18) using System; using System.Text; using System.Runtime.InteropServices; public class SystemAPI { [DllImport("kernel32")] [DllImport("kernel32")] private static extern uint GetVersion(); private static extern uint GetVersion(); [DllImport("kernel32", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)] [DllImport("kernel32", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)] private static extern uint OsGetSystemDirectoryW(StringBuilder lpBuf, uint uSize); private static extern uint OsGetSystemDirectoryW(StringBuilder lpBuf, uint uSize); [DllImport("unicows.dll", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)] [DllImport("unicows.dll", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)] private static extern uint MsluGetSystemDirectoryW(StringBuilder lpBuf, uint uSize); private static extern uint MsluGetSystemDirectoryW(StringBuilder lpBuf, uint uSize); public static uint GetSystemDirectory(StringBuilder lpBuf, uint uSize) public static uint GetSystemDirectory(StringBuilder lpBuf, uint uSize) { if(GetVersion() < 0x ) if(GetVersion() < 0x ) return(OsGetSystemDirectoryW(lpBuf, uSize)); return(OsGetSystemDirectoryW(lpBuf, uSize)); else else return(MsluGetSystemDirectoryW(lpBuf, uSize)); return(MsluGetSystemDirectoryW(lpBuf, uSize)); }} class Class1 { static void Main(string[] args) static void Main(string[] args) { StringBuilder pSysDir = new StringBuilder(256); StringBuilder pSysDir = new StringBuilder(256); SystemAPI.GetSystemDirectory(pSysDir, 256); SystemAPI.GetSystemDirectory(pSysDir, 256); Console.WriteLine(pSysDir); Console.WriteLine(pSysDir); return; return; }} C#.NET syntax

Unicode on Downlevel Windows (IUC 18) Imports System Imports System.Text Imports System.Runtime.InteropServices Public Class SystemAPI Declare Function GetVersion Lib "kernel32" () As Integer Declare Unicode Function OsGetSystemDirectoryW Lib "kernel32" Alias "GetSystemDirectoryW" _ (ByVal lpBuf As StringBuilder, ByVal uSize As Integer) As Integer Declare Unicode Function MsluGetSystemDirectoryW Lib "unicows.dll" Alias "GetSystemDirectoryW" _ (ByVal lpBuf As StringBuilder, ByVal uSize As Integer) As Integer Public Function GetSystemDirectory(ByVal lpBuf As StringBuilder, ByVal uSize As Integer) As Integer If (GetVersion() > &H ) Then GetSystemDirectory = OsGetSystemDirectoryW(lpBuf, uSize) Else GetSystemDirectory = MsluGetSystemDirectoryW(lpBuf, uSize) End If End Function End Class Module Module1 Sub main() Dim stSysDir As StringBuilder = New StringBuilder(256) Dim sapi As New SystemAPI() sapi.GetSystemDirectory(stSysDir, 256) Console.WriteLine(stSysDir) End Sub End Module VB.NET syntax

Unicode on Downlevel Windows (IUC 18) Agenda What is Unicode? What is Unicode? Why Unicode in your application? Why Unicode in your application? Introducing the MS Layer for Unicode Introducing the MS Layer for Unicode Integration of the layer Integration of the layer Covered APIs and functionality Covered APIs and functionality What the layer is not What the layer is not What about Visual Studio.Net? What about Visual Studio.Net? Where to get MSLU Where to get MSLU

Unicode on Downlevel Windows (IUC 18) Where to get MSLU the.LIB ships in the Platform SDK! the.LIB ships in the Platform SDK! The DLL is a redistributable component: The DLL is a redistributable component: (it will be somewhere up there, not sure where yet!)

Unicode on Downlevel Windows (IUC 18) Questions?

Don't Forget Your Evals! The Microsoft ® Layer for Unicode on Windows ® 95/98/ME Systems