Log in

No account? Create an account

Previous Web | Next Web

Windows terminology uses "language" to refer to what a Linux bod probably thinks of as a "locale"; the language choice does more than just specify which dictionary to use, it is the basis for date and time display, text sorting order, the keyboards the user can choose, and it even stipulates which characters are considered letters, spaces, etc (although whether *that* one is true on Windows, I don't know; it does on Linux though).

Now, to be honest, I'm only opening with that paragraph so that I can justifiably refer to "locale," *not* "language," for the rest of this post.

So why am I interested in locales on Windows today?

Well, because I've come back to test-building a Windows 8.1 system in a VM before I bite the bullet and (***finally***) reinstall my main laptop. (*Quite* how its still functional is beyond me... it seems to have settled down at a certain level of broken!).

(Why a new test-build? Well, because (1) the previous one is *so* outdated that as to be irrelevant; most things I'll be manually installing have been updated, and quite a few to new major-version releases... (and 2) the world of Chocolatey has moved on, and most of the packages that *were* stuck in the mire of the Chocolatey people getting package-validation,-testing-and-approval up and running are now un-stuck, and point (1) applies to a great many of these, but also, unfortunately, the period of "stuckness" means that a *lot* of packages are now *badly* outdated relative to upstream, presumably because package moderation appeared to be stalled completely, so the once-maintainers moved on)

So, your might be asking again, why am I interested in locales on Windows today?

Well, because I have multiple keyboard layouts set up on my machine, and because on Windows 7, I can give each of them a different coloured icon... although I don't often *use*, for example, the UK layout, it *is* occasionally useful (ie, if mum needs to use my machine for something), and so that gets a red icon, so I know at a glance towards the systray if the reason my password isn't working is because I'm on the UK layout and so typing, eg, '£'-signs instead of '#' characters!

And because this functionality doesn't appear to exist in Windows 8.x.

NOTE: I'm talking here about the "Language Bar," which displays either as a small toolbar or as a small keyboard icon to the left of the systray, I am *NOT* talking about the new-for-Win8.x language indicator that is located next to the clock and displays the language selection in an abbreviated textual form (ie, it shows 'ENG' on an English-language system). The former is the "legacy"/"win32" version of the language selection interface and has been around for a few versions of Windows now, the latter is the "Metro"/"Modern" interface and doesn't (to my knowledge atleast) allow any kind of customisation.

I know it sounds like a little thing, but honestly, its the one I *keep* running into in Win8, and even just today while I've been installing stuff, this useless indicator icon has been sitting there mocking me.....

I *have* successfully worked around and/or gotten used to all the bits of Windows 7 that initially did my head in... the most significant being that the Start Menu is next-to-useless if you install more than about 2 things.... the non-cascading-ness preventing any "discoverability" (as Microsoft themselves termed it in relation to the wonderful new Windows 95 Start Menu!)... and discoverability is *really* important to me; I have a couple of *hundred* things listed to install on this Win8.x system, and when I need a to do something in a hurry, I can usually only remember that I've "got something that does that" somewhere on the machine! I need to see the names to remember what they do; there is no way I could *guess* have the names of these things!

And, thanks to Classic Shell, I got my nice cascading Start Menu back, and it works perfectly on Win8.x too!

The only other really issue I have with Win8.x is just how much MS waved the 'ugly stick' over it!! When Vista was launched, we actually had an out-of-the-box nice-looking Windows, after all those years (and the lovely little hack for XP that enable unsigned themes to be used); indeed, Vista being pretty was about the only thing that even remotely *tempted* me to move to it (then I found out about that XP hack).... moving from Win7 to Win8.x is *jarring* to say the least, and it hasn't grown on me at *all*....

Luckily, I'm not alone in *loathing* the ugly Win8.x duckling, and there *is* a Fairy Hackmother out there who waved their magic wand and created Aero Glass for Windows 8.x+, and suddenly we have a pretty Windows 8.x swan!!

So I thought I'd see if MS has *really* broken the indicator, or if they'd just broken the method for *configuring* it..!

A post on SuperUser.com suggested that perhaps it was the former, so I had a bit of a play.

Now, this whole keyboards, locales and users combination is a bit... well, lets just say that it doesn't exactly feel *elegant*! And MS appear to feel the same; their documentation is.... rather lightweight and vague...!

The upshot of the whole caboodle is that to change the indicator icon, we have to understand how three three sets of inter-referenced registry keys relate. Those keys are:

  • HKEY_CURRENT_USER\Keyboard Layout\Substitutes
  • HKEY_CURRENT_USER\Keyboard Layout\Preload
  • HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon

I should point out that, while I'm only interested in keyboard layouts, Windows is actually dealing with "input methods" and these aren't limited to just the traditional physical keyboard; they include the virtual/touch keyboard, handwriting input and (presumably) predictive text.

So if you catch me talking about keyboard layouts instead of IMEs, you know what I really mean! ;-)

Anyway, IMEs each have an identifier (I'm calling one of these an "IME ID") associated with them, and in the Substitute key, it is these IME IDs that we assign to items as values. The Substitutes key is used to assign a user's selected IME(s) to a user's selected locale(s). The item values in the Substitutes keys refer to the system's IMEs, the item names are the user's IMEs.

The Preload key specifies an ordered list of the user's *prefered* IMEs, with the item named "1" denoting the user's default IME. Preload item values refer to the user's IMEs (assigned in the Substitutes key).

And the LayoutIcon key relates layout IDs to icons. This exists (or rather, I know it atleast exists after you've assigned an alternate indicator icon) on Windows 7, but if you create it, it also works on Windows 8.x! :-) This key uses a different structure to the Substitutes and Preload keys, but also refers to system IMEs.



Right, so thats nice and simple then, yes?



Well.... no, unfortunately... you see, as well as having item values under one key referencing item names in another, there's a nice lump of symbolic meaning in the values themselves... Which brings us back to IME IDs.

My Windows 8.x VM, is using the United Kingdom locale, and has 3 IMEs assigned to that locale;

  • United Kingdom
  • Faerieality (UK English)
  • United States-International

(The Faerieality one is my homebrew layout (built with MS's Keyboard Layout Creator), which is based on the US layout and has all sorts of extra characters via various key combinations and sequences that make sense, quite possibly only to me!)

The Substitutes key contains these items (name -> value);

  • d0010809 -> 00020409
  • d0020809 -> a0000809

But my Windows 7 install (which has (almost) the same locale and IMEs (except it has the basic US IME rather than the International variant), has as *its* Substitutes key;

  • 00000809 -> 00000409
  • d0010809 -> 00000809
  • d0020809 -> a0000809

If I pull out the IME IDs, there are 4 in total;

  • 00020409 is US International
  • a0000809 is Fae UK
  • 00000809 is UK standard
  • 00000409 is US standard

But if we consider only the last 4 characters, we can see that;

  • 0409 only appears in the "US" IME IDs
  • 0809 only appears in the "UK" IME IDs

These values are also present in the Substitute item *names*, so it appears that the last four characters of the IME ID identify a locale.

It seems we can think of part of the IME ID as being a sort of reference against an list of locales, but what about the rest of the ID value?

In order to figure this out a little better, I added Welsh as a locale, then assigned the US Dvorak IME to it, which gave me another Substitute value;

  • d0010452 -> 00010409

Which suggests that 0452 is the Welsh locale, and reinforces the idea that </code>0409</code> is the US locale... but it also suggests something else... if we consider the second through fourth character of all the US IME IDs;

  • 000 is the basic default US
  • 001 is US Dvorak
  • 002 is US International

Then the obvious conclusion is that this *too* is a reference, this time to a list of IMEs *belonging* *to* a locale... and a reasonable design decision would be to use atleast 2 characters for this purpose (allowing 256 IMEs within some locale-specific grouping), or even all 3... although 4096 might be a lot of IMEs to group together...!

It *could*, of course, just be that the second (and maybe third) character is always 0, is reserved for future use or I just haven't run into it being used.

It doesn't really matter a great deal to us if its only the fourth character, or a group of three; this allows us to differentiate different IMEs from the same locale.

But what about the 'd' and the 'a' that appear in the Substitutes list?

So far as I can see, the first character of an IME ID is a "flag" character. I've only seen it have three values;

  • '0' which seems to mean "this is a completely normal, default IME ID/locale combination"
  • 'a' which I've only seen for my custom keyboard, so it probably means "this is an OEM IME that is not part of Windows" (which probably means it implies "contact the OEM for support")... (maybe 'a' stands for "addition"??)
  • 'd' which seems to get used in a Substitute item name for all but the first mapping relating to the locale. As I've been typing this, two possibilities have occurred to me;

    • it could denote something like "do not fall back to this IME" ie, when the prefered IME is removed
    • it could relate to support for codepages in the virtual-x86 MS-DOS environment, but I'm on 64-bit, so the DOSVM doesn't exist, and the GUI for 'select codepage' isn't present on my Win7/x64 install, so I can't try see if this is the case

Finally, it should be noted that, even though my Windows 8.x VM has (essentially) the same IMEs assigned to the same locale as on my Windows 7 system, it has one fewer Substitute item... And the way this appears to work is that;

  • When a user adds a new locale, the system's default IME for that locale is automatically assigned as the user's default IME for their new locale. For the UK locale and its default IME, this results in a Substitutes entry for 00000809 -> 00000809, but because this is a no-brainer, Windows doesn't bother with the Registry entry. It *does*, however, add the IME to the Preload list.

  • On my machine, initially I replaced the UK IME with the US one, and it appears that if you assign any other IME to the locale and remove the default IME all in one go (ie, you open the Text Services and Input Languages panel, add a keyboard and remove the other without clicking "Apply" or "OK"), then;
    • The removal is done first; Windows takes the relevant IME out of the Preload list and (presumably) removes it from Substitutes if there is an entry there for it.
    • The system then adds the new IME, mapping the user's default IME for the locale to an IME that *isn't* the system's default IME for the locale.

      Obviously Windows needs to remember this mapping beyond the current login session, so it has to create an item in the Substitutes key to record the mapping (in my case, the user's default UK IME, 00000809 maps to the system's standard US IME 00000409).

    • Finally it adds the new user IME to the Preload list.
  • If the user subsequently adds back the IME that was removed (the system's default IME for the locale), Windows has a somewhat peculiar scenario to handle; the entry in Substitutes for the user's default IME for the locale (in my case, 00000809 -> 00000409) has already *got* am IME mapped to it (the standard US IME in my case), so Windows now has to add an entry to Substitutes for the system's default IME for the locale as that user's Nth additional IME for that locale (giving a mapping of d0010809 -> 00000809 on my Win7 machine).



Right, are we confused yet? I know I was by now!! (Especially after looking at my Windows 7 config!)



So after all that, how do we change the icon colour?

Well, now we have a handle on all those IDs, we can put it together...

On Windows 8.x, we navigate to HKEY_CURRENT_USER\Software\Microsoft\CTF and if the subkey HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon doesn't exist, we create it.

Next, we create a subkey for our locale, so for the UK locale (the locale id being, as we've worked out, 0809), we create HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809

The final subkey(s) to create, are for the IME's the user has assigned to the locale;

  • First up, we need to consider the default IME for the locale. This is the "missing" entry that makes my Windows 8.x VM's Substitute list shorter than on my Windows 7 system.

    • We need to determine this IME ID, so we have to go through the Preload list and cross-reference the item values with the item values (not the item names) on the Substitutes list. Unless you've messed around with your IMEs like me, you'll come across an item value from the Preload list that doesn't correspond with any item value in the Substitutes list; this item value is the missing IME ID for the locale.
    • Now we know the IME ID, we can create the LayoutIcon\locale subkey for it.

      On my Win8.x VM, the IME ID 00000809 is not a value in my Substitutes list, so I create HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809\00000809

    • Now we need to add the other IMEs for the locale. This is much less faff; we just create a subkey for each of the other item *values* in the Substitutes list whose item *name* refers to the locale, so on my system, for the United Kingdom locale, I
      • Skip the entry d0010452 -> 00020409, as it references the Welsh locale (0452)
      • Create HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809\00020409 and HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809\a0000809 as the mappings d0010809 -> 00020409 and d0020809 -> a0000809 both refer to locale 0809

Now (at long last!) we can assign some icons!

Starting with the UK layout (00000809), under HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809\00000809, I create two items:

IconFile String REG_SZ C:\Windows\system32\msctf.dll
IconIndex 32-bit DWORD REG_DWORD 0x17 (23)

And that's it! We've assigned an icon to one of the IMEs.

To check that this has worked (its really easy to mis-count the number of zeroes!), it is necessary to log off and log back on again. (Bear with me just this once; there is a way to do update the indicator icons while logged in, but this is easier for a one-off. I promise I'll tel you the Better Way in a minute).

Having logged back in, I can see in the IME picker that I have a little red keyboard icon for the UK layout. If you didn't change your default keyboard's icon, you'll have to click the indicator icon to see the menu, where the icon is also changed.

So now for my other two IMEs

  • HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809\00000809
    • IconFile -> C:\Windows\system32\msctf.dll
    • IconIndex -> 0x12 (18)
  • HKEY_CURRENT_USER\Software\Microsoft\CTF\LayoutIcon\0809\00000809
    • IconFile -> C:\Windows\system32\msctf.dll
    • IconIndex -> 0x15 (21)

Now, the most annoying thing about this whole rigmarole (once you've gotten used to the different IDs and keys and things, anyway) is getting Windows to update the icons that are actually being used! Logging-off-and-on-again gets very tedious very quickly if you're just playing with icon choices! So I had a bit more of a think, and I had an idea; what would happen if we hid the IME selector then un-hid it? We *can* still do *that* using the GUI; we just have to know where to look!! So;

  • On the Start Screen, type

            change input

    You should see "Change input methods" in the list of search results; click it.

  • In the window that opens, in the left-hand column is a link for "Advanced settings," click it.
  • Now go down to the section "Switching input methods;" on the right is a link for "Options;" click it.
  • I *know* its a lot of clicking; unforunately I can't help how MS have buried this...
  • Anyway, a new window ("Text Services and Input Languages) should appear (looking rather a lot like an old fashioned control panel applet, how quaint!), with a "Language Bar" tab that has a "Language Bar" section containing some radio buttons ("Floating On Desktop," "Docked in the Taskbar," and "Hidden."
  • Note which of the radio buttons is currently pressed, then press the "Hidden" button and click Apply. Your IME selection doo-dah should have gone.
  • Now press the originally-pressed radio button and Apply again, and the doo-dah should come back, but with the icon change(s) you made in the Registry applied! Happy days! ;-)

Hopefully you can see why I said it was easier for a one-off change to just log out-and-in; this approach is *really* useful when you're experimenting, but a real P-I-T-A for a quick tweak!

Now, I have to admit that I cheated a little bit with the icons I used for testing this, and just used the values from my Windows 7 system (in my defence, I was only seeing if I could get some different icons to show at that point).

Icon Palette

But, to make up for my cheating, here is the Windows 7 icon selection palette for msctf.dll (which is where these keyboard icons live). If you count down the columns, you will see that if we look up the icons I used, the IconIndex value is easily derived;

  • The red icon is 24th in the palette, but computers start counting at 0; in computer-think, IconIndex = 23.
  • The purple icon is 19th, so its IconIndex = 18
  • The orange icon is 22nd, so its IconIndex = 21

Oh, BTW, if you don't create a key for a particular IME's ID, it will receive the default icon (as per the US Dvorak IME in the image below).

In the end, the icon is just a normal Windows icon, so if you can determine the IconIndex to use, then you can specify any icon from any icon-containing file you like, for example, using;

  • IconFile -> C:\Windows\System32\user32.dll
  • IconIndex -> 0x6 (6)

would set the UAC Shield as an IME's icon!

Multicoloured Keyboard Icons