Microsoft Outlook AutoComplete research
The task was to investigate the behaviour of AutoComplete function in Microsoft Outlook software, specifically what happens during creation of a new profile, or copying an existing Outlook profile.
AutoComplete (or Auto-Complete) is a feature that suggests emails and names when you type them in To, Cc, and Bcc fields on new message in Outlook.
We need to know the location of auto-complete files and properties related to this feature, and investigate options of making a new Outlook profile to use auto-complete settings from an already existing profile.
For example, when a new MAPI / Outlook profile is created, how exactly do we make it reuse auto-complete from the existing profile, so that the behaviour of the feature is exactly the same?
A very useful reference regarding AutoComplete in Microsoft Outlook is this Slipstick article
. It looks like AutoComplete-related things are very much Outlook version-specific. The article mentions:
- Stream_Autocomplete_0_[GUID].dat files in Outlook 2010 and 2013.
- .nk2 files in Outlook 2003 and 2007.
- .nick files in prior versions.
To examine MAPI properties, we used MFCMAPI tool
Outlook Testing Environment
Our testing environment was as follows:
- OS: Windows 7 Ultimate 32 bit.
- Outlook 2013 32 bit from Office 365 Business Premium download.
- A spartan Outlook profile with a connection to IMAP and SMTP servers (local, Gmail, or Office 365 - does not matter). IMAP port 993 with SSL, SMTP port 587 with STARTTLS. Outgoing server (SMTP) requires authentication and uses the same login and password as incoming mail server.
IMAP and SMTP mail server ports
Configuring Outlook Profiles
Outlook / MAPI profiles are configured using the Mail applet in system control panel, which gets installed with Outlook.
Control Panel with Mail applet
Location of AutoComplete Data
Experiments show that the AutoComplete data is stored not in one location, but at least in two:
- Outlook data file (in this case it is an .ost file).
- Cache file named Stream_Autocomplete_0_[GUID].dat in %localappdata%\Microsoft\Outlook\RoamCache
Both sets of data do not exist initially. They appear at different time points.
- After you send a first email, .ost obtains initial small set of data.
- When .ost has the data, restarting Outlook also creates a corresponding cache file.
Let's examine the process in more detail.
New Outlook Profile
Let's create a new profile with the Mail applet without starting Outlook and see the MAPI properties of the associated store. Specifically, we are trying to determine whether IPM.Configuration.Autocomplete hidden message, as per kb2679568
exists. No, not at this time. MFCMAPI tool shows us that the Associated Contents Table
for the Inbox folder is empty.
Starting and closing Outlook will create a few hidden messages in there, but not IPM.Configuration.Autocomplete.
Now let's send our first message without closing Outlook. After we send the message, the AutoComplete feature starts working, meaning that it suggests this one email when we try to send another message. But there is still no IPM.Configuration.Autocomplete message. Closing Outlook does create it, and we can examine its properties.
PR_ROAMING_BINARYASTREAM property of type PT_BINARY in the message appears to hold AutoComplete raw data.
Note that at this time (Outlook closed), the data exists in .ost but there is no cache file.
Opening Outlook again (and doing nothing) now does create a new file Stream_Autocomplete_0_[GUID].dat with data for our sender.
The file name contains a GUID, where does it come from? If we look at the system registry under HKCU\Software\Microsoft\Office\15.0\Outlook\Perf\RoamingStreamsCache - we will find the GUIDs as sub-keys there.
Outlook roaming streams cache registry keys
The key with our GUID contains MsgEID value of REG_BINARY type, which matches PR_ENTRYID MAPI property of the IPM.Configuration.Autocomplete hidden message.
Location of AutoComplete Cache Files
Outlook 2013 stores reaming streams cache files in %localappdata%\Microsoft\Outlook\RoamCache
We can type it into Windows Explorer either as is, or a fully resolved path, which apparently depends on drive and user name.
Notice that deleting the IPM.Configuration.Autocomplete hidden message with MFCMAPI tool (when it works and does not crash when doing so) and redoing things with Outlook gets as a filename with a different GUID. A screenshot below shows 3 files with a different GUID, all used with 1 Outlook profile, after manual delete of the hidden message, restarting Outlook, resending a test message, and restarting Outlook again.
Outlook AutoComplete cache files
Retaining AutoComplete Data
What are our options if we wanted to retain existing user AutoComplete data in another new
Upon creating a new profile, AutoComplete feature has nothing to suggest, and there is no cache file.
After a user sends their first message and restarts Outlooks, then the IPM.Configuration.Autocomplete message appears in the associated contents table for Inbox in the message store. Restarting Oulook again generates a new corresponding cache file.
At this point we can:
- Close Outlook and replace the file content with the one from the old profile.
- Starting Outlook. At this point Outlook has old AutoComplete data. Meaning that it suggests emails from the old profile, and excludes the new email address just used.
What if we wanted to do everything programmatically without restarting Outlook 2 times? In theory, we can try to copy the hidden message from the old profile to new. This has not been tested, though, but seems like a good approach. If copying cannot work, we may also try creating a new hidden message there (with Outlook closed), and then setting its PR_ROAMING_BINARYASTREAM property to match the old profile.
Copied Outlook Profile
This appears to work a bit better, but has an unexpected side effect, which manifests itself like so:
- Start Outlook in a copied profile, AutoComplete feature already suggests as in the old profile.
- An existing AutoComplete cache file is now shared with both profiles. Meaning that changing things in one profile affects AutoComplete behaviour in another. Tested by sending email to another address in a copied profile, the address started to appear in AutoComplete of the original profile.
It happens because a profile copy reuses the same .ost file, therefore, the entry id for the hidden message is the same, and many things are, therefore, shared.
- Location of AutoComplete data depends on Outlook version. See the Slipstick article for more details.
- In Outlook 2013, the data is stored in IMP.Configuration.Autocomplete hidden message in Inbox, and duplicated in a cache file, with mapping between the two stored in registry.
- Although a new profile does not have any AutoComplete data, we can still get it by injecting already existing data into either a cache file or the hidden message (this last part was not tested, but it seems reasonable to expect that further experimentation may prove it possible, for example by removing the cache file and recycling the PR_ROAMING_BINARYASTREAM property from already existing profile).
- We can get AutoComplete working with both a copied profile and a new profile with some data manipulation.
You can leave a comment
on this project, or post a new project