A reader of my blog Evgeni Sergeev (well he might not be a reader – it just makes me feel more important) posted a very detailed set of questions about KVS that were excellent and justified a post in their own right. Evgeni, I hope that I have covered the answers that you need – if you require anything more, please just let me know (My answers are in blue and Evgeni questions are in Black)
I have been researching mail archiving products for our company and I find that the information I need is not easy to find. I am primarily interested in two things: the cost, and what the product actually is. However the information on official product websites, as a rule, skips these and goes straight into the benefits or importance: “Email has become a corporate asset…”.
Andy, if you are aware of a good technical description of Enterprise Vault, would you care to post a link. It’s the small things I am interested in. Definitely an email archiving product is expected to keep a copy of all emails, but it is the small details of accessing the archive which make all the difference for people using them.
For a good overview which is non company affiliated the check out the following link: http://en.wikipedia.org/wiki/Enterprise_Vault
The main difference between products is in how deeply they look into Exchange. Some just operate on the boundary and record each email as it moves past. They are unaware of mail being moved into subfolders within a mailbox, they do not take email out of mailboxes to archive and they do not provide links back to enable an email to be restored out of an archive from Outlook. Retrieval is usually through a web interface and is search-based: you could specify a recipient, time frame, keywords and you would get a list of archived emails matching the criteria. More advanced, and generally more expensive, products see into the structure within Exchange, and these products provide the effect of the infinite mailbox by archiving emails based on policies, usually marking an email as being archived using an modified icon and providing retrieval functionality right in Outlook. Enterprise Vault is one of the latter.
From what I have experienced in Enterprise Vault it is exceptionally detailed in how it looks at Exchange. It will archive mail from within a mailbox which matches the configured archival policy rules no matter where in a customer’s mailbox the item is located – for example; if you have a archival policy which is set to archive all items which are older than 3 months – it doesn’t matter if a mail item matching that criteria is six folders deep within the mailbox – it will get archived.
How exactly does EV integrate with Outlook? Can search of the archives be performed from Outlook?
It is the integration with Outlook and OWA that particularly attracted me to enterprise vault. When an item of mail is placed within the vault the default mail item icon changes to indicate that it is now stored within Enterprise Vault. However in all other ways the functionality within Outlook and OWA remains the same.
Essentially in the background what Enterprise Vault does is it copies the item of archived mail into the vault (but leaves the original mail item in Exchange – this called the “Safety Copy”) and then waits for the KVS backup to occur – when the backup has occurred the source item in Exchange is deleted and replaced with a small pointer (which has the new icon) and a x number character preview of the vaulted item (this is configurable – I set mine to 500 chars).
Users have the ability to manually archive and search for items via a tool bar extension which is installed into both Outlook and OWA – for example a manual archive action is two clicks!.
Searching for items within the Vault is excellent. The search backend for Enterprise Vault is currently based around Alter-Vista and users have the option of either using the integrated Outlook / OWA search functionality – or – using the dedicated website that is installed with KVS.
For example, if a user wishes to search within Outlook or OWA they simply click on the search Icon for KVS (which is installed as part of a plug-in) and from within Outlook the Window changes to a screen where they can use a simple search or provide more advanced criteria descriptions of the mail item they wish to retrieve.
How does its support of various storage mechanisms work? In particular, with tape? Does it tell you when it wants to write to the tape?
There are many ways in which you can configure your storage of archived items in KVS. We for example use IBM TSM (along with an IBM FASTt 700 and an IBM DS8100) – basically 3 tier storage.
The idea is that our short term vaulted items (3-8 months) are placed on tier 1 storage (DS8100) to maximise retrieval times, medium term items (8 months to 2 years) are placed on FASTt and long term vault is placed to tape (TSM using an IBM 3583 Ultrium).
In terms of does it tell you where the item is, no its the other way around, you configure you retention policy to move items within the vaults stores which are located on the different storage devices.
There is of course also a tape backup every night, which apart from the obvious resets the archive bit on the items (which is the indicator to remove the safety copy from Exchange).
Some products use a database, while others store files on the file system. Are there any advantages in using a database for this application, especially where large attachments are concerned? As far as I know databases are very good for dealing with a large amount of small items of data, but here we have a large amount of large items.
Right, there is common misconception about KVS that it uses a database to vault store items – it doesn’t.
The vaulting is carried out using a file system based structure – the database that KVS uses is purely for configuration information of the vault site.
The file system is broken down into folders which represent Days, weeks, months, years, hours, and minutes.
Within these folders are encrypted files that represent vaulted items for that period of time. The vault maintains single instance storage (which at this time I am not sure how) and uses a proprietary compression algorithm therefore I have not seen a massive problem with large attachments (but then again within my Exchange configuration I limit the size of attachments to around 10 MB anyway).
To guarantee tamper-proof storage, does Enterprise Vault require particular hardware? How can it give the guarantee otherwise?
KVS does not require an specialist hardware (beyond a half decent server) – security is handle via encryption on the vault items and NTFS permissions.
What does the SDK/API allow one to do?
There is a wide ranging SDK / API reference supplied with the documentation of the product – at this time I have not explored this in any depth, beyond how to script the addition of users to KVS.
Are permissions for archive access managed through Active Directory or through an internal user management system which may be synchronised with AD?
Ok, this is a little bit of both. In terms of access to archived vaults if there is a process that runs three times (or more if you wish) which synchronises the permissions on a vault with AD. You can also configure addition permissions on a vault store from within KVS that do not exist in AD from within the KVS management console.
I believe the official recommendation is to keep at most 5000 items in an Outlook mailbox – do archived items count as part of this figure, or is this recommendation based on average message size and aimed at keeping the mailbox within reasonable volume range?
Yes, archived items do contribute to this overall size (as a small pointer is placed in the mailbox when the item is archived). I have over 8000 items in my mailbox at the moment (most of which are archived items) and things are working well.
From Outlook, if an email marked as archived has been deleted, what methods remain to retrieve the email?
This can be accomplished via the vault web interface – essentially you access a URL on the Enterprise vault site and provide your credentials and you will be taken to a search interface where you can review the messages, or restore them to the mailbox.
Is there much other functionality except archiving email?
Yes, Journaling (very useful), file system archival, and PST migration.
The IBM product, CommonStore: why do you think it’s not so good? I admit I haven’t found a great deal of information about it online.
Mainly because of the amount of other IBM products that you have to purchase in order to use it (websphere for example) – this elevated the cost of CommonStore to outside our price range. Couple this with the fact that the IBM tender could not explain how it integrated into Outlook or OWA to a satisfactory level, nor explain how it exactly integrated with the Exchange store.
Thanks if you could answer some of the questions.
Evgeni
No – thank you Evgeni – if anything that I can provide helps someone then great. Bearing in mind that although I use and bought KVS I am aware that there are other solutions out there that may suit someone else’s business better. For me KVS matched our budget and fited our business requirement.
Cheers.
Andy
