The main means of accessing documents in document management systems is via folders. This makes sense because it’s what people are used to. Before they get a document management system they normally arrange their shared documents in a shared location, organized via folders. They’re intuitive, hierarchical and familiar; and thus people tend to look for document systems which are focused around folders as well. This makes the migration to the new system easier as well.
THIS IS A MISTAKE!
“Why are you switching to a document management system at all?”
If your file share is perfect for you why spend a lot of money on a system which is just going to replicate it? You’re trading a cheap, convenient, and reliable system for one that is much dearer, requires retraining and is all too often less reliable. If you need to share your documents in a folder structure across the Internet, use Dropbox, it’s a fantastic service and it’s very reasonable. If your company just needs a glorified (and expensive) folder share, I don’t want to con you out of your money and add zero value. With Signate, we want to add real value, and we don’t believe that is done by a web-based share.
Case Study in failure
At a large financial company where I consult on some electronic accounting issues, they had a massive shared drive where all documents were kept. They spent millions of rands implementing a company-wide Content Management System (CMS); money thrown at the software, hardware and numerous consultants involved. Most CMS systems make it easy to arrange your documents in folders, and so they reorganized the layout, designed it better and set it all up. All new projects were to use the new system it was decreed. They did, but only as a file storage medium. The more advanced features like wikis, monitoring, calendar, workflows and so on went almost unused. It was also decided that the existing file share was too large to migrate, so it coexists side by side with the CMS, and there is often confusion about where to find documents, and which version is the “current” one: the file share version or the CMS one. Net result: significant capital and operational spend, massively increased storage requirements (due to duplication), confusion, and little or no improvement. They have also been unable to get the CMS search working, which is a critical failure in my opinion (as we will see later).
Is this the fault of the CMS? Not at all. This particular CMS is a very powerful tool in the right hands. It’s configurability allows it to really shine when well-implemented. Unfortunately it is all too rarely implemented well, and this usually requires hordes of very expensive consultants.
Failure of Design
The underlying problem is that the CMS, along with all too many document management systems cater to people’s first instincts: the desire to keep things the same as they were. Let’s cast our minds back to 1998. The Web was a growing phenomenon, and the most popular portal was Yahoo!. Their web site was built around a directory, a folder structure exactly like that in your shared drive, except it consisted of links to web pages. People submitted their pages to Yahoo! and it would be placed in a category.
They had search, of a sort, but the focus was clearly on the directory structure; that was how you ensured that you found what you were looking for. You would browse through folders, hunting for the right category. Sometimes the categories were arranged somewhat haphazardly, so it could take a while to find the right one. However the task of maintaining this directory grew larger and larger, and the directory fell further and further behind.
I remember that my primary source of new pages started to be from friends’ emails rather than finding them in directories. All the while Google was making search their primary focus. We all know how that story played out; search became the dominant means of finding pages in the web. Why does search trump directories? For a few simple reasons:
- A directory imposes the directory organisers priorities on the consumer – If the organiser arranges things in a way that the consumer finds counter-intuitive it can be difficult or impossible for the consumer to find content that is present.
- A directory requires constant work to ensure relevance – Entries (or documents) can become stale or corrupted, newer locations may become popular causing duplication with work occurring in both locations.
- Search puts the consumers priorities first – You type what you’re looking for and the search engine finds it, what could be simpler than that? There is no organiser other than the content, so you don’t have to put up with odd filing hierarchies.
- Search ensures relevant content is found immediately – No hunting through folders and opening documents; the best matching results are returned first.
- Search allows for powerful search terms – You can use advanced features such as ranges for dates and numbers, exact matching, wildcards and so on very quickly and easily.
- Directories are categorised by perception, search by reality – When we decide to place a document under the “Technical Specifications” folder we’re doing so based upon our idea of what that document contains. Normally this would be done by the content author; so they’re generally pretty accurate, but there might be a better location or the categoriser may be mistaken in their assessment. Search categorises documents based on their content.
- Directories are static – Related to the above, documents change, and your system must cater for that. A directory structure tends not to change, even when it should. People are used to accessing a particular document in a particular place, and if you move the document they won’t find it at all. You’ll go from 100% accuracy to 0% in one swift go. Whereas with a search system, the document will move up and down in the search results for a particular set of search terms as it’s content changes.
- Directories take effort – You need policies and procedures and people who monitor them and control them. All of this is not productive work.
The Road Ahead
The future of document management lies in search. In my many years in the Document Management field, across industries as diverse as logistics, healthcare, insurance, financial, travel and many others I have seen finding documents again and again become the pain point for project after project. This is why we created Signate: as a response to the appalling inefficiencies of products spanning from the cheapest of the cheap to high-end enterprise servers. Signate puts search front and center, and whilst we are ahead of the game right now, I am under no illusions as to how long that advantage will last. Search is such a compelling feature that all document management systems will have to become search-centred or they’ll fail.
The question you have to ask yourself is where you need your company to be? Do you want an easy transition to document management but very little added value, or are you willing to learn a new way of finding your documents? It isn’t even that new if you’re used to searching the Web.
Search Quality
So now that I’ve made my case for search over folders in document management systems, let’s look at the quality of that search. Have a look at the screenshot to the left. It’s from another document management systems search screen and exemplifies pretty much everything I dislike about search in the document management space.
Each field you can search on is listed, each with it’s own box. Worse yet there are drop downs for “Contains”, “Exact Match” and so on. Whilst I hate the dropdowns for the date fields, at least they actually have a date range search as opposed to forcing you to pick one date at a time. But now, what if I knew that the document was created after a certain date, and the revision I was looking for was before another one. How would I enter that? Would I be able to leave the To date range empty for “creation date”, and the from date range empty for “revision date”. Possibly, it’s not clear. How would I search for where the author is either “Ray Bradbury” OR “Orson Scott Card”? I know it’s one, but am not sure which one. I’d probably have to do two searches.
Now consider the search screen on the right. This strangely enough is much clearer as to what you need to do, which is counter-intuitive if you think about it. You’d expect that the screen which spells everything out explicitly would be the easiest and most compelling, but it’s just not. An empty search box invites, a complex search screen repels. How would you search for both authors as above. Well, I’d type: “Ray Bradbury” OR “Orson Scott Card”
The normal reason document management companies use search forms like the left hand one is because their search form is a thin wrapper over their underlying database. This limits them, as databases are designed to be very specific, and cannot search across fields easily. Not only that, if they don’t get the search form exactly right, it is possible for a user to run updates and malicious scripts on the database. The database that’s storing all your document data. With Signate we use a completely separate search engine, which not only is designed to search and search well, but also cannot affect your underlying data. Oh yeah, and it’s fast. Blazingly fast. Much faster than a complex query run against a database. Plus it can easily scale up to billions of documents, which database-driven searches struggle with.
Conclusion
If you need a document management system, please, please, please choose one that puts search in the forefront. Ensure that before you buy you really kick the tires on the search system; that it’s quick and easy to use. It should not take your staff longer to find an internal document than to find a web page via Google. If it does, then you have a suboptimal system. A swift and powerful
document management system should pay you dividends across the board. You should have happier and more productive staff, faster processes, easy findability for your documents, and vastly improved turnaround times.
Use our Document Costs Calculator to work out the amount you’re probably wasting right now on your document costs, and thus the amount you can save every year. That financial company I discussed earlier? Our calculator shows that well designed and run document management system should be saving them between 12 and 70 million rands a year, and save between 180,000 and 490,000 staff hours annually. These figures are based on research, calculations and figures from Gartner, Cap Ventures and the Arbeidsgemeinschaft für wirtschaftliche Verwaltung.
Plug in your company’s figures and see the impact a good document management system could be having on your bottom line and client satisfaction. Plus it’s good the environment too.
Sean Hederman is Director of Palantir (Pty) Ltd, and Software Architect for the Signate Document Management System. He also writes the popular programming blog Codingsanity.
