Brave New World of Capture

clip_image002Hopefully the salesmen who have tried to sell me Capture solutions over the years aren’t reading this blog; if any are, I’m sorry. I have been telling those guys for years that:

We don’t use forms and we really don’t have a need for hardware or software to scan into SharePoint.”

I lied. OK, I really didn’t lie, I just couldn’t properly imagine the truth – we do have a need for a scan-to-SharePoint. Well, actually we don’t but we will for a while.

Sorry for the confusing lead-up to this post, but it has been very confusing for us as well. Despite the fact that we are an insurance company, we really don’t use hardly any forms in our business process. What we do need a scanning solution for is backfilling some important document libraries. We could simply take the approach of having a network scanner, or even desktop scanners  (since we have so few insureds) but I don’t think that will work. What makes me say that? Well we’ve had desktop scanners and high-speed Multi-function copies on our network for the entire time that we’ve had SharePoint but very little content has been added to SharePoint via those devices. The reason for that result is something that Marc Anderson mentioned recently and Steve Weissman has been saying forever – simply having the technology scanners or SharePoint) in place is not enough.

If technology is not the answer, then why am I excited about the arrival of these MFC’s and the configuration of the scan-to-SharePoint options next week? The answer is technology is only part of the answer this time. This time, we are going to attempt to address the business process side of the equation and the human side of the business process.

The copiers that we just replaced were “capable” of scanning to SharePoint, but only if you gave the copier Full Control the SharePoint sites that you wanted to scan to. That meant that people could scan documents into libraries that they couldn’t actually reach from their desktop. We trust our employees, but that’s dumb. The dumbest part of that would occur when someone accidentally scanned a document to the wrong library – then they couldn’t even delete it. In addition, the copiers needed to be on the Internet so the vendor could access the copier for maintenance and meter readings. Call me silly, but that sounds like a potential security risk. With that feature never activated, we were left scanning to a network drive and getting documents in the form of “Scan_20080616121254.PDF” – very helpful. Of course, there were better capture options around when we bought those MFC’s but we didn’t need them; remember? We also couldn’t afford them – we still can’t.

What we can afford is the following combination of hardware, software, workflows, training and administrivia:

The MFC’s communicate with server-based software that can map user rights and privileges back to the menu system of the copier.

The MFC’s are capable of rendering the scanned documents as PDF or PDF/A into the desired library and they are capable of producing metadata from barcodes or other scanned artifacts.

We are capable of creating SharePoint Designer workflows to process the scanned documents upon arrival in the library. In some cases, we may have the scanner deliver documents to a staging library so that the workflow can perform other operations first. For example: our policies are what are known as “continuous form” policies, meaning that we renew by endorsement. In non-insurance speak, that means that each year, we have to add 10-12 pages to an existing document / PDF. These MFC’s in conjunction with software from HarePoint and Muhimbi can stitch the incoming PDFs onto the back or front of an existing PDF.

Once we can demonstrate these capabilities, we can ask department managers to accept the assignment, on behalf of their staff, to make backfilling a requirement during the lease-term of these copiers. If our Policy Management software can be made to print to PDF in those libraries, we can eliminate the need for the scan-to-SharePoint option with the next generation MFC’s (see, I didn’t lie).

By the way, just to prove that “we get it,” rather than make people login at the copier using an on-screen keyboard, we paid extra for a reader that can accept the proximity badges we use for our security system as input – how cool is that?

The copiers just arrived. The software has not been configured. The workflows haven’t been written and the people haven’t been trained. But we have engaged a training partner to help us get our coworkers up-to-speed and we are starting with “Content Management Fundamentals” – We are going to start at the beginning, cover all the bases and build solutions that will help us treat “Backfilling” like the important business process that it is in real life.

Is This Right

clip_image002Last week, I was talking about accuracy. This week, I learned that there is something more important than accuracy – confidence. We have been working on the design of a SharePoint solution that is both simple and complicated at the same time. Of course, we are concerned about accuracy, but the people using our system will have to feel confident that they have selected the information that they need. This isn’t a challenging SharePoint problem; this is a design issue, a usability issue and ultimately an adoption issue.

Our project involves a contact list. That’s simple, but it’s really about 10 contact lists, and since they need to be exposed on both an internal and an Internet-facing server, it’s really 20 contact lists. The “master” list is a group of people who serve on a variety of committees, and that’s the only list we want to maintain. The people come and go and the committee membership changes from year-to-year. We need to know who is on each committee today, we need to know who was on each committee 2, 3, 5 and perhaps 10 years ago, and we need to know this from multiple angles. For example, I might simply want to be able to contact the members of Committee A. On the other hand, if I’m about to meet one of the people on the list, I might want to know every committee he or she serves on. In yet another scenario, if a person is retiring, I might want to know every committee the person ever served on. We’ve cracked the list design issues, and with the help of HarePoint Workflow Extensions, we’ve cracked how to keep the lists in sync on the Internet-facing server. OK, so what is this confidence issue?

Let’s think about those scenarios again. In the first one, I want to see the names of the people who are currently serving on Committee A. Obviously, I could simply filter the list on a few columns to distill the contents down to right group. That would work, but while most people know how to do this, they don’t want to have to do it every time; that’s what views are for. Views, now this is where we start to get into trouble with usability.

Views are a fantastic feature within SharePoint, but neither lists nor views scale very well. That’s not to say that you can’t put a ton of items in a list, or create a bunch of views, but sooner than later the results are untrustworthy. Once you get a couple of pages of items in a list, you have to resort to sorting or filtering to make it useful. Similarly, once you have 15 or 20 views on a list, the selection is equally unmanageable; in fact, a large selection of views might even be worse than a large collection of items. When I am setting sort and filter parameters, I know what I am doing; the trouble is that I have to be able to imagine the results as I proceed. When I choose a view, I am relying on someone’s ability to name a list intelligently. If I make the user figure out how to get the information they want, I put the burden of accuracy on them and it becomes a confidence issue.

There’s a very fine line between wanting my users to understand SharePoint and forcing them into a situation where they are uncomfortable. When they get to that point, they are going to ask someone else (maybe me) to get them the information they need, and at that point SharePoint and I have failed.

We have to find a way to tame a large list that can be rendered in a large number of ways. I can’t just substitute a bunch of input fields for a search or data view web part, that’s really no different than asking them to configure the raw list. I need to give them links and simple binary selections, coupled with a standard output format so they remain confident that they are getting what they want. Links like “Committee A Members” with a choice for “Current members only” or “Most recent five years” or “2000 – Present”. The output should be simple: Contact name (with a link to details), his or her role on the committee and the company they work for. I would argue that this kind of solution is the poster child for “perfect is the enemy of good enough” – making this process more elaborate, or the making the results fancier will only erode the confidence of the average user.

The simplest contact list that we have ever prepared, is rendered from our annual Policyholder Meeting survey, it’s a dirt-simple list: “Who’s Golfing on Thursday”, and it’s an absolute favorite among my coworkers.

HarePoint

clip_image002One of the things that I like most about managing a small technology shop is the agility with which we can operate. We are constrained by the usual suspects, i.e. the limits of technology, budgets and time; but most decisions are easy. If we like something, we can look at it. If we like it after we look at it, we can buy it (we usually don’t look at the stuff we can’t afford) and if we have to install something to a server or change a database, we can do that too. Long term readers of this blog know that I like to write about the good people we work with, whether they have developed an add-on product or have rolled up their sleeves and helped us to build something. One of the companies I have wanted to write about is HarePoint, but I wanted to wait until we could use their product to solve one of our most vexing problems – that day has come!

I learned about HarePoint several months ago when I was looking for better ways to work with dates in SharePoint workflows. While poking around their website, I discovered this list of Workflow Extensions. Some of the extensions looked pretty cool, including the ability to work with arrays, and those for manipulating lists, libraries and individual documents and images. By the time I read about the ability to execute a SQL Query from within a workflow, I was starting to drool a little. All of the extensions seemed cool, but one that really piqued our interest was the ability to move a document to a different library. That feature may sound like kid stuff, but it’s not as simple as it appears. We wanted to copy a document, along with some of its metadata from a document library on our internal farm, to a library in a site on our Internet-facing farm. We asked the people at HarePoint if their extensions could do that and they thought that they could. Unfortunately, our first attempt failed.

Some of the best vendors we work with have distinguished themselves when things didn’t work – HarePoint is now a member of that club. We told them what we were trying to do. We told them that it didn’t work, and we anticipated being told that, in retrospect, the feature wasn’t designed to move documents between farms. Instead, the support crew at HarePoint told us that they thought this would be a good feature to have, and they worked with us to make it work. It took a couple of attempts, but last week we were able to create a workflow that moves a document from a library in our engineering site to a library on our Internet-facing SharePoint server.

This isn’t just an interesting technical challenge; this is the final piece of an intricate puzzle that was mostly assembled over a year ago. You can read a ton about that project by searching this blog for ‘inspection’, but the short story goes like this: When our engineers write an inspection report, a series of SharePoint workflows marshal the reports through various reviews, updates a variety of metadata and stores a final copy of the report in a Records Library. The final step was always supposed to have been to create a copy of the report for our customers to access in a SharePoint site we provide for them, but that has remained a manual process – until today. We successfully tested moving one of these inspection reports from our library to a target library on our test farm. The problem seems to have been solved, and the implementation couldn’t be easier:

clip_image004

I want to publically acknowledge the technical support and development groups at HarePoint. The company has a great product, and these people went the extra-mile to make it even better. I’d also like to acknowledge the members of my staff who did battle with the ever present nemesis in SharePoint (permissions) to put these awesome capabilities to work. If you’re looking for ways to extend the functionality of your workflow-driven SharePoint solutions, you might want to take a hard look at HarePoint.