Apparently I am a Dumbass

imageEarlier this week, I gave two presentations in Orlando, FL. One, at the AIIM Conference was well received; it was about SharePoint and ECM and will take a few blog posts to sort out.

The second presentation was to the Service Provider Executive Forum (SPEF) and unfortunately, it’s very easy to describe. The vendors attending SPEF are the ones who sell scanning services, document storage, printers, copiers and scanners. SPEF was one of those conference-within-a-conference deals, and my presentation was part of a ‘User Feedback’ panel.

I worked hard on this presentation. It was a good presentation. It just wasn’t the message this group wanted to hear. I’m sorry for that, but I don’t have a great history with this industry. My message wasn’t a feel-good story. I talked about why we don’t use many of these services, and how our most recent encounter has gone somewhat off the rails. I had hoped that the organizer was right when she told me:

They will appreciate your candor

Here’s a tip: if you ever hear those words, run. Drop everything, don’t tweet about it, don’t snap a picture, don’t think twice – just run.

Our story in a nutshell – we are small. This industry’s story in a my-opinion-nutshell – they are commodity brokers seeking a sale at any cost.

The best analogy I can come up with is that buying these services or products is like buying a digital camera; they all take pictures, they all have good battery life and they all let me move pictures to my PC, tablet, Cloud storage, etc. Once I decide the few, somewhat unique features (zoom, video, and form-factor) that I want, it comes down to picking the one that’s most comfortable.

The problem – that we have experienced – is that some salespeople want to be the mystic swami in your life. They don’t want informed customers. They don’t want to hear your story; they want to tell you theirs.

My presentation carefully presented our position, our real-life experience and then I took “questions” I put that in quotes because most of what I heard were not questions per se.

Why didn’t you buy a scanner? Scanning to SharePoint from dedicated scanners seems to provide more robust options than scanning from multi-function copies (MFCs)…who knew? More importantly, who cares? The fact of the matter is that we don’t have a space for a dedicated scanner, nor do we have the volume to justify one. In addition, we only want to have to learn how to use one device.

Why didn’t you do an RFI first?  Seriously, an RFI for two MFCs?

You’re ignoring the money you could save by outsourcing all the scanning to us. I reminded this person that I had explained the very, very high cost of teaching someone how to classify and index our documents.

Finally, a white knight emerged. Addressing the audience, he said:

What I’m hearing is that you guys are saying that Dan is a dumbass


His point was that I failed to engage with them according to their preferred sales method. Apparently, from their perspective, that was my mistake.

One woman came up to me afterwards and said “I don’t think you’re a dumbass, but this managed metadata thing sounds like a unique requirement.” I reminded her that it’s a mainstream feature of a major product. I didn’t invent it; I just want to use it. More importantly, it was a written requirement that we were willing to explain.

One guy followed me to my next presentation, all the way telling me that if I had done an RFI he would have been able to explain why we needed a scanner. I explained, again, why we chose MFCs over a dedicated scanner.

He turned up again later as I was boarding the bus to the conference social event. He started telling me about the features of the scanner. The one we don’t want – as if he thought that if he gave the same pitch often enough I might just buy one. My friend (who sells a high-end ECM product) said:

This guy sells hammers. You have a screw, but he still has to try and sell you a hammer.” He added, “It’s a sales technique we call show-up and throw-up.”

I’ll leave you with the message in my final two slides. When we look for vendors to help us, we look for a vendor who will:

  • Be a business partner with us
  • Let us do the portions of the project that we feel we need to closely control
  • Accept that we know our business and know enough about yours to evaluate you/your product or services as compared to others that are available
  • Think long term; think beyond this sale and this commission.

Oh, and don’t lie to me. I can handle the truth.

Is This Right

clip_image002Last week, I was talking about accuracy. This week, I learned that there is something more important than accuracy – confidence. We have been working on the design of a SharePoint solution that is both simple and complicated at the same time. Of course, we are concerned about accuracy, but the people using our system will have to feel confident that they have selected the information that they need. This isn’t a challenging SharePoint problem; this is a design issue, a usability issue and ultimately an adoption issue.

Our project involves a contact list. That’s simple, but it’s really about 10 contact lists, and since they need to be exposed on both an internal and an Internet-facing server, it’s really 20 contact lists. The “master” list is a group of people who serve on a variety of committees, and that’s the only list we want to maintain. The people come and go and the committee membership changes from year-to-year. We need to know who is on each committee today, we need to know who was on each committee 2, 3, 5 and perhaps 10 years ago, and we need to know this from multiple angles. For example, I might simply want to be able to contact the members of Committee A. On the other hand, if I’m about to meet one of the people on the list, I might want to know every committee he or she serves on. In yet another scenario, if a person is retiring, I might want to know every committee the person ever served on. We’ve cracked the list design issues, and with the help of HarePoint Workflow Extensions, we’ve cracked how to keep the lists in sync on the Internet-facing server. OK, so what is this confidence issue?

Let’s think about those scenarios again. In the first one, I want to see the names of the people who are currently serving on Committee A. Obviously, I could simply filter the list on a few columns to distill the contents down to right group. That would work, but while most people know how to do this, they don’t want to have to do it every time; that’s what views are for. Views, now this is where we start to get into trouble with usability.

Views are a fantastic feature within SharePoint, but neither lists nor views scale very well. That’s not to say that you can’t put a ton of items in a list, or create a bunch of views, but sooner than later the results are untrustworthy. Once you get a couple of pages of items in a list, you have to resort to sorting or filtering to make it useful. Similarly, once you have 15 or 20 views on a list, the selection is equally unmanageable; in fact, a large selection of views might even be worse than a large collection of items. When I am setting sort and filter parameters, I know what I am doing; the trouble is that I have to be able to imagine the results as I proceed. When I choose a view, I am relying on someone’s ability to name a list intelligently. If I make the user figure out how to get the information they want, I put the burden of accuracy on them and it becomes a confidence issue.

There’s a very fine line between wanting my users to understand SharePoint and forcing them into a situation where they are uncomfortable. When they get to that point, they are going to ask someone else (maybe me) to get them the information they need, and at that point SharePoint and I have failed.

We have to find a way to tame a large list that can be rendered in a large number of ways. I can’t just substitute a bunch of input fields for a search or data view web part, that’s really no different than asking them to configure the raw list. I need to give them links and simple binary selections, coupled with a standard output format so they remain confident that they are getting what they want. Links like “Committee A Members” with a choice for “Current members only” or “Most recent five years” or “2000 – Present”. The output should be simple: Contact name (with a link to details), his or her role on the committee and the company they work for. I would argue that this kind of solution is the poster child for “perfect is the enemy of good enough” – making this process more elaborate, or the making the results fancier will only erode the confidence of the average user.

The simplest contact list that we have ever prepared, is rendered from our annual Policyholder Meeting survey, it’s a dirt-simple list: “Who’s Golfing on Thursday”, and it’s an absolute favorite among my coworkers.

Communication Failures

clip_image002I’ve been following a communication disaster on LinkedIn this week that could have been avoided so easily, and yet, as easy as it is to say that, I might just make a similar mistake. Toastmasters International released a new dashboard for district officials to review their district stats, and lots of people had a lot of problems. The new version was built in Silverlight, and wouldn’t run on Linux systems, older Macs, and most mobile devices – #fail. After over 100 comments were added to the discussion, it was announced that an HTML version is coming “soon”. After the comment count climbed even higher, it was announced that an iOS version is also “in the works”. When you combine those and other answers, you have a really good story. The problem is: they never actually told the story. Communication failures occur for a variety of reasons, but the big three include:

I Never Told You – Information is like money during periods of high inflation. The longer you keep it, the less it is worth. Unlike money, information’s value can actually turn negative if you hold onto it too long. If someone ever earns the right to say “when did you know that?” – you are in trouble. Companies have lots of reasons for withholding information, including not wanting to disappoint investors or give a head’s up to the competition. Internally, there is no reason to withhold what you know from your customers (I’m trying to not call them users). We are currently telling people what our plans are, and we are letting them know what is/may be coming in the future that will impact those plans. We are talking about SharePoint 2013, about Windows 8 and about the next version of Office; we are even offering to let people look at our test installations. We are also talking about ECM and why the things that make SharePoint different from the K:-Drive really matter. There’s very little downside for us in sharing what we know, when we know it. Notice that I didn’t mention the next version of SQL Server – see the next point.

I Didn’t Translate – The folks at Toastmasters started out talking about Silverlight requirements, Intel chipsets, screen resolutions and monitor size, long before they said “oh wait, there is an HTML version coming.” The only thing that those comments did was add to the confusion. People outside of our department don’t want to hear anything with version numbers, hardware/software requirements and service packs. In addition, there’s a list of words I really try to avoid, including: platform, content, repository, database, Data View Web Part, JavaScript, HTML, web service and many, many more that only serve to make their eyes roll back in their head. We are trying to remember to talk about SharePoint in business terms, or in terms of the business process we are working with. Not only do our customers understand those terms, they appreciate the fact that we do, too.

I Told You My Problems – Those of us working the IT side of the fence always have a new language to learn, a new version of something on the horizon and a new bug we can’t crack. Newsflash – nobody cares! In the old George of the Jungle cartoon series, there was a cartoon called Super Chicken. When his sidekick would complain, Super Chicken would chide him with “You knew the job was dangerous when you took it, Fred!” Well, we all chose to work in this field. Droning on and on about how hard you are working, all of the things that should be but aren’t working or even how awesome your last accomplishment was, doesn’t count as communication. Save those stories for the next user group. Your customers want to hear about the progress you are making on the new feature they asked for. They want to see a demo and they want to hear you describe things in a way that tells them you really do understand the business requirements they trusted you to meet.

The Toastmasters discussion is still raging on LinkedIn. People are getting defensive on one side and angry on the other. Messages are being written in UPPERCASE. It’s reached the point, like all long discussions on LinkedIn, where newcomers who haven’t read the earlier comments are repeating complaints that have already been aired. There’s no way to stop this; there’s no ‘Undo’ option on a communication failure. These are mistakes you simply have to avoid making in the first place.

Baby Steps

clip_image002What do SQL Server, Social Media and SharePoint have in common? They are all things that we are learning, or still learning and they are all places where we just might make some mistakes. Earlier this week, one of my team members had to make a change to the structure of a table that we are using while testing our connections between SharePoint and our backend database. The change triggered that queasy-feeling-generating message about having to drop and recreate the table:

This young woman leans toward cautious; she started exploring the options for altering her approach. I’m not sure she appreciated my advice, which came wrapped in a question: “what’s the worst that can happen?” After all, it’s a test database. I tried to sweeten the deal by adding that “we might as well see if this works” and that even if it failed, it would “give our system administrator a chance to test his backup and restore process”.

While she was struggling with SQL Server, I was learning more about using social media in our business. I spent a good part of this past week at the Enterprise 2.0 Conference – Boston, now officially known as E2Social, listening to people talk about effectively using social media to promote or grow your business or using it to improve communication and collaboration within your organization. In all these sessions, one message came through clear – mistakes are going to be made. After reading this far, you probably won’t be surprised to learn that I’m not bothered by that fact; we’ve been making mistakes with SharePoint for years. The important thing to know about those mistakes is that none were very large, none cost our company any much money and every mistake was a learning experience. Nathan Bricklin, Head of Social Strategy at Wells Fargo summed it up in his keynote presentation at E2. I’m paraphrasing here, but the quote I loved was:

“if you take baby steps and fall, you can learn walk. If you plan a project for a year and fail, you won’t be able to figure out what you did wrong”

The other feel-good moment in his presentation was when he reminded us that: “…it’s like baseball, you can’t win or lose the game in the first inning” that quote might not have been all that well received at this point in Boston, but we all understood the underlying message.

In our office, we have been playing this game all year as we have been experimenting with the various ways we can connect SharePoint to our backend data. We realize that if we can tie structured data to unstructured data, we can automate, or support through automation, a much more significant part of the target business process. Unfortunately, trying to make these connections has been underscoring why the word “frustrating” in in my tag line. We have managed to assemble successful proof-of-concept pages, parts, lists and workflows for almost every feature that we need to use. We have run into almost every error you can generate (they all seem to be related to permissions) and we have solved or worked around most of them. In short, we have fallen, but we are learning how to walk.

I never have much luck when I predict what next week’s post will be, but if things go according to plan, I’ll be sharing some of the specific experiences we have had with our connection quest. If things go according to history, I’ll be talking about something totally unexpected that will happen during the next few days.

By the way, if you want to know how to make it possible to let SQL Server drop and recreate those tables, see Geoffrey Emery’s excellent blog entry from 2009. I discovered that shortly after we began our migration from DB2 to SQL Server. DB2 warns you when a table has to be dropped and recreated, but it lets you decide whether or not you want to make the attempt. Unless you change your settings in SQL Server Management Studio, you get the warning shown above, but you can’t actually proceed. I will add for the faint of heart, that once you make the changes that Mr. Emery outlines, the drops and recreations just happen, no muss, no fuss, but no warning.

Up’s and Down’s

Somebody once told me that the thing they like about this blog and my presentations is that I include the stuff that just didn’t work. Somebody else told me they like reading stories about SharePoint really being used in business. Well, this past week included both elements, occasionally on the same project. 
The week started with the alleged final step in our time reporting system, the Accounting View. We wanted what appeared to be a simple view, grouped by two columns and with totals on two others. Try as we might, we got totals without groups, or groups without totals. The later was the most frustrating because the “Total” operation was working, but it was performing a Count instead of a Sum. Research ensued to see if we missed an update, forgot to turn on a service, or simply didn’t understand the operation. Nope, our stuff looked just like the examples, it just didn’t work. “No problem”, I thought, “let’s just create an Access View.”
The Access View appeared promising at first, but something told us it wasn’t going to work. The example on Microsoft’s site clearly showed Access with a “Publish to SharePoint” option on the toolbar – ours said “Save to SharePoint”; cue the ominous music now.  We quickly managed to get the report looking great, but it would not show up as a selectable view. Once again, we trotted out the usual suspects. This time we hit pay dirt; the Access services were not turned on. With those started, we tried and failed again. Then, my Systems Admin pointed out that we had to start these and one other service somewhere else. He made those changes but still no luck with the Access View. That’s OK, I hate Access anyway. Since we only run the report once a month, we will run it in Access this month and then setup the report using SQL Server Reporting Services. I hate SSRS too, but that’s because I pretty much hate anything that ever was advertised as a Report Writer. Programmers hate writing reports, but they hate using Report Writers even more. Programmers like writing Report Writers for other people to use.
While this “last little thing” was consuming my week, I tried to get back to the other SharePoint task on my list; setting up a demonstration of the ways SharePoint can work with back-end data. I set up a test database and a few tables on my development server. Then, I wired up a BDC connection and immediately fell into the “you don’t have permission to do this” trap. I knew I had done it before, but that was on a different test server, the one that crashed just before Christmas. No problem, my Systems Admin made all the appropriate entries and I was on my way. I also wanted to include a Data View Web Part example, but I discovered that, although SharePoint Designer can connect to SQL Server to create an External List, it can’t connect to create a Data Source. I’m still researching this one, so if you know the answer, please comment below. If you provide the answer and I ever meet you at an event, I will buy you several beers, I promise.
The good news is that we ended the week making progress on all fronts. The other good news is that our users seem to both understand that we are learning our way around new territory and they like the solution we are giving them.
If you found yourself reading this and thinking that I seem to dump all the hard stuff on my Systems Admin, you’re right, but not because I’m lazy. When SharePoint was my vision of the future, we ran a down and dirty installation. When we upgraded to SharePoint 2010, we knew SharePoint was a keeper and we installed everything the way you should install it. We also put an end to my just going out there and tweaking things based on a blog entry I read during the football game. I have a great Systems Administrator, and I have a great SharePoint developer working with me now, so we are trying to set ourselves up to operate according to plans, procedures, policies and rules. This might mean that problems take a little longer to solve, but when we solve them, we know we won’t have to solve them again. 

PS, in my quest for ways of manipulating data from SharePoint, I am looking into some third-party products. I will start by saying I don’t want to buy one, simply because I don’t want one more thing on the upgrade checklist. But, I am open to suggestions. Keep in mind, we develop systems outside of SharePoint, so we are only looking to extend SharePoint’s reach a little bit.

Take Two

clip_image002On Monday, my small development team will take on our first SharePoint project. Whaaaaaaat? I know, I have been writing this blog for over a year and a half, and I have been talking about SharePoint since 2006, but most of our work to-date has been the result individual efforts. More accurately, our success has been the result of a combination of loosely aligned individual efforts. Maybe a group of people decided what we needed, but one person usually ended up with the “you’ll set this up in SharePoint?” task. Fortunately, SharePoint has matured, and our efforts to get it accepted as a platform for business solutions are starting to pay off. Our understanding of SharePoint has also matured and, not surprisingly our first team effort is going to be to redesign and rebuild of an existing solution.

What We Did Right – Every time you rebuild a “system” you need to consider what worked and what didn’t work. I like to start with what worked. The main thing that we did right was to move this system into SharePoint. We are collecting attorney time entries so we can better allocate expenses to individual projects. Seems simple enough for SharePoint, and since almost every other technical thing our attorneys do involves SharePoint, it was certainly a good place to put the application. Apparently, the only other thing we did right was to decide up front that we would collect time entries for one year and then start a new list. Initially, that was to accommodate SharePoint 2007’s limits, but it works now that we realize we really do have to start over.

What We Did Wrong – In fairness to me, we only made a few mistakes, but since this is such a small process, they are obvious. The first mistake was getting a little bit ahead of the curve. Our little process made use of several Lookup Columns before Microsoft added the ability to prevent people from deleting values that are in use. If you delete the item from the supplying list, the column in the consuming list goes blank. Our second mistake was not understanding why they would want to delete these items in the first place. Our third mistake was forgetting the myriad ways SharePoint solutions can fail. I want to talk about this third mistake, because it highlights the major difference between building a fat-client or web-based system and building a system across SharePoint – decentralized control.

In a fat-client solution, the stuff I’ve been building for thirty years, you collect transaction data through a well controlled user interface. The supporting data, lookup tables, references and validation rules, are tucked neatly behind the scenes, out of harm’s way. In SharePoint, all of that data is visible to someone. No matter how well you set things up, if you are building a solution on a user’s site, they have access to it – more access than you would give them to SQL Server. Of course, that’s the whole point, it is their data; they want to be able to use it and they want you to use it. We built this solution partially to eliminate the need for our attorneys to maintain duplicate lists. The challenge with SharePoint is that I can break a Lookup Column by deleting an item, by deleting a list, by changing the permissions on a list or by moving a list to a new location. All of these actions are valid things for a user to do in SharePoint, so they all have to be dealt with. I could build a separate solution in SharePoint, where all of this was locked-down, but that would defeat the purpose; that would be simply putting a fat-client inside SharePoint.

We need a better solution, and that’s why I am bringing two more people onto this team. I built this solution from the point of view of the attorneys, so one of the people I am adding to the team is going to represent the accountants. It’s not that I ignored the accountants the first time through, but it’s safe to say I could have done a better job implementing their requirements. The second person I am adding to the team is the guy who is going to document this process. Part of the reason things went wrong in the first version is that people didn’t understand how changing something in one part of SharePoint could affect something located somewhere else. In building this revision, we will document everything everyone needs to know, and we will put that documentation in front of them. We have a pretty good working solution today; next week, we are going to make this a great solution!