One Stop Shopping Plus Mail Order

imageA few weeks ago, I received an email about an old post about managing a walking contest in SharePoint. Ironically, we are preparing to kick off the 2014 version of that contest in about 2 weeks and once again, we are managing it in SharePoint. I like to use these little side projects to demonstrate what SharePoint can do out-of-the-box. Some might ask “why focus on out-of-the-box? SharePoint can be so much more.

Their question is not quite correct and I am lying just a little bit.

The problem with their question is that “SharePoint can be made to be so much more” but the making can take a lot of time.

The lie that I’ve told is a lie of omission. I didn’t tell you that my box is bigger than Microsoft’s box. My box includes things like HarePoint Workflow Extensions and Nintex Workflows. HarePoint’s extensions add some very cool features to SharePoint Designer workflows and Nintex, well Nintex Workflows are like a slice of SharePoint Heaven here on Earth.

So, truth be told, I like to show people what we can do very quickly in SharePoint with the tools that we have available to us. That’s important for a reason that most IT departments don’t consider often enough.

Sometimes, people don’t ask for things because they think those things will be hard to build or expensive or that they will take too long.

They aren’t trying to save my time or my budget; they’re just trying to avoid being told “no, you can’t have that.

In the 2014 version of our SharePoint-driven walking contest, we have added two new features. Both are aimed at improving the user experience and both came at the request of my new young colleague Stacy. Stacy is not only the architect on this project, she’s the user. She’s managing the walking contest and she’s building the site with some help from me.

For those of you who are unfamiliar with a walking contest, it’s pretty much what you would imagine:

  • Our company is divided into teams.
  • Each person tracks and records their steps each day during the contest.
  • At the end of the contest, the team with the most steps wins a team prize and the person with the overall highest number of steps wins an individual prize.

Stacy wanted to make two improvements to the accounting process for the contest. She wanted to add options for mobile entry and she wanted a dashboard of sorts for reporting progress.

Mobile entry was easy but, again, it uses a few tricks from our bigger box. You can send your entry into a SharePoint Remote Entry library by including the subject line “10-11-2014 8,996” i.e. the date and the number of steps. A SharePoint Designer workflow, aided by HarePoint’s Regular Expression actions parses the subject line and adds the steps to your step count. A second workflow adds your step count to your team’s total.

We could do all the processing in one step, but I like breaking things into small chunks. That is a carryover from my history of coding in Smalltalk, but it’s a good practice for SharePoint. Small workflows are easier to test and they are easier to “debug” since there really isn’t a “debugger” available in SharePoint.

Employees with iPhones can also easily enter their steps via a mobile view of the Steps Entry form. Actually, anybody could do this, but “iPhone” is linked with “easy” because our MaaS360 mobile device management software allows us to push that mobile form through our firewall without the need for a VPN connection (which people hate to make on their phones).

Finally, we needed to build that dashboard, but we decided to make it functional instead of just informative – that’s where the “one stop shopping” comes from. We started with a Web Part Page and we added an Announcement part and an Instructions part in those top-of-the-page whole width zones. Then we added three useful parts. On the left, we have “My Steps” which is a view of the Steps list filtered on the current user. In the center, we added a view of Team Status that shows the current ranking of teams and on the right; we added a simple entry form for steps.


I have to admit, this is the first time I have ever put an entry form on a dashboard. It works. Having the entry form on the page makes this page the only thing people actually have to look at. My Steps, My Team and, as I look at my steps and realize that I forgot to enter yesterday’s value, I can do it without leaving the page.

Stacy’s homework assignment is to add a chart to graphically display some of these statistics and to make the page a little prettier. Mine is to start walking.

Personalized to the Point of Failure

imageIn 1997 I wrote a bit of code that has finally come back to haunt me. The idea was sound, the implementation was flawed but I didn’t know it at the time. I wanted a way to include some rudimentary (by today’s standards) personalization into our desktop systems. I crafted a Parent Class that contained a System Setup utility and a startup routine to access and apply preferences. I stored things like the fonts and font sizes, the position and size of the main window and certain basic application parameters. This was at a time when most Windows applications contained UI elements that didn’t even scale when you stretched the window, so I was quite happy with the outcome. My mistake? – following Microsoft’s advice to abandon the trusted .ini file in favor of storing this stuff in the Registry. I wrote a Registry interface before a good commercial class library was available for Smalltalk, and that bit of code has reached its breaking point.

It seems that errors with personalization are moving like ripples on a lake, because while I am rewriting a 15-yr old process, we are facing the other side of the personalization dilemma, when to say “no.” We’ve been asked to reduce the detail in a dashboard view in SharePoint. Saying “no” isn’t something I enjoy, but we do have to know where to draw the line between a personal view and the minimal amount of information a user should receive. This is a tricky endeavor. I used to work for a man who viewed the world as a series of binary equations. Status to him was “done or not done?” He would prefer a dashboard with a simple red/green indicator (no yellow required), but that isn’t always what people need to see. “Not done” inevitably leads to questions, and good dashboards should answer the most common questions as well as report the status. This is especially true in SharePoint, where finding those answers may not be easy. Our dashboard highlights reports that are nearly overdue as well as reports that are actually overdue. Seeing that 7 reports are approaching their delivery deadline will absolutely cause you to ask “which reports?” but finding them takes a bit of sleuth work. So, if someone asks me to remove the actionable list of those reports from the dashboard, I hesitate; I want to talk to them and walk them through the scenario(s) that might unfold and test their resolve on that request.

Another oddball request we received this week was actually a multi-part mind numbing exercise. We have a list of contacts, which are organized into three categories. To avoid adding enough detail to identify the suspects, I’ll just call them Type A, B and C contacts. We had to produce a series of reports that include A’s and occasionally B’s but never C’s. That was pretty easy. Now we have a request to include a single B-type contact on a report that doesn’t contain B’s. In addition, we need to include a contact that is neither an A, nor a B nor a C on a report that only includes A’s.

These are the kind of problems that make you question your design effort, not to mention your chosen profession, but a bad design wasn’t the cause of our problem. The root cause in this case is the nature of this list – contacts are people and people aren’t as easy to manage and categorize as things. Rather than try to manage our way through this issue, we opted for a few technical tweaks that provide the necessary capabilities while stopping short of hard-coding individual reports.

I hope that you can see from my opening story that I am a fan of personalization; however, I do think there are practical limits that should be considered:

1 – There should not be an ability to eliminate information that is widely considered to be essential to the understanding of the content.

2 – Personalization should not require arduous coding. This is another way of saying that personalization should be intuitive and available to all visitors to a site. When we start trying to make micro-adjustments to accommodate specific individuals, it’s time to push back.

3 – Personalization should not interfere with knowledge transfer. This might be a special case of the first item in the list, but it’s worth considering separately. When we look at what information is essential to the viewer, we have to consider what an experienced user does with that information as opposed to what the new guy might do with it. It’s fair to say that not all people know what information they might need under certain situations.

Automating People

iPhoneRecently, I was discussing process automation with a few friends – yes that is the kind of life that I have, at least during the NFL off-season. One subject that came up during the discussion is the problem that is caused by automation that goes into a holding pattern. When processes were manual and routing and storage were analog, something was physically on a pile on someone’s desk, an obvious constant reminder of the work that needed to be done. Once we suck the content into SharePoint and wrap the process up in a few workflows, the constant reminder is replaced by a single email – do you know how easy it is to ignore an email? Of course, the just-in-time nature of workflows means that while person A is ignoring the task, persons B through G don’t even know the task exists.

Since I am frequently cast in the role of person A, I might be a good example. One of my tasks is to approve expenses added to a cloud-based expense tracking system. This system notifies me of every charge I make, every charge I have to approve and every subsequent status change during the lifespan of an expense. This system has proven beyond doubt that the only thing easier to ignore than one email, is 100 emails on the same subject. Not only do I ignore the emails I receive, I’ve gone so far as to create a rule in Outlook to ignore the emails automatically on my behalf. Based on this unscientific study, I’ll conclude that having SharePoint send more notifications isn’t the answer. OK, what about a dashboard?

Despite not liking the buzzword, we are rapidly becoming fans of building meaningful dashboards around SharePoint managed content. It doesn’t take very long for list items or documents to pile up and turn a list or library into an unreadable mess. Since we have a few of these pages up and running, we have decided to add a personalized Data View Webpart to one of the pages that will show “Stuff you need to do”, but I’m not sure that’s going to help much. I say that because the rule that I created in Outlook wasn’t designed to ignore notifications, it was actually designed to help me pay attention to them. Based on the subject line, the rule puts the notifications into one of two folders for follow-up. The problem with that rule/folder combination is that it is a Pull operation – I have to go to the folder. Dashboards or status pages are also Pull operations, so those tasks will only get done if I go looking for them. Pull operations are forgotten, push operations are ignored – what’s a process to do?

As I think about this, I realize that there are only two types of reminders that I always respond to: calendar alerts and direct requests from people that I like. I recall Marc Anderson saying at the recent AIIM NE event, that he is more likely to respond to humorous notifications. I would agree with that, but there’s no guarantee. If I know that I need to stop for donuts on the way to work, I create a calendar item, set an alert time so that my phone beeps while I am driving; it works, I stop every time. When the nice woman from accounting calls or sends me an email reminding me that I have (n) expense reports to approve, I login and take care of them.


On the other hand, I told my wife that I was planning to stop at the ATM earlier this week; she wrote my planned withdrawal in the checkbook, but I forgot to stop. I also forgot to tell her that I forgot to stop, putting our checkbook out of balance – my bad.

I think a combination of push and pull solutions might help. Something like an alert that says “You have to do something” where the link takes you to a DVWP that includes actionable items. I.E. if you need to review a document, the link will open the document for you. If you need to approve a process step, the Approval button is right there. Maybe calling a person’s attention to an item coupled with an easy-to-use option to act on the item, will be enough to even get busy people to respond. We are even looking into VPN on Demand, so we could send these notifications as directly actionable items to an iPhone. If I add a bit of humor, maybe I can even increase the success ratio.

Road Trip

clip_image002Road trips come in three varieties: You need to get from point A to point B, you want to discover something or you want to show somebody something. I’ve been on way too many of the first type, including two where points A and B were 3,000 miles apart; those trips are not always fun. Trips of the second type are almost always interesting, but the third kind of trips are the ones that are fun. When it comes to SharePoint, it’s the same story.

We have been on the first type of road trip for the past few months. We have built out a complex document handling project, aided by content types, workflows and bits of metadata. In addition, we spent time wiring up a bunch of Data View Web Parts and detail views of this content to elicit some management information about the process and present it on a dashboard. Those were the major goals of the project, comprising points A, B and several others in the alphabet. We are just about to call the project done. Handing the management dashboard over to the department VP was a rewarding moment, but late last week we took this project on its own little trip.

As we were completing the management dashboard for this process, I asked the VP Engineering if he would mind if I showed it to others in the company as an example. He offered something better – an opportunity to present the dashboard to the entire company at a roundtable style meeting the engineering department holds. This was a fantastic opportunity to market SharePoint within our company for several reasons:

Real Solution – In the past, we have tried to get people excited about SharePoint by showing example solutions designed to highlight a specific feature. Unfortunately, in those cases, we are always asking the audience to work too hard. We are asking them to understand what we are doing, and imagine how they might map the demonstrated capability to their department. We, SharePoint practitioners, do this all the time, but that’s because this is our job, and we already understand SharePoint. It’s hard for people to juggle an unknown and an imaginary scenario at the same time.

Not an IT Solution – The next worst thing after a hypothetical solution is an IT solution. I have demonstrated libraries, processes, lists and workflows around IT Asset Control, communication with IT vendors and even the way I handle the AIIM New England Chapter minutes. These were all real working examples, but they were all in support of a world my audience still had to imagine. Everyone in our company knows what an engineering inspection is. Everyone has seen at least one inspection report and everyone knows how important the inspection activity is to our operation. When we demonstrated the SharePoint solution, they only had to understand the part that SharePoint was adding to the process.

In taking advantage of this opportunity, we were careful not to mess it up by making it an “IT Thing”. My coworker, the woman who built the solution, was careful to express everything from an engineering point of view. She talked about how the engineers work, and how the Vice President uses the dashboard. She pointed out the benefit they derive from the project. She didn’t use the words ‘workflow’, ‘metadata’ or ‘content type’. OK, she may have used ‘content type’ when answering a question, but the point is, she kept her demonstration in business terms. When we were describing the dashboard, we emphasized how the information being displayed was gleaned from the document library. I may have used the word ‘metadata’ at that point, but I tried to avoid saying it twice. We tried to put distance between SharePoint and the fat-client / database server applications our fellow employees are used to. Yes, I know SharePoint is or at least resides in a database, I even mentioned that to my coworker and we both agreed “let’s not go there.”

Even though the process is largely similar or at least analogous to a database system, there is a beneficial distinction between reporting from SharePoint metadata and reporting transactions. Reporting from SharePoint appears to be a bonus; we aren’t adding metadata as a transactional element so we can report on it. We add the metadata so we can find our reports, so we can organize our reports and so we can manage the process. By manipulating the metadata behind the scenes of a management dashboard, we get additional benefit. You can split hairs and point out that our dashboard is basically the same as a monthly premium report. You may even be right, but as I said at the outset, this was a marketing exercise.

The Problem with Metrics

If you can’t measure it, you can’t manage it.” I don’t know who coined that phrase, but I have to tell you, I have some problems with the concept. My primary problem is that “it” often refers to a person, or at least the activity of a person and I think people should be managed more holistically. Another problem that I have is that since we are managing people, we are subject to the behavior of people who know they are being managed.

My first job after college was as a programmer/analyst for Burroughs Corporation, working in one of their manufacturing plants. System responsibilities had been hastily reassigned during the period between my accepting the position and my start date, so I found myself responsible for payroll, HR and work management systems. The product made at our plant was memory sub-systems and I was exposed very quickly to two problems with metrics. Problem one was the ease with which metrics can be manipulated. Our plant manager, aware that a key metric of his was the number of days between receiving components and shipping an assembly, simply purchased a used trailer from a trucking company. Material that arrived before we were ready to deal with it was unloaded onto our loading dock and reloaded into his trailer. When we were ready to start using the components, they were “received.” This little bit of subterfuge kept the metric that determined his bonus, nicely in the acceptable range.

Inside the plant, and much more problematic for me, was the fact that we measured every operation that was conducted by every person on the assembly floor. If five people were operating machines that stamped memory chips into circuit boards, one of the systems I was responsible for would calculate and report the cost per chip inserted. Later, I would use that value in other calculations. Unfortunately, the collection system didn’t differentiate between new assemblies and rework. So, while the guy building new circuits was feeding racks of memory chips into an automated press, the woman at the next station was unsoldering and re-soldering defective chips by hand. He might insert hundreds of chips per hour, while she struggled with five. Mine wasn’t an ethical dilemma though. The nature of the repair business meant that sometimes we would receive a part for repair that was no longer in active production. In the reporting side of my system, when I had to do the math that involved “average insertion times” the results were too small to measure and they would fail on a divide-by-zero error. Discarding these results made the woman look bad, including them crashed the system.

I am recalling those early struggles as we study the results of our newly implemented analysis of engineering inspection reports. One of the changes we have to make is to properly reflect inspections that people performed, prepared and participated in. These aren’t just three categories, or three bits of metadata, they are three distinctly different statements about someone’s role in a very important process. Not only is the distinction important, it’s important to know what is what, once these variables are calculated so the right metric is used in the right formula. For instance, the lead time on scheduling the inspection is properly attributed to the person who was in charge, so a guilt-by-association attribute should not be applied to every participant. There also has to be a distinction drawn between the person preparing the report and the person who performed the inspection. We aren’t talking anything on the scale of squirreling stuff away in a trailer, but if the relative performance of individuals is being compared, the right metric has to be used.

So why am I trying to commit this to memory (that is why some of us have blogs) “because it’s so easy to get this stuff wrong” and once your mistake is incorporated into a management report, nobody will ever know. I don’t know about you, but when I get a number to appear in the box on a screen prepared in SharePoint Designer, I am happy, and I feel close to being done. When I look back at that code and see “@Prepared_x0020_By” I know that it’s the right column, but when my coworker is reviewing the code where I refer to “$EngName”, she might be tempted to simply assume it’s the right variable. This is where SharePoint development has to mirror systems development. We have to document what these reports are supposed to show, and we have to document what they actually show. Then, we have to test the process, and the results.

Ta Da!

imageWhen I was a kid, I tried several times to make a product from the bowling pins my father swapped out of his bowling alley every summer. I bought a used lathe and I came up with ideas from mortar and pestle sets to small planters. Pictured to the right is a paperweight and paper clip holder. I would sell these things door-to-door, starting with a neighbor. I would always be excited and a bit nervous as I began the sales process. Sometimes, the initial feedback was negative, like when people realized that a top-heavy paperweight is an accident waiting to happen. This week, I had that same queasy feeling as we delivered our first homemade SharePoint solution.

Eight months after rolling out our workflow-driven process to manage engineering inspection reports, and three months after perfecting that process on the heels of a post-implementation review, we unveiled the management dashboard for the head of the department. This guy manages a group of engineers, so to say that he’s case-hardened is an understatement. We were optimistic, given that he had defined a few general concepts that he was interested in, but we were also a little nervous. After all, this is the first time we had built a dashboard. I have mentioned before that the hard part of introducing technology to people is getting the design right. It’s hard because you don’t know enough about their business to know how they manage it, and they don’t understand the technology well enough to know what to ask for. In an agile environment, you can work from a build-test-review-redesign cycle that keeps things flowing, but that process didn’t suit our circumstances. In this case, we needed to keep working while the customer travels and conducts business. He didn’t have a lot of time to interact with us. This time, we got a “here’s what I want” speech to be followed a few months later by a “let’s see what you’ve got for me” meeting. Fortunately, I have a resourceful team.

We took a pretty simple approach for a dashboard; low on glitz but high on useful. We worked from a web part page with four columns which gave us a pretty good canvas. The left column holds the aggregate statistics about the current activity, including the things that might be going off the rails. The right column presents the historic view of this process, including the reports we are back-filling into the library. In the center is a group of detailed activity by engineer – one column for the current year and one for the prior year. In addition to presenting the information requested, the parts we assembled illustrated the key things we wanted the VP Engineering to know we can do, including:

  • We can count things based on metadata and content type.
  • We can perform date math on various status milestones. As I read that, I feel cheated, since it took a lot of work to figure out some of the bits and pieces of date math.
  • We can create a wide variety of views that let you quickly drill down into the details behind a particular statistic.
  • We can group things by person, type, facility, date, or any other attribute you like.
  • We can perform basic math and logic functions like Average, Total, And, Or, Not, etc.
  • We can use fonts and colors to draw your attention to certain elements (although we didn’t spend a lot of time proving this).

Jane Zupan at Nuxeo introduced me to a phrase that I love – Content-centric Applications. I told her I was going to steal that phrase to describe this process; because this is a business application. It just happens to be gleaned from the normal processing of reports that used to be tossed onto a virtual heap called the K: Drive. This is truly the beautiful part of SharePoint, the fact that we can collect, store, manipulate and display data about a process during the normal course of a business activity, and then turn that data into information.

Drawing conclusions from the demonstration, I’d say we met our goals. I counted at least three “I like that’s” and a couple of “nice’s”, and the all-important “whoa, what the heck is going on there?” That last observation led to a request for a few changes, but the request itself indicated two important things to me: 1) The VP understood the tool we had given him, and 2) He had begun to understand what else we could do. That’s a win as far as I’m concerned. We have a little more work to do before we can put this project in the “done” column, but we aren’t throwing anything away and we aren’t far from the bulls-eye.

Reveal Codes – A Different Kind of Dashboard

clip_image002A few weeks ago, I was one of the presenters at AIIM New England’s event “Usability Matters.” I included a demonstration during my presentation and I felt compelled to point out that the default view of a particular library was the one requested by the user. That view extends several miles beyond the right border of the browser, and I can’t figure out why anyone would want to look at it. After the event was over, several people told me that their users also seem to prefer the comprehensive unending view to any of the custom views they have created. I put all of this out of my mind until I read about Bill Gates testifying in an antitrust case brought by Novell (who owned Word Perfect) where he is quoted as saying “Word won because it was the better program.” Thank you Bill, I’ve been waiting 13 years for someone to say that. You see, 13 years after switching to Microsoft Office, some of my users still complain that they miss Word Perfect, and they especially miss Reveal Codes mode.

Of course, Bill Gates saying Word is better won’t convince my Reveal-Codes-loving users that Word was ever any match for Word Perfect. I still wonder why people wanted to see all the details behind the results, especially when those details were practically indecipherable; Reveal Codes was like reading a website in the View Source window. Reveal Codes was only useful to me to see why my document didn’t look right, more like Reveal Errors mode. The same is true with the All Items view in SharePoint, the only time it’s useful on a large list is when something is wrong with one item that keeps it off a useful list. Unlike Word Perfect, where the rendered document was on display above the codes, SharePoint offers no real insight into the things we’ve done wrong – we have to build that ourselves.

Our recent experience building dashboards and analysis pages with Data View Web Parts has pointed out their usefulness in highlighting errors, although that was never our intent. The first error we encountered was on the analysis pages for our Policyholder Meeting. The DVWP summary page said that 90 people were coming to Wednesday’s dinner, but the view of dinner guests included 92 names. The reason for the discrepancy was that two people had indicated that their spouse was attending dinner, but didn’t check their own box. I only noticed this because I was testing the accuracy of the DVWP, but it pointed out a problem with our survey, albeit an odd one. As we have been building a dashboard for our engineering inspection reports, we are finding similar odd results. There have been a number of times that we wire up a Data View only to find ourselves saying “that can’t be right…” Subsequent investigations invariably show that someone made a bad choice. Clearly these aren’t the statistics we were hoping to show, but I realize that they are important, and that we won’t find them any other way – we are already near the point where a list of all reports is overwhelming.

Since SharePoint doesn’t give us an abundance of error-checking options, it’s incumbent upon us to design views or widgets that tell us when something is wrong. We have found that pairing a simple indicator, say a count of items matching a condition, with a view of those items works very well. An example is a DVWP that shows Inspections by engineer. The part includes the number of inspections and the number of reports completed. One engineer popped off the list immediately by having completed 10 inspections but only 5 reports. A view sorted by engineer and showing all the gory details highlighted the problem inspections. One was an inspection report that was uploaded early in the life of this tracking system; the engineer had so many issues that he started over, but he never deleted his first attempt. A second error was caused when the engineer, in preparation for sharing the Word document with his customer, removed all the metadata before saving the document one final time. Since Word manages SharePoint metadata, this action put our list in a state of great discord.

We have been focusing a lot lately on ways to improve SharePoint adoption. I am discovering that adoption brings with it a new set of demands – if people are using SharePoint as a business tool, we have to ensure the quality of the information stored there. The analysis techniques we are using to build management dashboards are also helping us find documents and list items that need our attention. Maybe, instead of just highlighting any odd looking results, I’ll add a link that says “Reveal Codes” – that just might make everybody happy.