Sometimes it is Not User Error

clip_image002The most aggressive content management solution we have put in place within SharePoint is the process that we now use for managing and storing engineering inspection reports. This solution evolved from a simple document library to a workflow-driven process that keys off of both document status and content type. The status portion of the process seems to work well. By that I mean, the engineers understand it and the workflows work as designed. The content type portion seemed to have issues; some people were prone to call everything an Inspection Report rather than agendas, confirmation letters and supporting documents. The person who manages these documents for the engineers assumed that these guys simply weren’t following instructions, and she was manually correcting the content type. A decision that “the problem seems to be user error” sits very well with me and, I must say, is supported by historic events.

Unfortunately, it wasn’t user error, or at least, it may not have been user error in every case. Earlier this week, we needed to add a new content type for Response Letters. We inherited this type from one that uses the same template, set up all the metadata we needed and modified the workflows. In testing, we noticed some very curious behavior – when we clicked New and selected Response Letter, Word opened with the correct template, but when we saved the document, the content type changed to Inspection Report. My coworker fought this issue for a while before asking me to look at it. What I witnessed made no sense at all; the content type was changing before our eyes. Both of us ran out of time while dealing with this problem during the day, but later that night, I decided to dig deeper. Note: the description below is brief by design; the steps I am describing played out over a 3-hour period.

  • A check of the advanced properties of the newly created document revealed a ContentTypeID that was consistent with the Response Letter content type.
  • Log entries, added as the first step in each workflow involved, indicated that the ContentTypeID of the newly saved document was “Inspection Report.”
  • Documents saved to the local file system and then uploaded into SharePoint, retained the correct content type.
  • Changing the content type in the document properties on SharePoint, or the ContentTypeID in the advanced properties after the document was saved, fixed the problem and the content type remained unchanged throughout the rest of the process.
  • The problem occurred on documents saved to the library or into a Document Set (as is normal procedure).
  • This same behavior was evident in other content types; it wasn’t limited to the new one.

The workflows were not causing the problem. The users were not causing the problem. Word was not causing the problem. SharePoint was causing the problem; we didn’t know how, or where, but we knew it was happening as soon as the document was saved (not uploaded) to the library for the first time. In addition, the content type these documents were changing to IS NOT the default content type for this library. I started searching for every imaginable search term, but I was having no luck finding anyone that even had experienced this problem, let alone someone who had a solution. Everything I found seemed to indicate that content types should work exactly as we expected them to, not in the mysterios zombie way they were working for us. Finally, I stumbled onto a blog post by John White talking about using templates. The post is excellent, but it seemed to reinforce that what we had done should have worked. It wasn’t until I read through the lengthy post and into the second comment that I noticed that someone else had experienced the same problem we were having. If you read John’s post, we were working according to what he describes as Option C. As Najro points out in his comment, the problem (of content types changing) can be eliminated if you create the content types and templates according to Option B. I will add that the instructions in Option B need to be followed painfully to the letter – I tried taking a few shortcuts and the problem remained. My coworker plied along step-by-step and was successful.

In the absence of a real debugger for SharePoint and SharePoint Designer workflows, we are left with trial and error and logging. Beyond that, all we have are the helping hands of the good people in the community who just happen to be struggling with the same issue. Thanks to John and Najro on this one. Hopefully my contribution will be to point people to the solution in less time than it took me to find it.

A Case for Logging

clip_image002I have been writing off and on about a report processing automation project we are working on for our engineers. As far as report production goes, this is a pretty complex task. These reports go through a healthy review cycle and then they have an afterlife where they feed separate analysis and review processes for years to come. One of the features we added to this process this past week is a Log List.

You might ask “do you really need to log this activity?” – I mean a workflow controlled process is pretty easy to review. The process is driven by a status field, a task list and it feeds a few libraries; surely a quick look in those places will tell you what is going on with any particular document. True, but why make someone look in all those places when we can let them look in one place and see everything? Are we recording duplicate information in this log? Yes! Is this log necessary for control or auditing purposes? No! Did the users ask for this feature? No! The users did not ask for this feature, but I am betting that when we review this with the department manager next week, he is going to say “I like that”.

Logging is a function that we have been making greater use of recent years. It used to be that you logged transactions; period. That was because you needed to log transactions and logging was somehow expensive. Log files take up space. If you don’t think that is a serious concern, ask your Systems Administrator about log files. Logging activity requires extra code. A log database or list has to be built, information has to be gathered and somewhere, a line of code is added every time a log entry needs to be made. Logs need to be explained. Depending on the function being logged, the log files range from self-explanatory to cryptic. At the far extreme, we have Microsoft, whose system logs have spawned an industry that builds log analyzers and log reporting applications. At the opposite extreme though, is the workflow log that simply says “workflow complete”. Our target is somewhere in between, something with enough detail to chart the progress in layman’s terms, but not something that requires the completion of a course in log reading.

What about Dashboards? That question came up in our internal discussion, and we decided that there is an important difference between dashboards and logs. Dashboards are snapshots, a delightful view of critical information that you can quickly check and get happy or become concerned with something you are responsible for. Logs are more historical and more detailed in nature; logs are meant to support a more thoughtful review.

What about Auditing? When something needs to have an audit trail, it’s because somebody want to see that audit trail. They want to know specific things, but I don’t think an audit trail needs to support a thoughtful review. Audit trails are more like a Watchman’s Clock, they record activity for compliance sake. A useful process log is just that, useful. It contains the information that charts the progress of a process, points to where something went off the rails, and tells you about activity between milestones. That last point is important. If we simply look at document status or the activity of a workflow that is driven by document status, we will notice when the status changes. In our log, we also track when the document is saved but the status wasn’t changed. This indicates two things. First, our workflow is working, and second, that the document owner is still working too. No one needs to wonder if the owner just forgot about this document or is ignoring this task. That’s a simple example, but it’s the kind of information that is useful in an environment where people travel and there is no opportunity to simply visit someone’s cubicle and ask “how are things going with the … report?

We introduced this type of log in a transaction processing system over a year ago, and it has captured some interesting results. One pattern of activity we see tells me that they system doesn’t work well enough in support of the user. Too many attempted (or perhaps test) transactions are being performed. As I discuss this with the user, I hope to make this system better. I’m not sure what the process log in SharePoint will tell me, but if it helps us improve the process down the line, it will have been worth our time. If it proves to be useless, we can always delete it.