Happiness is…

imageRealizing that the thing you need your coworkers to do in order for your SharePoint solution to work, is something they already like doing.

Yeah, that was too long for a title, but that’s how I feel right now. It’s as if my doctor had told me that “in order to remain healthy, you need to eat more Heath Bar Klondikes.” In addition, my new found happiness reveals one of SharePoint’s often overlooked strengths. Yes, if you want to, go get a Klondike before reading; it’s ok.

The project that I’ll be working on while my coworker and friends are in Vegas is one where we hope to link several aspects of our engineering process. To do this, we needed to refine an identifying number that has been in use forever. To really achieve that value we are looking for, we need a way to make it seem like this new way is the way we’ve always done things. The really cool thing is that SharePoint can make this all possible.

I’ve talked about bits and pieces of the neighborhood in which I’m working before, loss control inspection reports, recommendations and composite views of data. We have a solution that lets us track and support the development of a loss control inspection report. Ultimately, that process ends with the final report being stored as a PDF, in a records library – locked away forever. However, the process leaves the Word document that created the PDF, behind for review. Keep that in mind.

We also have a solution that tracks the recommendations our engineers make in those reports. These ‘recs’ have a long and complicated life. They get written. They get reviewed. Customers comment on them, engineering peers comment on them and once or twice a year, they get ranked:

  • How important are they?
  • How are they being addressed?
  • Where are they in that life-cycle?

We built a series of custom lists to aid in this process. We also built a couple of composite pages, aided by some data view web parts to assist in that review. Still, something is missing.

What’s missing is the link between the reports in which the recs originated and the recs and the rec-review process. My job is to establish those links.

This is where ECM projects often run into trouble. We have hundreds of reports. We have hundreds of recommendations. Most of the recommendations are ‘closed’ but they still have value. A lot of information is coded into those custom lists, but the real value, the mother-load of information, is in the original reports. The trouble is that this isn’t a one-and-done process.

The recs can appear in many reports over time and any single report can refer to multiple recommendations. It takes time to review, reread, search and add metadata to those reports in order to link them to the lists…or does it?

Well, it seems that it will take some time, but not much. It also appears that going forward we can keep the reports and the recs in sync, with little or no effort at all.

The key to my happy day lies in the fact that SharePoint, with the aid of some workflow add-on products, can read those reports. Further, it turns out that the way our engineers like to write those reports, their “style,” the style they’ve been using forever, lends itself to being picked apart by Regular Expressions.

The reports already have metadata that tell us what policy they refer to. The recommendations have an ID number that tell us what facility they refer to. Today, I built another SharePoint custom list that links those facility ID numbers and our policy numbers. Now it looks like I’ll be able to have a workflow read the reports, pick out the recommendations and automatically maintain the activity in the recommendation lists. If this is successful, once an engineer finalizes a report, a SharePoint workflow will create a stub new recommendation or a stub recommendation update item, and then create a “please complete this item” task.

I’m talking in the future because I’ve only completed the proof-of-concept steps. I can read the reports. I can find the recommendations. I can build a new composite identifier and I can stick it in every custom list where it’s required. I usually wait until I have a solution to write about it, but this was so much fun, I needed to share it. When I finish this project, I’ll let you know how it works, in this blog and at AIIM14. If I fail or partially fail, I’ll write about that, too.

Blah blah InfoPath blah blah

imageWow, has anything ever attracted as much attention as Microsoft’s decision to kill a mediocre product? My inbox is full of “InfoPath is Dead” and “Microsoft is Killing InfoPath” emails. I’ve seen Tweets, Facebook posts, invitations to join discussions on LinkedIn and invitations to attend Webinars practically every single day since the announcement was made. I get it! InfoPath will die in 2023. Guess what, I will retire before then.

The fact that I will be retired before then really won’t be considered as input to our decisions regarding InfoPath usage. As it stands today, we are making limited use of InfoPath in a fairly important project. In addition, we have plans to use InfoPath in a slightly larger, more important project later this year. I don’t see us changing these plans. Here’s why:

2023 – 2014 = 9 Years – Seriously, 9 years! That’s a long time. It’s not like “ding dong the witch is dead; click your shoes and go home.” The metaphorical Dorothy will be old enough to drink by the time she gets back to Kansas. We made an investment in InfoPath, and well before 2023, we will have achieved a return on that investment. That is what we are supposed to do isn’t it?

What’s on Other Roadmaps? – I realize that there are other ways to publish and process forms. Some of these alternatives are every bit as good as InfoPath, many are superior. If you made me choose a new forms platform today, I would probably start by taking a hard look at Nintex. We use Nintex’s workflow product and I imagine their form product works well too. That said, what’s Nintex’s plan for the future? Will Nintex Forms exist in 2023? Will the forms we produce with Nintex Forms today still run in 2023? Well, we really don’t know. In terms of vendors you can trust for the long haul, there really aren’t many. It’s not like they are untrustworthy, but what company can really say that they and their products will be here in 10 years? HP? Dell? BlackBerry (RIM)? Kodak? Go back 10 years and think about how those guys looked at the time and then consider how they look today. If you’ve been reading this blog, you might be surprised to see me say that I trust Microsoft, but when they say that they put something on a long, supported slide to the back door, I think we can believe them.

Now, let’s take a look at those scary emails. Here are some of the things they are warning me about and why I’m not concerned:

Support and Compatibility Questions Abound – Sure they do, but when haven’t they? Microsoft says that InfoPath 2013 will be the last version and support will end in April of 2023. As far as I know, they didn’t actually say that InfoPath 2013 and SharePoint 20nn will be compatible, but I’m not going to worry about that. Let’s assume that the next version of SharePoint is the 2016 version (I know they want to stop the “big release thing” but humor me). Chances are good that SharePoint 2016 will be compatible with InfoPath 2013. Chances are also good that we could run SharePoint 2016 until 2020 (we’re still running SharePoint 2010). Since we know that InfoPath is on a collision course with the trash bin in 2023, I’m pretty sure that my successors will have an alternative in mind before 2020. I don’t think they will be building any new InfoPath Forms in 2018-2020, but I think we can safely build them in 2014-2016 and still get that ROI.

No Certain Migration Path – In other words what if Microsoft creates a new Forms product and we can’t convert our InfoPath forms to that product. Well, that would be dumb. If Microsoft wants to play in the forms arena, they would be silly to alienate their previous forms customers. Still, since we know that Microsoft is capable of doing dumb things, I’ll accept this as a possibility. On the other hand, you have to ask: is there a way to migrate one forms product today to another forms product today? I am assuming that if we wanted to move to Nintex Forms today, we’d be recreating the functionality of our InfoPath forms on that platform. That’s OK. Also, that’s really the worst thing that can happen to us, that in 5-7 years we will have to recreate a form that probably will be due for a major update anyway.

A member of my team will be at the SharePoint Conference. We will hear about the future of InfoPath from the horse, and we will plan or adjust our plans accordingly. In the meantime, I’m not losing any sleep.

Mea Culpa

imageA couple of weeks ago I wrote about how we are using SharePoint to support a Holiday Door Decorating Contest in our office. Well, the doors are decorated and the votes are in. The winning door is shown at the right, and some lucky charity is going to receive a nice donation. Unfortunately, the SharePoint part wasn’t as simple as I had hoped. Although some people think others said this first, I’m going with the fact that Albert Einstein said:

“Everything Should Be Made as Simple as Possible, But Not Simpler”

In a developer’s twist on Einstein, I would say that “perhaps I was hoping for my task to be too simple.”

I made a newbie’s mistake. I made a mistake that I shouldn’t have made. I made a mistake that many people in the SharePoint community complain about people making. I took something out of a blog, pasted it into SharePoint without taking the time to fully understand it. To make matters worse, the person who wrote the blog had explained everything very well. I glossed over that section of his post because; well because A) I was in a hurry. B) I was thinking “yeah, yeah, I get this” and C) It was a door decorating contest! I mean it’s not like the site was calculating premium for a nuclear reactor (we actually do that, so…).

As I said in my earlier post, we don’t work with images very often. In fact, all of the other times that we have worked with images, we’ve just plopped a default picture library onto a site and dumped a bunch of JPG’s into them. Here’s a partial list of things I didn’t know about picture libraries:

The default columns (file size, picture, size, selection, etc.) activate themselves when you edit a view. So, unless you turn them each time you edit a view, they show up again and your neat and tidy view looks stupid.

The thumbnail is available as an optional column that you can show in your views (I mentioned in my earlier post that I didn’t know where this could be found and I used a solution to build the link to the small image).

The thumbnail image is linked to the view properties. I wanted to have a link from the thumbnail to a full sized image, preferably one that opens in a new window, so I ended up using the solution I mentioned in my previous post anyway.

The default view of a picture library doesn’t seem to preserve all the view settings. A picture library view can show details or thumbnails or be a filmstrip but the URL is the same. Some people saw the details view (that I included in my instructions) and some saw the thumbnail view that made no sense at all since it didn’t have the link to cast the vote.


As for that link, the one I built to start the workflow that “casts” the vote (a.k.a. the idea I copied from the blog) I only didn’t understand one thing, but it was a comically critical thing. The GUID of that workflow changes each time the workflow is published because the workflow is versioned. That makes sense, but because I didn’t know that, I was reacting to the error I got when clicking on the link as though it was an error in my workflow. You can imagine the frustration that ensued as I kept trying to correct a workflow that really had nothing wrong with it. As I write this, I’m trying to figure out which of my friends are just laughing and which are also thinking “it serves you right” while they laugh.

Let’s count the mistakes I made:

  1. I skimmed over a blog entry, modified a solution and stuffed it into my library.
  2. I didn’t fully investigate the attributes and behavior of the SharePoint feature (the picture library) that I was using.
  3. I published a solution without having a user other than me test it. If I had had one other person test this, I would have discovered the issue with the view because the person I pick on for testing saw the thumbnail view, not the details view.
  4. I decided that the nature of the business process I was working on wasn’t worthy of my full attention and / or best effort.

I am most disappointed in myself over number 4. Everything we do in SharePoint deserves our best effort. Every project that doesn’t represent our best effort contributes to the various negative perceptions people have about SharePoint. A fun project like this is a great way to show what SharePoint can do and I almost squandered that opportunity – ho ho ho!

Note: I didn’t compound my newbie mistake by complaining to the author of the blog. I went back and studied what he wrote, pulled my head out of my … the darkness and made the library work like it should have worked at the start.

Lightning Conductor Web Part

New Haven RailroadA while back, I suggested that I wasn’t worried about the changes that Microsoft was making / has made to SharePoint because the market would come to our rescue. The pronoun here is referring to those of us who had gotten used to wiring up Data View Web Parts and customizing pages in SharePoint Designer. The collective changes in SharePoint 2013 will make building those solutions a different exercise and one that brings us closer to programming than my staff wants to be. I’ll save until later the philosophical debate(s) around whether script-coding is programming; whether programming is evil and to be avoided or a necessary skill that should be required, and whether JavaScript is the official language of the new world order and we should all just suck-it-up. Today, I want to point out one vendor that has stepped into the role I outlined last October.

We have been previewing the Lightning Conductor Web Part from Lightning Tools and I am happy to report that it looks like it can easily help us provide one of the solutions that we used to turn to XSL to provide – the custom view of data from places our users don’t want to go. OK, it’s not that they don’t want to go there; they just don’t want to be on the road forever. The people using SharePoint in our company don’t really want to use SharePoint to conduct business; they want SharePoint to work for them. We made a promise to those people years ago when we (Information Services) chose SharePoint; we said that we would invest time and budget into making SharePoint easy to use. I take that promise very seriously. If I start telling people “you have to learn a lot more about SharePoint in order to have it help you with that task…” I’m practicing bait and switch. This new tool is pretty cool, and it appears to be a really good use of my budget. So, what does this tool actually do? Well, I can’t describe this any better than the Lightning Tools website, but here’s what we like about it:

Using the tool, we can quickly put a web part on a page that aggregates content from multiple libraries on the site, from within the site collection, from within multiple site collections or from across our entire farm. We can filter this content, we can sort this content and we can decide what columns to include. We can do all of this from within the browser, without opening SharePoint Designer and without even leaving the page that we want to put the web part on. Configuration is fast, and the results are immediate. And, if we screw it up, we can simply delete the part without having messed up the entire page.

For example, we will be able to gather policies, underwriting correspondence, inspection reports, engineering metrics, previous claims and presentations given to any one of our customers, on any page on our farm. These documents (someday will) all reside in the sites where they belong, but I (will be able to) pull them together – without waiting for a search to complete – wherever they might be useful. Parenthetical expressions aside, we already have content in multiple locations that needs to be aggregated to be more useful.

Of course there are a lot of other ways to accomplish these tasks. I have watched lots of people demonstrate various techniques at seminars and, inspired by those demos, I urged my team to learn these techniques. These days, I want my team building solutions. I want them to show off their content and information management prowess, not their syntax skills. The people on my team can imagine great solutions by merging their understanding of SharePoint, content management and the unique attributes of our crazy little business. If I can buy tools like this to let them quickly build what they can imagine, it seems like a good move.

I think we are moving forward with this purchase We just purchased this product! We had a few questions during our evaluation, and they were quickly answered by the responsive service group at Lightning Tools.

Let me dwell on that last thought for a second. I love it when a vendor lets you work with the actual technical support group during a sales evaluation. Not only did we get our answers, we were able to evaluate the support as well. BTW, the support is as good as the product.

The Lightning Conductor looks like a nice fit for us, but I’ll fill you in on our actual experience after we put it to use. Maybe I’ll get one of my team members to write about and I’ll get a week off – this is sounding better and better.

Accuracy Above All Else

imageThis is a modified cross-post of my AIIM blog, and although I rarely reuse this content, this week is different for two reasons. One, I am extremely busy and Saturday seems to have fallen out of my sleeve this morning. Two I had three separate discussions after publiishing the AIIM post, about new SharePoint solutions that are exciting and, in some ways terrifying. All three solutions involve gathering information into one location while linking in information from other locations and displaying a variety of back-end data related to the documents / topics in the SharePoint site. It sounds like we have begun to turn the corner on adoption (the exciting part), so what is my concern? What happens if the information I am providing turns out to be incorrect?

When we build information management solutions, we worry about a lot of things; we worry about capacity, we worry about bandwidth, we worry about search and findability and, increasingly we worry about usability. Today, I am starting to worry about accuracy. Why the sudden interest in accuracy? Actually, wouldn’t accuracy always have been at the top of the list in information management? I think the answer to the last question is “yes and no”, and I think understanding why it has been that way serves to answer the first question.

Separate and Accurate – When there was a sharp division between structured and unstructured data, accuracy was easy. Structured data is manipulated accurately by algorithms that are well understood and tested. If the algorithms need to change, the tests change and we march forward in lock-step. Unstructured data is what it is. A document’s contents might be accurate or inaccurate, but the document itself is just a container. Two things are happening that are changing this simple dynamic. One, we are working to combine structured and unstructured data. For example, we have a project on the drawing board that will let you see a policy document, the related reports about the insured (customer) that we have, and the premium and other numbers associated with that policy. Two, we are starting to derive structured data from the unstructured content. We have a number of projects where we are counting documents, measuring activity, charting progress; all based on the content and on metadata in a library. In each of these cases, we are relying on things beyond our control remaining accurate. Systems can change, and we could find ourselves hooked-up to the wrong data element. Procedures can change, and we could be counting activity that no longer holds the same meaning. In other words, there is now an additional degree of separation between the underlying data and the information we are presenting – that is always cause for concern.

But It’s On the Portal – There’s a TV commercial for an insurance company that has one character stating that “you can’t put anything on the Internet that isn’t true” a fact that she heard, on the Internet. Of course we all know that everything on the Internet isn’t true, but we expect the stuff on our company’s intranet to be true and we expect it to be accurate. This expectation, and the solid acceptance that the expected behavior will be delivered is what makes it so important to bridge those degrees of separation to maintain accuracy. We have to remain aware of the links that we establish, and we have to incorporate downstream usage in our change control process. It’s no longer good enough to fix our system, or to update our procedures; we have to know who uses our information and we have to keep them in the loop.

I started having this conversation with the constituents of the three upcoming projects. The good news is that, at least at a conceptual level, they understand the importance of accuracy in categorization, in definition and in calculating and rendering information from metadata. The bad news is that now that they want stuff in SharePoint, they want that stuff to be there now – I am suggesting that the process must be more involved and accordingly will occur at a slower pace. Well, it’s going to occur at a slower pace simply due to the fact that I have a small team, but that’s a different issue. Despite this initial frustration, I think that we are in a very good place. The people who want these projects completed are beginning to understand the fact that they too are information professionals, and they have indicated that they are willing to learn, willing to help and those are clearly good things. Now the challenge for my team is to prioritize multiple requests and keep everything moving at an acceptable pace – if you have to have a problem, that’s a good one to have.

Information Management 101

clip_image002When AIIM decided to increase their emphasis on people, and spread their net beyond Content Managers to attract Information Professionals, I was thrilled. It may be minor semantics, but ‘information’ is easy to understand and ‘content’ – well content makes me think of ingredients. Ingredients are sometimes managed, like when a factory produces bread, or shampoo or beer, but ingredients are just as likely to be thrown together when you and your 6-year-old bake a pretty good cake. Ingredients sound like the stuff hidden in the fine print. Information sounds important, it sounds vetted and validated and downright useful. I decided to launch a series of training sessions around the concepts of information management because we are involved in a wide variety of systems development projects right now, and many more lie ahead.

The first training session was short, in fact I finished 5 minutes ahead of my 45 minute goal, but that may have been the only thing that the audience enjoyed. The message was simple but not entirely appreciated. That we are all ‘information professionals’ sounds flattering, but pointing out what constitutes ‘professional behavior’ isn’t so appealing. One of the slides that wasn’t accepted well at all was the one titled “What’s Not a System”. From the thousands of things that aren’t systems, I chose four that cause the most problems for me:

Templates – Word, PowerPoint and even SharePoint all make great use of templates, but the idea that building a solution from a template is enough to insure consistency, reliability and usefulness is folly. Templates are like the recipes that guide our use of ingredients. We can change templates, ignore portions, delete portions or put the wrong things in the wrong place. Saying “I used the template” isn’t really the equivalent of saying “I followed the rules” – It’s more like saying “I had the rulebook open at the time.

Spreadsheets – I like to think about spreadsheets the way I think about sports cars. In the hands of the right people, they are both fantastic. If the people are unskilled, unwilling to follow the rules of the road, on the wrong road or trying to do the wrong thing, the end result will not be pretty. You can move furniture in a sports car, but… Spreadsheets are good at so many things, but pretending to be an automated system isn’t one of them. Spreadsheets are fragile, hard to debug, hard to document, way too easy to copy (and then confuse the copies), and way too easy to be made to look official. Even worse than when people try to use spreadsheets as a data processing system, is when they try to use them as a content management system. The simple row-column interface of a spreadsheet encourages people to store, track and categorize stuff the way the top left kitchen drawer attracts gadgets. When you go back and try to sort and filter your list though, you realize that ‘Cars’ are not ‘Automobiles’ are not ‘Chevys’ are not ‘Vehicles’ and so on and so on.

Folders – My target here wasn’t just the myriad file folders on the remaining share drives in use on our network. I also took aim at the SharePoint libraries that were created, from some of those file folders. Folders and unmanaged libraries are not systems, and they provide minimal (at best) support for managing information. Yes, folders do help us to organize information, but they can’t enforce rules, they can be moved, renamed, and they can hold so many things that don’t belong in them that they are the top left kitchen drawer. Folders are a good starting point, if you have better places to put the things that are in them and if you are willing to throw some things away.

People – Actually, in a failed attempt to be cute, I said “You” are not a system. I went on to explain that people are pretty unreliable when it comes to managing information. People make the rules, but they also forget the rules they make. People take short-cuts through the processes they define, ignore the procedures or convince themselves that rules and procedures were meant to apply to “other” people. The most dangerous thing about people is that they think, and when they think, they convince themselves that changes should be made and then they make them. Systems can change, but only when the changes are confirmed to be appropriate by everyone who will be affected and then only under the control of the same process by which the systems were built. I wish people wanted to manage information the way some want to play football, then I could try something like this.