Going – Going –

clip_image002They’re not gone yet, but by the middle of 2013, our network shared folders become read-only. That was the message delivered a few days ago to a group representing every functioning department that hasn’t already moved their content onto SharePoint. Tough love? No, reality. Let me be clear about my motives, I am not advocating SharePoint, I’m advocating the value of information. Shared network folders have been considered to be among the best things I ever suggested since I first mapped a directory on our NetWare server to K: in 1988. However, not only are they less capable than SharePoint, shared folders have become dysfunctional. They are hard to navigate unless particularly well organized (most are not). Also, because searching and sorting are done on the client, they are pathetically slow for remote workers. So I find myself having to put an end to the era I started 25 years ago.

Sometimes, in order to move forward, you have to make it impossible to move backward or even to stay in the same place. I didn’t invent this concept, I read about others doing it, and I held it in reserve hoping that education and cooperation would suffice. They won’t. I realize that something I wrote about a long time ago, something I learned as an undergraduate in chemistry (of all things) is at work – activation energy. Chemical reactions have an activation energy which if not established, will not proceed. Information management has the same requirement. As long as people have the ability to keep tossing stuff onto the K: drive, they will continue to do just that. The incentives and promises of easy access, findability, sharing, remote and / or mobile connections will only inspire a limited few people to embrace ECM. For the people who create more content than they consume, manage the content others create, or primarily consume the content stored in their own silo, ECM offers little benefit. For them, ECM is “the stuff I have to do to make someone else’s job easier” and oftentimes, the “someone else” is a future employee. Once again, I find myself facing the unenviable task of changing behavior.

I wrote about that task very early on this blog, and I mentioned that my boss had advised me that changing behavior wasn’t like driving a speedboat; it was more like driving an aircraft carrier. He told me to get comfortable with only being able to turn one or two degrees at a time. That was great advice, but when it comes to changing the way people work with content, sometimes I feel like I’m driving Africa, like I am moving at the speed of the tectonic plates. The problem that I face is that if I don’t start making 2-3° changes soon and 5-7° changes in the not too distant future, some of the ships in my carrier task group are going to run aground. Still, there are two important nuances contained in the opening sentence above. One is the fact that we are targeting mid-year. The second fact is that we are not eliminating the K: drive.

Mid-year is an important target both for my team and the people we support. From my team’s point of view, we know that if it doesn’t happen by the end of June, it won’t happen. We will be delayed by vacations and then we will roll into the 4th quarter “busy season” and it will be year-end and 2014 and then we will be starting over. For the people in those operating departments, mid-year means that they have 4-5 months to figure out what they are going to do. During that time, we can help them define the sites and libraries that they need, establish basic metadata and get the process underway.

Making the K: drive read-only has numerous benefits. Nothing will be lost, nothing has to be moved right now and we don’t have to solve every problem before the end of June. Before the end of June, we have to have a place for all the stuff we are likely to create in July and August. We have to have a plan for moving the most important files from the K: drive into SharePoint, and we have to convince people that some of the files on the K: drive are garbage. That’s why we aren’t simply moving the entire contents of K: into SharePoint! Read-only means if they need a file, they can get it, but if they need to edit it, they have to figure out where it belongs and they have to set the basic metadata. If nothing else, everything we create after 6/30/2013 will be easier to work with. That sounds like a workable plan that offers enough benefit for now. Stay tuned, I’ll report on our progress.

This Has to Go – No Wait

imageOne of the amazing things you will frequently witness during the process of automating something is the transition from unconcerned consumer to connoisseur. People who were perfectly happy tossing a bunch of files into a folder on the K: drive suddenly become concerned with the induction, retention and disposition process. People, who were comfortable pawing their way through a bunch of similarly named documents, worry about the effectiveness of versioning. People who were un-phased by using email as the transport layer for collaboration question the reliability of alerts and people who have probably recreated content they couldn’t find obsess over the accuracy of search. The good news is that no matter how unfounded their concerns, the answer with SharePoint is usually “yes you can”, “yes it does” or “we can make that work the way you want!” Our most recent struggle with these questions revolved around the destruction of documents.

There are at least two points in the document life-cycle where a destruction policy is necessary. One is at the end of the retention period, when the document no longer has value. The other is during the creation of major version. We want to keep the major version, but we may need to delete the drafts and / or the prior version. This makes perfect sense, and is easy to understand for any given document, but it becomes difficult when you start thinking about hundreds of documents. SharePoint provides numerous tools to help us through this process, but it also complicates our policy making process. Library settings, alerts and workflows in SharePoint, enable us to implement retention, destruction, notification and even legal holds on content, but they apply to libraries. This means that we have to extend our analysis up-front to the decisions that lie in the future.

“Will all the documents in this library be disposed of according to the same policy?

If the answer is “no” then you need a different library.

The other problem we have to deal with is the standard double-edged sword that is SharePoint, namely that we tend to mix collaboration and content management in the same place. I truly love the fact that we can support a collaborative, iterative content development process inside the library where complex content will be stored. It reminds me of the new construction within the industry we insure, where fabrication and storage facilities have been built proximate to the emerging power plant. When completed, the construction remnants will be removed, just like the draft documents and many of the data sets and spreadsheets that supported the analysis and decision making along the way. However, just like with the documents themselves, these are decisions that have to be made up-front. How many drafts do we keep? How long do we keep them? Who decides when a draft document is published as a major version? How many major versions do we keep?

The real problem isn’t with SharePoint, and, to avoid the faux pas revealed in last week’s post, I will add that it isn’t really a problem. As we work to change the collective behavior of people, we are asking them to give up the control they perceive to be inherent in the current process. People have a sense, however false, that they are in control of the contents of a shared folder, and when we ask them to yield that control to workflows and policies, and they instinctively push back. The fact that I can point to 60,000 documents under a root-level shared folder as indication that the current process is out of control is irrelevant. The thought process says “I can take control of this content at any point” and we don’t see that capability in SharePoint. I can modify the parameters that control the automatic actions, but that is buried beneath the surface; I can’t touch those parameters as easily as I can the documents themselves. In addition, the decisions we make affect all the documents in a library. That is a great benefit, but it’s also a scary proposition.


CT River from North End Bridge

Whenever I give a presentation, I like to think that I have a huge advantage over Microsoft employees and SharePoint consultants. In the first case, I can freely talk about the things SharePoint doesn’t do well, and the things I know it could do better. In the second case, I can talk about the things I don’t do well – I don’t have to try to impress you with my prowess so that you might hire me. I do sometimes worry that if my boss actually reads this blog, he might wonder “why the heck did I leave this bozo in charge,” but I recall that he was the first to identify the therapeutic effect of blogging, so he won’t hold it against me when I air the laundry.

The one consistent truth in systems development, ECM solution development or just about any development process that involves people is that things go off the rails from time to time. I have been building solutions that change the way people work for 35 years, and I have seen just about every reason why solutions fail, from sabotage to resistance at a level that would confound the Borg Queen. I’ve seen solutions that are as good as the technology would permit, as good as the budget would permit, and/or as good as the deadline would permit, but were still not good enough. I have also seen solutions fail because the infrastructure they were built upon evolved in a different direction. None of this is happening today in our case, because our system isn’t failing, and my experience tells me that it isn’t about to fail. Our system is just waiting to be “the way we do things.” Right now, we are still fighting the memory of “the way we’ve always done things” and that is one tough battle. You may have noticed that I haven’t described what went wrong to inspire this blog entry – it doesn’t matter. In fact, if I tell you the questions we were asking as this week ended, I’m sure you will recognize them:

  • What is so hard about this?
  • Why didn’t they tell us they were having this issue?
  • What made them think that (the workaround) would help?

Then we asked the most important question of all – why aren’t they learning how to work with this process?

When we build information management systems, we like to think that we are improving a business process. That’s a comfortable way of looking at development because it creates an abstraction layer between us and the people involved. Considering the business process allows us to ignore history; we don’t care why they “used to do it this way”, we just assume it was because they didn’t have our solution. We aggravate this situation by assuming that everyone wants every process to be as efficient as it can be. We know better. In reality, we are trying to change the behavior of a group of people, and in general, people don’t like change. When things start to go wrong though, we tend to skip past the abstraction layer and zoom in on the people involved. One of the common complaints that I hear about people, when I talk to others in this industry, is the so-called generation effect on technology adoption. People will talk about how much more willing younger people are to embrace new technology than the old guard employees they are being trained to replace. While there certainly are generational differences when it comes to affinity for technology, we also have to consider that new (younger) employees are learning the process and our solution at the same time. In the absence of a historic regimen, any solution that gets the job done has a certain appeal.

As we consider our final question, we have to consider the answers our users aren’t giving us. Nobody is going to just come right out and say “I don’t want my job to change,” but they don’t want their job to change. We can take the Borg Queen’s approach, and remind them that resistance is futile and that our solution is here to stay, or we can step back and look at the new solution from their point of view. How did the system we just build impact this person’s job? We tend to focus on the things we made easier, but did we make anything more difficult? Resistance is futile; in time, people will adapt their behavior to accommodate the requirements of the new process, but what if we missed something? What if that new process could actually be a little bit better? When we see people pushing back, instead of embracing our solutions, we need to try and discover the underlying reason – maybe it’s something we can fix.

Where Is Your Content?

Last Tuesday, I had the privilege to attend an event around the topic of Email Archiving put on by the AIIM New England Chapter. Bill Tolson, Director of Product Marketing for Iron Mountain, and Brett Burney, Lawyer, Burney and Associates talked about the various reasons email archiving is important, best practices around the subject, and as you might expect, the scary realities of eDiscovery requests. There were many interesting and valuable tid-bits offered up during the presentations, but the one statement that stuck in my head was when Bill Tolson said “you have to know what to throw away!
When we build out SharePoint to support ECM goals, we are usually focused on creating a place to put content. I have heard that there are some enlightened ones among us, who add features like retention policies and destruction dates, but that isn’t the kind of “throwing away” I am talking about today. I am thinking about the content that went into SharePoint – was it moved? Was it copied? Does it still exist? Do you know the answer? Well, if you don’t know the answer, then you need to be prepared to search. Searching leads to finding things, things that you find have to be reviewed and things that aren’t privileged need to be served up when a discover request arrives. I have only been involved in one eDiscovery request, but I learned what Brett Burney meant when he said “the answer to what has to be provided in response to a discovery request is – it depends.” Here’s one example:

Among the places we had to search for instances of three specific text strings, were the folders holding old flat-file database tables. In one folder was a table that included one of the search phrases. In a folder below that, was a collection of 10 sorted versions of that table. The sub-folder was called “TempReport” the sorted files all included ‘temp’ in the file name, and we could show that the files held records identical to those in the table, to be used for reporting. None of that mattered, we were ultimately required to convert each file into a spreadsheet, describe it in detail and provide it in response to the request.

When I migrate files from a shared folder into SharePoint, I copy them. After they arrive in SharePoint, I rename the folder structure to indicate that the files were uploaded to SharePoint, and I make the folders read-only. After a few backup cycles, once I can convince the users that I have everything and I shouldn’t be able to lose it, I delete the files from the shared folder. That process works well, but I am in charge of the whole process. What happens when individual employees move content into SharePoint? When my coworkers move email, or email attachments into SharePoint from Outlook (see previous post), do they also move the email into a folder in Outlook? Do they put the content in SharePoint and then send a link around to others, or do they move it after forwarding the email to others? Do we have these answers? Do we have a policy that will allow us to search only SharePoint for those documents? I think we do, but I know that the answer is “it depends.” It depends on the judge at the time the request is made. The only way you can know that it doesn’t depend on anything, is when you know that you threw it away. (yes, I know there are times you can’t throw things away, but I do impose an 800 word limit on myself, so…)

Those are easy examples to wrap our heads around, but they are far from the only things we need to consider. For example, in our latest SharePoint project, we are managing the production, review and disposition of engineering inspection reports. During the creation and review process, we keep minor versions. During the distribution process, we create a major version, delete the minor versions and move a PDF copy of the final version to a Records Center. That’s not just our policy; it is a process that is executed by SharePoint workflows – it is also the first time we’ve automated that process.
After Bill and Brett were done speaking at the AIIM NE event, I was talking to a few other members of the audience. The conclusion we came to is that many of us are only providing the means to proper content management. Even governance rules and strict policies can only go so far. There is no real way to prevent people from making and keeping copies of company records. Solving that problem requires changing behavior, and changing behavior requires education and communication.

My Digital Document – R.I.P.

Recent information from AIIM and others indicates that high percentages of digitally created documents are being scanned into our document repositories – that is sad for so many reasons.

Quality – The overwhelming reason to keep a digital document in digital form is to preserve the quality of the original. Scanned documents, regardless of the scanner used, do not look as good as the original document unless you invest serious time and energy to re-mastering the resulting image. On the other hand, scanned documents, that are often skewed, offset on the page and littered with gray-scale debris, can look a lot worse without any effort at all.

Size – The second biggest reason to avoid scanning documents that were born digital is file size. This is only in second place if we’re using integers, if we included two decimals, I’d rank this at 1.01. Consider that a 35 page PowerPoint presentation with a healthy mix of text and graphics is a 1.4 MB pptx file. That same presentation, saved as PDF from within PowerPoint is a 3.9 MB pdf file. That same presentation, printed and then scanned to PDF (using maximum compression) requires 21 MB for storage. That’s over 5 times the size of the saved-to-PDF version and over 15 times the size of the original file! More important, it’s over twice the size of the most common email file size limit of 10 MB. That translates directly to slower email send times (if you could send it) and slower downloads from SharePoint.

Utility – Clocking in at 1.02 on our scale of worst reasons to scan content that was born digital is the usefulness or, in the case of the scanned document, uselessness of the result. If we go back to the presentation in the above example, let’s consider what we can do with the PowerPoint version. First, we can edit it. That means we can extend its life by updating it, reusing it, and repurposing the content . That also means that we can share certain good slides with others and save them the time to prepare those slides. Second, we can search for the words in the presentation. We can search our hard drives, network drives and we can search SharePoint. And, if we can search, that means others can search too.

Now let’s look at what we can do with the save-as-PDF version, hint: almost as much. We can’t actually edit the presentation but we could correct a few spelling errors if we had to, say if I was the original author. We can search for individual words in the PDF file, and in SharePoint. We can repurpose the text and the graphics as well. Acrobat holds these artifacts in their original form so we can copy text out of the PDF and we can copy graphics as JPGs. If we save-as-PDF/A, we can also extend the useful life of that presentation or keep it for archive purposes.

What about that scanned-to-PDF version? Nothing, zip, nada, we can’t do anything with that puppy. We can’t search, we can’t reuse, we can’t edit, we can’t send it, and even if we could, we would likely be embarrassed by the quality. Yeah, yeah, I know, Acrobat includes OCR but the concept of performing OCR on a scanned copy of a born-digital document is, well, it hurts my head.

If this practice is so bad, why does it happen? Usually, the reason is time; it’s faster for the person who has to deal with the document(s) to simply scan them. Of course, that person isn’t considering the time the other people will waste if they need the content out of those documents. Other reasons I hear include “I couldn’t find the originals” and “…is a composite presentation of multiple original documents”. These conditions are the fault of the authors. The solution is easy, make that material easy to find. Better yet, save your originals (in original format or as PDF) and keep them in a Document Library called Source Material. Of course, sometimes, composite documents are not all born digital. Even in this case, you are better off adding digital PDFs to graphic PDF’s, at least that’s heading in the right direction.

How about for 2010, we steal a phrase from Vegas, and agree that: “What’s born digital stays digital

Forecast for Microsoft – Looks Good to Me!

This is the blog post I’ve been dreading, the one immediately following the SharePoint Conference. Unfortunately, I wasn’t able to attend SPC09. I could still blog about SharePoint 2010 though, my friends, real and virtual, kept me informed during the conference to the point that I feel like I was there. Still, blogging about SP2010 should be their privilege.

Fortunately, the New York Times, aided by Apple and Google, threw this blog a life preserver. They published an article in which they drone on about how Microsoft missed the boat, fell behind the curve and is moving too slow. My SharePoint side wants to yell “tell that to the 7,000 plus folks in Vegas”. My cynical side wants to point out that this is the NY Times criticizing Microsoft for being too slow to respond to emerging technology. I’ll give you a few minutes to lookup synonyms for “irony” and then I’m going to ask you to think about the real world around you; perhaps you will agree that, if anything, Microsoft is moving too fast. OK, not too fast, but you can agree that most people haven’t begun to fully use what Microsoft, and particularly SharePoint has to offer today.

If Microsoft were really being left in the dust by technology craving consumers, why do they have to “announce” that SharePoint 2010 won’t work with IE6? Why do they have to caution people about trying to use Office 2003 with SharePoint 2010? Why am I barraged with email and cold calls from vendors wanting to help me “prepare for Exchange 2007”? Seriously, look around your home and your office and ask “how fast is this marketplace really moving?” Yes, I know Macs are cool and my daughter’s iPhone makes my WinMo-based Smartphone look pathetic and millions of people are using the on-going beta of Google Apps. But, others are upgrading their Fax machines, plugging along on XP and I drive past hundreds people in New England toll plazas who still don’t have an EZ-Pass! The real world is trying to get some work done and however much it pains those of us in IT, most of them are happy with yesterday’s technology.

Some of my colleagues will return from Vegas and begin blogging, authoring and preparing for the demand for expertise in SP 2010 – those would be the consultants. Most of my peers will be drafting a business case for SP 2010 in their free time while still trying to get their users to actually adopt the features in MOSS 2007. Oh, and to my consultant connections, you might want to practice the phrase “this would work better if you upgraded”; I think it’s in your future.

We make excellent use of SharePoint in our company but I could continue expanding SP’s presence without upgrading well into the future. We will upgrade early to SharePoint 2010. We’re small, the product is covered by Software Assurance and we have an awesome network admin who can’t wait to move us forward. My challenge during that process will be to first assure people that “nothing is really changing” with respect to our daily business. Then, I will point out individual new features to those people who are struggling with weaknesses in the current version. Then we will rebuild a few sites and build a few new ones to highlight the true benefit – I’m going to focus on the expanded services because, let’s face it, Excel still rules the office.