Lightning Conductor Web Part

New Haven RailroadA while back, I suggested that I wasn’t worried about the changes that Microsoft was making / has made to SharePoint because the market would come to our rescue. The pronoun here is referring to those of us who had gotten used to wiring up Data View Web Parts and customizing pages in SharePoint Designer. The collective changes in SharePoint 2013 will make building those solutions a different exercise and one that brings us closer to programming than my staff wants to be. I’ll save until later the philosophical debate(s) around whether script-coding is programming; whether programming is evil and to be avoided or a necessary skill that should be required, and whether JavaScript is the official language of the new world order and we should all just suck-it-up. Today, I want to point out one vendor that has stepped into the role I outlined last October.

We have been previewing the Lightning Conductor Web Part from Lightning Tools and I am happy to report that it looks like it can easily help us provide one of the solutions that we used to turn to XSL to provide – the custom view of data from places our users don’t want to go. OK, it’s not that they don’t want to go there; they just don’t want to be on the road forever. The people using SharePoint in our company don’t really want to use SharePoint to conduct business; they want SharePoint to work for them. We made a promise to those people years ago when we (Information Services) chose SharePoint; we said that we would invest time and budget into making SharePoint easy to use. I take that promise very seriously. If I start telling people “you have to learn a lot more about SharePoint in order to have it help you with that task…” I’m practicing bait and switch. This new tool is pretty cool, and it appears to be a really good use of my budget. So, what does this tool actually do? Well, I can’t describe this any better than the Lightning Tools website, but here’s what we like about it:

Using the tool, we can quickly put a web part on a page that aggregates content from multiple libraries on the site, from within the site collection, from within multiple site collections or from across our entire farm. We can filter this content, we can sort this content and we can decide what columns to include. We can do all of this from within the browser, without opening SharePoint Designer and without even leaving the page that we want to put the web part on. Configuration is fast, and the results are immediate. And, if we screw it up, we can simply delete the part without having messed up the entire page.

For example, we will be able to gather policies, underwriting correspondence, inspection reports, engineering metrics, previous claims and presentations given to any one of our customers, on any page on our farm. These documents (someday will) all reside in the sites where they belong, but I (will be able to) pull them together – without waiting for a search to complete – wherever they might be useful. Parenthetical expressions aside, we already have content in multiple locations that needs to be aggregated to be more useful.

Of course there are a lot of other ways to accomplish these tasks. I have watched lots of people demonstrate various techniques at seminars and, inspired by those demos, I urged my team to learn these techniques. These days, I want my team building solutions. I want them to show off their content and information management prowess, not their syntax skills. The people on my team can imagine great solutions by merging their understanding of SharePoint, content management and the unique attributes of our crazy little business. If I can buy tools like this to let them quickly build what they can imagine, it seems like a good move.

I think we are moving forward with this purchase We just purchased this product! We had a few questions during our evaluation, and they were quickly answered by the responsive service group at Lightning Tools.

Let me dwell on that last thought for a second. I love it when a vendor lets you work with the actual technical support group during a sales evaluation. Not only did we get our answers, we were able to evaluate the support as well. BTW, the support is as good as the product.

The Lightning Conductor looks like a nice fit for us, but I’ll fill you in on our actual experience after we put it to use. Maybe I’ll get one of my team members to write about and I’ll get a week off – this is sounding better and better.

SharePoint FM

clip_image002I spent my freshman year at the University of Georgia (go Bulldogs) and thanks to some friends on my dorm floor, I was able to visit some beautiful parts of that state. Once while driving around some desolate south-Georgia back roads, we stumbled across a Pirate game on the radio. I recognized KDKA AM 1020 Pittsburgh immediately by the announcer’s voice. We could hear that game in Georgia due to the ability for AM radio waves to skip across the planet, bouncing off the Ionosphere. FM radio waves, due to the different way they propagate, are limited to line-of-site reception. Hmm, if this is an analogy to SharePoint, wouldn’t I want to be AM? No; we chose an FM model because it’s more narrowly focused, but capable of delivering a higher quality signal (think classical music vs. political talking heads). If we extend this analogy just a little closer to the breaking point, we like operating in a narrow band of the “development spectrum” within SharePoint so we can:

Focus on Quality – This is very important to my team. We don’t want to simply be a company that uses SharePoint; we want to implement Content Management in SharePoint. That requires adherence to the standards and best practices associated with ECM, not just the creation of a bunch of libraries. In addition, we want to improve the business process associated with the creation, use and distribution of the information that we store in SharePoint. In the current project that I’m working on, I have worked with the owner of the process to build a better way of entering data, segregating data and presenting data for review. In the course of defining the requirements, we are expanding the use case from one where we keep track of stuff to one where we keep track, manage, analyze and share this stuff with our coworkers and our customers.

Stay close to home – We keep our solutions close to “out-of-the-box”, because we want our users to be able to understand, combine, extend and reuse the work we’ve done. We also don’t want to invest too heavily in development because we have better things to be developing. No offense to ECM, but out-of-the-box SharePoint can look pretty good, and if it’s easy to use and satisfies the customers, we don’t need to make it overly snazzy. Plus, I keep coming back to the notion that, in using SharePoint, people learn how to use SharePoint, and that knowledge is cumulative. I have also found that my coworkers are more interested in how the solution works, than appearance. I think there’s a fine line between adding functionality to have the page look tricked out and modern, and adding functionality that improves the user experience – we’re working hard to find, but not cross that line.

Be a wee bit anal – I know that some of you might be asking “why not just let your users develop their own solutions from the ground up?” The answer is simple and a bit selfish; I don’t want to join some friends of mine who have been left with half-finished or half-baked SharePoint solutions when some department level guru decides to leave. I want our solutions to be documented. I want to know what needs to change if we move the solution to a new server, or to our Internet-facing farm, and I want to be able to maintain the “systems” that our employees have come to rely upon. I also want our solutions to follow some standards. For example, where we have managed metadata in use, I don’t want to see a solution that is using a Choice column that has (at the moment) the same information as what is in the term store.

The system that I mentioned above organizes data from a series of inspection reports and follow-up meetings in three related lists. In order to review that data, we built a Web Part Page that contains about 5 Data View Web Parts to pull the related bits from those lists and present them in a useful format. To setup the page, we pass a bunch of parameters in with the URL. For example, based on source parameters, the page can be grouped by Facility, or by Engineer, or by Category. We construct those URLs in a workflow and store them with each item in the master list so they can be exposed as links in a wide variety of views. The Facility link, for example will configure the page to be grouped by Facility. We could have just built a second DVWP page to drive the sort and filter process and then build the URL on-the-fly, but that would put us in maintenance mode. SharePoint Views look pretty good, and many of our users can build them. Letting them add the URLs to their own views, lets them extend our solution. It’s still a line-of-sight system, but it has a little better reach.

FDMI

clip_image002I began driving in the era when Interstate highways were beginning to replace the convoluted network of US and State highways interconnected by local roads. One of the first major bits of Interstate to open up in western PA was I-79 between Bridgeville and Washington, PA a.k.a. “little Washington” so you didn’t confuse it with Washington, D.C. I-79 was an alternative to, but did not replace US Rt-19. Unfortunately for me, my father stood by Rt-19 as his first choice for any destination between our house and little Washington. I can remember his arguments, which varied between: “Rt-19 is actually shorter” to “by the time you deal with getting to and from the ramps, the highway isn’t that much faster” – I call results like that a Failure Due to Marginal Improvement (FDMI).

I have recently been reminded of those conversations with my dad, as I try to replace an Excel spreadsheet with a series of SharePoint lists. The issue is that we aren’t talking about replicating data; we are talking about changing, and hopefully improving the process that the data is associated with. The question is, are we doing enough.

The spreadsheet works – The current “system” in Excel doesn’t work well, but has the advantage that people know how it works. This is often the reason why people want to just move what they have into SharePoint, the thought is that they will still understand it. When we are replacing a list, we are often changing the way items are created, maintained and viewed. The result should be a better overall experience, but will it be good enough to offset the change?

Several weeks ago, Marc Anderson raised the question of whether form or function was more important. I suggested that the form-function ratio should never be less than 4:3, and this little project is an example why I think that is true. SharePoint lists are about as close to a database solution as you can get without invoking SQL, but we’re still talking rows and columns, and rows and columns are Excel’s forte. When you start moving Excel data into SharePoint, you create one or two (in this case three) Custom Lists. If you show the user an All Items view of their data in SharePoint, it’s going to look worse than it did in Excel. SharePoint doesn’t use its screen real estate as efficiently as Excel, and people who like Excel will pick up on that immediately.

The Excel-based solution I am replacing is a collection of observations and recommendations. Each row has details about the observations on the left, a huge column of text containing the observations in the center, and a series of recommendations on the right side of an A-X layout. The text is nearly unreadable and it is hard to stay on the same row as you scroll right to left, but the process is easy to understand. Read the details, read the text, write the recommendation.

clip_image004
Moving to a data view web part of items, with options to expose the details in a second DVWP, the observations in another and the previous recommendations in a third, give us a composite view of each item in an easy to read format.

clip_image006
I am planning to add an entry part to write the new recommendations, but that can wait. What we have so far is based on the initial discussion, and it’s time to check-in and show the results.

I’ve got one more concern as I get ready to touch base with the future owner of this solution. Since the stuff isn’t in the same “row” (it’s not even in the same list) he needs to have an increased level of trust in the underlying process. Sometimes it’s hard for people to accept that everything on that page is everything there is. Again, you gain that confidence in the spreadsheet visually; it’s not a pretty sight, but you can see it. I plan to walk the new owner through the process of building an entry from the ground up. By the time we come back to the composite page, he will know exactly what he should see. I’ve made that trip eight times, so I know it’s going to be a sweet ride. In addition to making it work, the user experience needs to be clearly superior to Excel if we’re going to avoid FDMI.

The Market Will Save Us

clip_image002I just finished reading a blog post by Sahil Malik which refers to a blog post by Marc Anderson and other posts and comments on the subject of the soon-to-be-gone-but-not-forgotten design view in SharePoint 2013. I have followed this story from the beginning when Marc was hinting that the design view was missing, up to and now well beyond, the seemingly official death sentence that was published by the Microsoft SharePoint Team on their blog. That the design view is going, going, gone seems certain at this point, but whether that will result in a wave of power users learning the art and craft of development or just skyrocketing demand for existing developers is not so clear. I can only say with clarity what will happen in my little world – nothing.

I should rephrase that “nothing will happen right away” and by “nothing” I mean SharePoint 2013. It won’t land here at least until this time next year because activity is underway that relies on the work that members of my team are performing in SharePoint Designer, using design view. Those projects will be completed and an annual event that depends on some custom Data View Web Parts that we assemble will be supported on SharePoint 2010 as it has for the previous two years. Switching canoes in the middle of this stream isn’t appealing and it won’t happen. This isn’t going to hurt Microsoft, (unless SharePoint 2013 marketing success depends on posts in this blog) because we have SharePoint under Software Assurance.

While people are lining up to prognosticate as to which way companies will go (developer of just plain SharePoint) I have faith that good old marketplace economics will save the day. Somebody will build a tool or an add-on tool for something like Dreamweaver, to once again empower the end-users and designers to push SharePoint beyond the limits of its dumpy box. If that doesn’t happen, then other market forces will bear down on the situation and other platforms may benefit from this decision. Again, I can’t speak for the market, or the broad content management / collaboration community, but if I have to turn developers loose on an ECM solution, I’m going to take a hard look at Nuxeo before I start buying more copies of Visual Studio.

Of course, I probably don’t even have to go that far. Before watching Marc Anderson deftly build a Data View Web Part using j-this, j-that and SPServices, I was using Ruby and Dreamweaver to put the information I wanted in a Web Part. I certainly prefer the approach Marc taught my crew, but I can always revert to other options. In fact, I may not have to revert to Ruby; much of what we display in Data View Web Parts can be rendered by SQL Server Reporting Services, another technology my crew is adept at using. Then again, the market may save the day in other ways. When I was using Ruby, I coupled it with Dreamweaver because someone had written a Dreamweaver extension for working with RHTML. I’m willing to bet that there are way more SharePoint power users than there are Ruby developers so I wouldn’t be surprised to see a solution become available for building the kind of solutions that we continue to need in SharePoint from a different development starting point.

Microsoft may want SharePoint to look like SharePoint, and as I have said before on this blog, I support that to a degree. The more SharePoint solutions share a common look and feel, the easier it is for people using SharePoint to move from one solution to another. Still, sometimes what you need isn’t in the box. When intelligent and enterprising people are confronted by the absence of the things they need, they find them elsewhere or they figure out how to build them. When enough people start building things, vendors bring tools to the market. Maybe Stanley will come out with the Fat Max Editor for SharePoint – I would buy that in a heartbeat.

Head Bone Connected to the Neck Bone

imageRemember that song, Dem Bones? All those connections, from toes to head and back again; it was a fun way to think about our skeleton, and as long as I don’t hear my doctor humming it, it’s a fond memory. As it turns out, connections have continued to play an important role in my life. With respect to my skeleton, it’s the connections that caused the problems that sent me to physical therapy. With respect to people, it’s the connections that allow me to enjoy life, and get my job done. With respect to SharePoint, well, sometimes connections in SharePoint help us do remarkable things, and sometimes they send me looking for therapy.

Recently, we have been experimenting with the various ways we can connect SharePoint to the structured data in our SQL Server. There are two primary reasons for wanting to exploit this capability. One, we can enhance the value of our content-centric applications on SharePoint when we can add bits of data to the picture. Two, sometimes, processes in SharePoint grow until they reach the threshold of being an application; nudging them over that threshold saves us the effort of developing another application. Here are two examples of the connections we are working on right now, and why:

One of our most complex content-centric applications is the solution that helps our engineers manage the documents associated with their loss-control inspections. The solution relies on a few supporting custom lists. For example, one list includes, among other things, the URL of the document library in the customer’s site on our Internet-facing server so we know where to put their copy of the report. Recently, a manager in our underwriting department asked if the underwriters can be notified when inspection reports are distributed. Of course, they could set alerts on these libraries, but they only want to know when one document reaches its final stage in the process. Since there is a workflow running, and a lookup-list, we could easily add the underwriter to that list and shoot him an email. Then again, underwriter assignments change, and those assignments are already being maintained in our policy rating system. Instead of manually duplicating that information in the look-up list, the workflow, with a little help from a function library that we are testing, can look up the underwriter in the rating system. If this library doesn’t work, the workflow can look up the underwriter in an External List rendered from the table in question; that would be duplicate data but not duplicate work, SharePoint would be doing the heavy lifting.

The second example features one of our earliest (and ongoing) workflow driven success stories, a time tracking solution that our attorneys use. Our first step was to automate the time entry, which relies on a series of related custom lists. For example, we use a custom list to let the attorneys identify which projects are active. By toggling that status, the pick lists in the entry forms are modified to only show valid choices. The evolution of this solution included a hand-crafted set of Data View Webparts to let the attorneys enter their time from their iPads and integrated SSRS reports to provide the summary data to the accountants. Of course, the accountants need to turn that summary data into journal entries. The time-honored method has been to read the report and type the entries into our General Ledger system. Starting next month, SharePoint will be making those journal entries, courtesy of some fancy script work by Marc Anderson, aided by the woman on my team who has guided this solution through its various stages. The last stage was not without a certain amount of drama. We ran into trouble trying to update an External List from the script. Perhaps the capability doesn’t exist, perhaps the feature is broken, perhaps we still didn’t manage to get the permissions right. However we can update the External List from a workflow. So, Marc put his journal entries into a SharePoint list and a workflow puts them into SQL Server via the External List. It sounds like a kludge, but it works well and it doesn’t require any additional input during the process. I do hope that at some point, Marc describes the details behind his middle-tier magic over on his blog; I’ll leave it at “it works, and we love it!

Web Parts, Data sources, External Lists, related SharePoint lists; it’s not as easy as “the toe bone connecting to the foot bone”, but it’s actually not that hard. As I mentioned last week, we keep these solutions moving forward by taking small steps – sometimes we fail and learn, but sometimes we succeed.