Teamwork – Meet the Team

clip_image002I will first introduce myself, as it is not Dan who is writing this blog entry today. My name is Doreen Jacius and I work for Dan. My “real” title is Systems Analyst, but as most of you know from following Dan, I, like Dan participate in many parts of the development efforts within the IT department. Dan has mentioned me (without name) MANY times in earlier blog entries and now it is my turn to attempt a blog entry. I stress the word attempt since it is not something I have ever done before! I have been coaxed into a public speaking situation one time in my life and that is the extent of my experience in this domain. Keeping that in mind, here is my story:

Earlier this week my 11 year old daughter was getting fitted for hearing aids and struggling with how the world sounds to her now. She was very frustrated and the look of agony on her face made me question if we were doing the right thing. When her audiologist told us that it is going to take a lot of time for her brain to get used to processing new sounds that she has not heard in many years, it made me think that is how I felt when I first came back to work and started learning SharePoint. I had never seen SharePoint before and it looked much different than a Lotus Notes environment I had left 10 years earlier. For the first few months of learning SharePoint I was questioning whether coming back to work was really the right choice. I was frustrated that I could not immediately figure out the difference between a list and a library. I then became frustrated with setting up workflows and trying to learn the difference between list, reusable and site workflows. I became frustrated with many things! This is how my daughter is feeling with adjusting to hearing new sounds – she doesn’t ever think her own voice will sound normal again.

Two years later I can look back and say Wow, I sure have learned a lot. Yes, there is a lot of functionality in SharePoint that I am comfortable working with. And holy cow, I know how to create a fairly complex data view web part – when even just 18 months ago, I thought Dan was talking in a foreign language when he attempted to show me what a dvwp was. I cannot count the number of times that Dan would tell me “Just watch and you will eventually get it”. I didn’t always believe him back then, but looking back at the progress our department has made with SharePoint projects; I am starting to believe his words.

This week I am finding that I need to remind myself of this scenario as we move forward into our next, more complex SharePoint project. I am worried about how I am going to deliver a user friendly solution when I am not even sure where we are going to start. The users are becoming more confident that my participation can help ensure a solution with features they are excited about, and that is what is making me anxious. I have heard about “how” we can give them many of the features they would like in SharePoint, but I haven’t discovered the best way to do it yet. SharePoint is a robust product and there is always something new to learn: how to connect to reporting services, how to take advantage of more complicated workflows, how to work seamlessly with a SQL Server database. Although I am now comfortable with many aspects of SharePoint design, I am still faced with the same dilemma. I need to gain more confidence in my abilities to learn something new. SharePoint has been an excellent product to help me and the SharePoint community has helped me when I can’t help myself. I look forward to the new challenges we face and I know SharePoint will play an important role in the solution we roll out. How do I know this? Because looking at the past has shown me that I can learn to do what others have shown me is possible to do.

As my daughter struggles with the adjustment to hearing sounds she has no memory of ever hearing, I am reminded of how I felt when I first started learning SharePoint. In the 24 hours that she has had the hearing aids, I already see a huge difference in her tolerance of them – I know my SharePoint learning curve won’t always be as speedy, but as I venture into new projects I need to remember it wasn’t that long ago that I didn’t know the difference between a list and a library.

Avoid the Excel Trap

clip_image002Earlier this week, Marc Anderson called attention to an article in the Boston Globe about a company that is requiring all of its employees to learn JavaScript. That spawned some comments on Twitter which led to a blog post by Marc, which led to a comment by me. Before I start singing “there’s a hole in the bucket dear Lisa…” let me jump to the point. Systems development, of any kind, by anybody, for any reason, should always follow a few rules.

In my comment on Marc’s blog, I called attention to my three favorite development rules:

  • Design your solution following generally accepted principals of systems development
  • Test your solution thoroughly
  • Document your solution adequately

Now, I am going to add a fourth rule:

If your solution is even moderately complex, don’t build it in Excel!

Excel has been biting me in the hind parts, all week long. I have written several blog posts about the fact that SharePoint is a much better choice than Excel when building solutions that primarily store and lookup information. My suggestions sometimes fall on deaf ears, but when things go south, the problem-child spreadsheets often end up in my lap. Here are two that landed there this week:

Seek and Ye Shall Find – One powerful feature of Excel is the ability to write formulas that gather data from other parts of your sheet, other tabs, other spreadsheets, and even data sources outside of Excel. The problem is that these formulas work even if the lookup data is out of date. Earlier this week, we suffered a curious error due to the fact that someone left a 2010 version of a lookup list in a spreadsheet that had been updated to 2012. This is precisely why I prefer building these solutions in SharePoint and using External Lists or Data View Web Parts to access back-end data. These are live connections to the data and they don’t require the person using them to push “Refresh All” to make sure the list is populated.

It Should Work – One of the most powerful functions Excel has is vLookup(), the ability to find a value associated with an item by finding the item in one column of a table and retrieving the item of interest in an adjacent column. You specify the place to look, and the offset to use when retrieving. The place to look, is a Range of rows and columns either in the a1..c17 format or a defined Range Name. Unfortunately, when you add rows to the data you are looking things up in, you also have to make sure that the range gets expanded. If you don’t, you have the other error we experienced this week, the item isn’t found. Even more unfortunate is the fact that vLookup() doesn’t return “Item not found” by default, it returns the value associated with the last item it checked, which is the last item in the range. Here again, I find myself longing for a solution where one SharePoint list looking up stuff in another SharePoint list. I just have to point to the right list; I don’t have to define the top, bottom, left and right sides of the list.

I know that spreadsheets were the first “killer app”, and some say that nothing has come along on the PC that can rival them; I disagree. Spreadsheet use is out of control; people trust them to do too much work, and they ignore all the rules while building them. Every really complex spreadsheet that I’ve ever seen, was written while one or more people were “in the zone”. They weren’t really designed; they simply morphed into shape as needed. In addition, those solutions are undocumented or poorly documented, poorly tested and they remain in the same fragile state they were born in. Version control is performed by pressing “Save As” and appending a year, event, or people string to the file name. On top of that, I’d be willing to bet 50% or more of the spreadsheets in use today include at least one formula that has been overwritten with a fixed value.

SharePoint offers us an escape from these traps. Rather than try and force Excel into pseudo-database mode, we have a product that is actually is capable of supporting a solution that requires the power and reliability of a database and the control of a content management platform. On top of that, SharePoint gives us a wide variety of ways to extend solutions through workflows and programming. We can automate, protect, distribute and decentralize functions and features without exposing critical elements to random, unauthorized and undocumented changes. I know that it’s easy to reach for Excel. I know how tempting that row-column blank slate can be, but it’s a trap, pure and simple.

Form vs. Function

imageWhen we get firewood delivered, I usually poke around the pile to see if there are any interesting bits that might work better in my woodshop than the woodstove. The piece pictured below seemed to want to be a bowl. Unfortunately, as you can see it was difficult to expose the natural features and make it round. In turning it, I had to decide where to stop, as I was opening up more of the missing 80° gap with each pass. I stopped just before plowing through the undercut side, and ruining the piece. That left me with something between a functional piece and an artistic statement. Of course I’m sharing this because I see parallels to the discussion we are having at work.

Our current challenge is to decide which systems or parts of systems are going to be hand-crafted fat-client desktop applications (the stuff Microsoft has labeled ‘legacy’), which are going to end up in SharePoint and which will land on a phone or tablet. This is heady stuff, but like my bowl, I think our results will be defined by balancing our goals between form and function.

imageDesktop vs. Browser – Despite the legacy stamp, I can’t imagine anyone choosing to run transaction processing systems in a browser. I might enjoy the novelty of processing a transaction in a browser, but I can tell you from my experience with our expense reporting system, performing multiple transactions in a browser is painful. On the other hand, viewing certain reports in a browser as opposed to on paper has definite appeal. Of course we still need printed reports in some cases, but fewer than we have had in the past. Fortunately SharePoint and SQL Server Reporting Services (SSRS) lets us give that choice to the user. In between those extremes will be the functional elements that we can break away from the full-blown transaction. For example, allowing an underwriter to mock-up and request one of a few changes to a policy would make sense for several reasons:

  1. To add at these features to our rating system would be difficult
  2. These types of transactions occur less than 1/10 as often as complete rating transactions
  3. The component transactions appeal to several people and those people travel

Browser vs. Mobile – Notice that I’m not starting with Desktop vs. Mobile and for clarity, when I say mobile, I mean native mobile apps, not simply exposing a browser-based application on a mobile device. Until we get well beyond the point of the tablet replacing the laptop, I don’t see tablets or smartphones being the domain of transaction processing. I’m sorry, as much as people are all hyped up about being able to run full-blown Windows on a tablet, they aren’t going to replace laptops or desktops for processing transactions. You might be able to get everthing running, but who wants to work like that? For the foreseeable future, there will be a distinct difference between apps and applications. Apps will be rifle-shot accurate programs that allow for handling specific tasks from anywhere. Applications will remain shotgun style programs that cover a lot of ground from the comfort of a desk. Starting an automobile claim from an accident site, with a picture and a minimum of on-scene information is easy. Processing a complex cash receipt covering multiple transactions in multiple currencies isn’t going to happen while I am:

  • Using a handheld device, or any device that prevents simultaneously viewing several legible windows/widgets
  • In the absence of a horizontal work surface
  • Using a tenuous or unsecure Internet connection
  • More than 50’ from a printer

As with browser-based solutions, there are tons of opportunities to break out specific functions to mobile apps. The approval process is a great example. Approval is a rifle shot function, and it has to happen when it has to happen – a perfect match for a mobile device.

As I have mentioned in recent blog entries, we have been investigating SharePoint’s capability to fill the “in browser” circle of the Venn diagram I am imagining, and I am very happy with those capabilities. The key to SharePoint’s awesome potential is its ability to connect to the back-end data and the myriad ways it can manipulate and render that data. The person marshaling the transactions may want to be at a desk, the person approving a single transaction can use their phone, but the person who has an interest in the initiation or the results of those transactions will be very happy in a browser.

Taking the Next Step

imageIf you follow this blog, or even if you drop in from time to time, you know two things about our SharePoint implementation – we have had some success, and much work remains. Given that situation, this might not seem like the time to be more aggressive with our SharePoint goals, but I think it is. I want to build on the success we have had; I want to extend our reach into other areas of our business, while also expanding the number of ways that we can take advantage of SharePoint.

Lately, we have been working to find something in between the boring out-of-the-box look and feel of SharePoint and a totally custom solution. We want to avoid recreating, as Owen Baern said: “The Problem SharePoint Solved”, but we are trying to give our users the experience they deserve. I am also trying to save time and money by extending SharePoint into the realm previously dominated by fat-client desktop applications. During the past 10 months, we have had some good success building off the lessons we learned during a training session last year with Marc D. Anderson, but as the title suggests, it was time to move the bar. Last year, we learned a lot about working with Data View Web Parts and script, and we used that knowledge to build a couple of very cool summary analysis pages (dashboards if you prefer buzzwords). This year, we wanted to get started with Marc’s SPServices library, so we had him take us to school again.

I have seen a large number of shout-out’s and thank you’s to Marc on Twitter about SPServices, but we really hadn’t come up with a reason to use it, until a couple of months ago. Having split our Internet-facing server into several site collections, we wanted a way to intelligently share items from a master list at the root level of the farm. The list contains employee contact information, including relational items so we can assign them to the sites we insure, the committees of our Board of Directors, and the business partners we deal with. All of these assignments are made by role, so an insured knows who the underwriter is, who the engineer(s) are and who to call for technical support. We have done this in the past with a separate master list in each site collection, using views or DVWPs, but we wanted a single list, so the people who maintain the contact information didn’t have to bounce around between sites. We also wanted the list to be easy to load on each/any page, and we wanted it to look good.

Marc quickly put us on the road to success, with a simple extract of data from the master list, displayed in a Content Editor Webpart that had that familiar SharePoint look and feel. We were impressed, but then Marc put us on the highway, making the list look a little better and driving its contents dynamically from the page it was on. I dropped that link on a test page, in a different site collection, and it sprang to life with the appropriate data. Next, Marc put us on a Trans-AM course, when he used his library’s feature to return a JSON array, so that we could group the results in a way that made sense to our customers, as opposed to the way they were returned by the query. image

Some of our insureds operate multiple reactor sites and there may be numerous underwriters and/or engineers assigned to the account. In those cases, we wanted the account staff to be grouped by insured site, but we wanted the technical staff only listed once. For single unit locations, we wanted the team listed without any grouping or at least with the single group opening in an expanded state.

As Marc worked his middle-tier magic, we got the clear impression that he could put us on a Formula-1 course if we wanted to go that far – we didn’t. We want a good user experience, but we still wanted our users to know they were in SharePoint. This is the point when I am glad we teamed up with Marc. He didn’t come here with a solution in search of a problem; he didn’t walk in trying to sell us the same approach he used at his last client, or his favorite approach, in fact, he ended up trying something new. He came to teach, and he taught us how to do what we wanted to be able to do. If we ever need help building a solution, he’s the first person I would call. This time around it was a learning event, and it was a total success.

Reflection

image
CT River from North End Bridge

Whenever I give a presentation, I like to think that I have a huge advantage over Microsoft employees and SharePoint consultants. In the first case, I can freely talk about the things SharePoint doesn’t do well, and the things I know it could do better. In the second case, I can talk about the things I don’t do well – I don’t have to try to impress you with my prowess so that you might hire me. I do sometimes worry that if my boss actually reads this blog, he might wonder “why the heck did I leave this bozo in charge,” but I recall that he was the first to identify the therapeutic effect of blogging, so he won’t hold it against me when I air the laundry.

The one consistent truth in systems development, ECM solution development or just about any development process that involves people is that things go off the rails from time to time. I have been building solutions that change the way people work for 35 years, and I have seen just about every reason why solutions fail, from sabotage to resistance at a level that would confound the Borg Queen. I’ve seen solutions that are as good as the technology would permit, as good as the budget would permit, and/or as good as the deadline would permit, but were still not good enough. I have also seen solutions fail because the infrastructure they were built upon evolved in a different direction. None of this is happening today in our case, because our system isn’t failing, and my experience tells me that it isn’t about to fail. Our system is just waiting to be “the way we do things.” Right now, we are still fighting the memory of “the way we’ve always done things” and that is one tough battle. You may have noticed that I haven’t described what went wrong to inspire this blog entry – it doesn’t matter. In fact, if I tell you the questions we were asking as this week ended, I’m sure you will recognize them:

  • What is so hard about this?
  • Why didn’t they tell us they were having this issue?
  • What made them think that (the workaround) would help?

Then we asked the most important question of all – why aren’t they learning how to work with this process?

When we build information management systems, we like to think that we are improving a business process. That’s a comfortable way of looking at development because it creates an abstraction layer between us and the people involved. Considering the business process allows us to ignore history; we don’t care why they “used to do it this way”, we just assume it was because they didn’t have our solution. We aggravate this situation by assuming that everyone wants every process to be as efficient as it can be. We know better. In reality, we are trying to change the behavior of a group of people, and in general, people don’t like change. When things start to go wrong though, we tend to skip past the abstraction layer and zoom in on the people involved. One of the common complaints that I hear about people, when I talk to others in this industry, is the so-called generation effect on technology adoption. People will talk about how much more willing younger people are to embrace new technology than the old guard employees they are being trained to replace. While there certainly are generational differences when it comes to affinity for technology, we also have to consider that new (younger) employees are learning the process and our solution at the same time. In the absence of a historic regimen, any solution that gets the job done has a certain appeal.

As we consider our final question, we have to consider the answers our users aren’t giving us. Nobody is going to just come right out and say “I don’t want my job to change,” but they don’t want their job to change. We can take the Borg Queen’s approach, and remind them that resistance is futile and that our solution is here to stay, or we can step back and look at the new solution from their point of view. How did the system we just build impact this person’s job? We tend to focus on the things we made easier, but did we make anything more difficult? Resistance is futile; in time, people will adapt their behavior to accommodate the requirements of the new process, but what if we missed something? What if that new process could actually be a little bit better? When we see people pushing back, instead of embracing our solutions, we need to try and discover the underlying reason – maybe it’s something we can fix.

Is SharePoint Too…?

clip_image001Two great things happened this week. Thing one was that I managed to get my first iOS app deployed to my iPhone. Thing two was receiving an email from a satisfied SharePoint user. I can’t demonstrate the app here, but I can share the entire email with you:

     “Thanks, it’s looking good and easy to navigate

You have to admit, that is one sweet message. Here’s the thing though, the SharePoint solution that is looking good and easy to navigate is a large collection of Data View Web Parts, wired up with a bunch of custom XSL code. Why am bringing these two issues up in the same context? Because of the irony; let me explain.

Both solutions were hand-crafted, and both required about the same amount of code. The reaction I received when I completed the SharePoint solution is the first time one of our employees has used the words “easy” and “navigation” in the same sentence when speaking about SharePoint. On the other hand, I showed five people the iPhone app, and all five simply said “cool!” The iPhone app worked exactly like they expected it would. There are hundreds of thousands of iOS apps out there, and they all share a common and intuitive look and feel. That’s no accident, when I was writing the iOS app, I was following the iOS Human Interface Guidelines as provided by Apple on their Development Support Resource page. It is a well-known fact that if you stray too far from those guidelines, your app will never see a slot in the App Store. My app doesn’t have to be sold in the store, or vetted by Apple, but my users certainly expect an “iPhone experience.” My users never know what to expect from SharePoint.

We have delivered dozens of SharePoint solutions over the course of 5 or 6 years, and almost all of them have been well received by the users who rely on them. We have worked hard to make those solutions effective, efficient and acceptable to our users, but this might be the first time we tried to make a SharePoint solution look cool. SharePoint is big, capable, scalable and largely undefined. I once described SharePoint as vacant office space waiting to be built out the way the new tenants desired. If I had to rewrite that blog post today, I would extend the construction metaphor and add that “sadly, there is no building inspector on duty.” Let’s face it; it is very easy to make SharePoint look like crap. The fact that most of our solutions are out-of-the-box boring is my fault, but Microsoft has been my accomplice. SharePoint, like everything Microsoft owns and offers, has been growing in size, scope and complexity since it was released. Apple drives solutions to the simplicity side of the “Ease of Use” scale while Microsoft seems Hell-bent on extending the difficulty side of the scale.

Somebody recently asked me if I would be switching our users over to Windows Phone7. Seriously? Most of my coworkers jumped at the chance to trade in the Windows Mobile phone I was providing them for free for an iPhone for which I only reimburse them between $15 and $60 a month – oh, I’m one of them! When I asked the person “what possible reason would I have to switch them to Phone7?” they mentioned how well it works with SharePoint. The last thing my users want is to navigate out-of-the-box SharePoint pages on a miniature device. Ironically, the solution that garnered me that awesome email looks great and works perfectly on an iPhone; it looks awesome on an iPad; it just doesn’t look like SharePoint.

When I think about the future of SharePoint, I seriously doubt that any other company will come out with a comprehensive platform (there’s that word) that will dethrone SharePoint as king. That said, I could see a combination of easy to use solutions like Box.net running on phones and tablets putting a serious dent into SharePoint’s market share. We have a vested interest in SharePoint, and in Microsoft, as my reseller likes to point out: “I drank the Kool-Aid years ago.” We use Microsoft solutions for networking, email, voicemail, phone, Office, database and we have recently switched from the software development environment I love (Smalltalk) to Visual Studio. Every solution we have is more complicated than the solution it replaced, but every one offers enough benefit to make the effort worthwhile. Unfortunately, “Simplicity” isn’t in Microsoft’s vocabulary, but I worry that it will be the word of choice of our next generation users. In an attempt to prop up the value of Microsoft solutions, my team and I will continue to work to make those solutions look better than they want to look.

Symply the Best

clip_image002Earlier this week, I did something I rarely do, I hired a SharePoint consultant. Based on the corny word-play in my title you probably have already guessed that I am talking about Marc Anderson ( @sympmarc). You may also know that Marc’s forte is, as he describes it, “Development in SharePoint’s Middle Tier” a.k.a. that space between the stuff you can do out of the box, and the stuff that requires Visual Studio.

I have been watching Marc and others ply those waters for a couple of years at different seminars, adding client-side scripting to make pages pretty and more functional, and putting DataView Webparts to work at tasks most people think are beyond SharePoint’s capability. You know my mantra, “make it work, make it fast, and then make it pretty” – jazzing up the page was not a priority for me. As for the impressive DVWP action, well, that’s why we build desktop applications. Marc is a patient man; somehow I think he knew he would reel me in one day. Earlier this year, my SharePoint developer (she’s new and she hates when I call her that, but let’s not start that debate) said something about wanting to make a page look nicer for her users. I said “I know a guy who could show you how to do that.” I didn’t follow through because, well, you know the whole “make it work” thing. Later, as our project grew in complexity, we started to see the need emerging for ways to cut through the overwhelming content in the library and present salient information to the users; I don’t like buzzwords, but yes, I’m talking about a dashboard. That sounded a little more like work, but perhaps some custom views would do the trick. Then, my developer said she wanted to make bits of her solution easier to perform – hmm, that sounded like work.

Why bring in a consultant? – If you follow this blog, you know that I have a hands-on solve-our-own-problems kind of team, so why didn’t we just start messing around with this stuff? There are two reasons: 1) I wanted to avoid “chasing the wrong rabbit.” When I have looked into this area of development in the past, it was clear that there were multiple avenues of attack. The best avenues, however, were not so clear. 2) I wanted to set the right context; a few hundred lines of JavaScript and jQuery code looks daunting. I wanted my team to realize that it isn’t as bad as it appears. To send that message, I needed someone at the keyboard that wouldn’t be figuring it out as he goes.

Why Marc? – I could say “I read his blog… I’ve seen the questions he answers… he wrote the library...” I could say “I had seen Marc in person and I liked what I saw.” Those are true statements, but before I spend someone else’s money, I usually look into things a bit closer. I spoke to Marc – I told him I could only gather my team for two days, and that I had several objectives for those days. I told him that my team includes developers, designers and our Systems Admin, and I wanted him to speak to all of them. I also told him that I was still a bit skeptical; I mean, I am an IT guy and client-side isn’t where I normally do my development work. Marc suggested that we have more of a conversation than a training class, and that we let my team take the conversation where it needed to go. He assured me that he understood the goals and the constraints, and that he was up for the challenge.

Marc began by showing us one of the most impressive web pages I have ever seen in SharePoint. We were impressed and, after he showed us the code, a bit scared. Next, he started building some very small DVWPs from content on our site. As the morning rolled on, the fear started to dissipate and heads started to nod; by the end of the session, people were getting excited. On the second day, Marc tackled our dashboard. He clearly, methodically and patiently illustrated how we should proceed, while building a small working example. He walked us through what we could do in jScript, and then he walked us through his SPServices library and what we could do with jQuery, including making those few things easier to perform. We may not be ready to code that complex app yet, but we are ready to start.