When I was making the coffee table that sits in my office, I learned a lot about bending steel. Specifically, I learned that bending square steel tubing isn’t easy and often doesn’t end well. Some bends were wide enough that I could negotiate the turn and repair the damage, but others were just a bend too far. For those, I took it in steps: cut, bend a little, cut again and then weld back together. I thought about that process this week, as I struggled with a workflow issue. I’m not sure if this is a common issue, but it seems to come up for us quite often:
“Is it better to build one big, ugly, complicated workflow or is it better to create a few additional lists and spread the work across them?”
We almost always tend to spread the complexity around. In the solution I’m working on today. I need to accomplish three major steps in order to reduce a 12-15 page Word document into a series of entries in a custom list:
- Parse all of the recommendations out of the Word document
- Determine if the recommendations are New, Open, Pending or Closed as of the date of this report, and
- Create entries into either a custom list of recommendations or in a custom list of updates to existing recommendations.
All of this can be done by a workflow, but it’s a complicated task. A single big ugly workflow would need to keep track of individual recommendations, whether they are new or some other status and look up the details about the inspection report metadata and copy it into the other lists. I opted for a process that involves two additional custom lists:
One is a list of what I call “splits.” Splits are the big chunks of parsed text. All the new recommendations are in a split. Likewise, all the pending recommendations are in a split, and so on. A workflow that runs when the splits are created doesn’t have to determine what kind of recommendation it’s dealing with, they are all new, or pending, or some other status.
That workflow parses all the recommendations into individual entries in the second list. That list is then built up of individual recommendations which have a status (new, pending, etc.) and all the metadata that they need to be transformed into an item in the ultimate destination list. A final workflow that runs as these items are created takes care of dispatching those items.
The cost of this solution is the creation of a couple additional custom lists and a bit of duplication of data (although the items in these intermediate lists can safely be deleted). The benefits include:
Easier to plan – The overall process is easier to plan and stage. The process becomes a fairly simple routing exercise. Big boxes of the same thing arrive at one list where they are opened, separated and distributed to a second list. It’s like a wholesale-to-retail operation.
Easier to build the workflows – Rather than a complicated workflow with lots of variables and lots of steps, I have three single-purpose workflows that are each of a manageable size. I built the process in discreet steps, only moving forward once things are working at each level.
Easier to control new vs. update action – By the time I need to know whether to create a new recommendation or an update to an existing recommendation, the list item I am working with includes that information as well as everything I need to create those items.
Easier to debug – This is a big deal for the developer (me). As I am configuring these workflows, there’s really only one way any of them can fail. That is so much easier to deal with than a workflow that can fail any of three or more ways, especially given the current state of SharePoint Designer’s debugging tools.
Easier to rerun – Some of the workflows trigger alerts, some set metadata in other lists and change the status of other processes. Working in smaller segments means there are fewer things to fix if something ultimately goes off the rails.
It might be tempting to just keep on grinding out steps in a single workflow. However, dividing the steps into logical groups and taking advantage of storing partially processed items in interim lists is a very powerful concept. This approach makes design and development easier. By the way, it worked for that coffee table too. If you want to read more about that project, take a look over here.