Document, Automate, Delegate

Everyone I know who works in IT is overworked. Really overworked! Overworked like nothing I have seen before in any other field (excluding hedge fund managers). Is IT the only profession where there is a never-ending supply of urgent work? Has the youthfulness of the IT industry blinded it to traditional work-flow management techniques? Are a few evil-minded overlords afraid they will not be valuable if they do not keep all the secrets hidden?

Why are highly skilled (paid?) IT staff performing repetitious work? Does IT just attract egos who refuse to believe something that was "that hard" to figure out could be done by someone without a degree and without 10 years of experience in the technology field?

Could it be even more simple? If we think about what challenges IT workers, maybe we'll gain some insight into what they like to do. When asked, most IT workers describe the reasons they got into IT as (in no particular order):

1. Solving Puzzles
2. Money
3. Dazzled by wires and buttons

If we understand that IT workers generally tend to think about their tasks in terms of getting them done (solving the puzzle), not in terms of how that task fits into the organization's processes, we get a hint as to how this situation may have arisen. Perhaps it is not a failure of the IT worker, but of IT management. Since many IT managers are promoted based upon their technical skill, rather than their managerial acumen, maybe many of them are missing the point of their managerial duties. Could it be that IT managers and IT staff don't think about how IT integrates into the rest of the organization? Perhaps IT has been relegated to the dark corners of the organization for too long and maybe it's time for IT to unload some of it's burden on the rest of the organization (err... empower the users).

The way I have attempted to rectify this problem in the past is not with highly paid consultants or thousands of pages of procedures, but with a simple philosophy that seems to work quite well when applied rigorously. I call it Document, Automate, Delegate. The central idea is that many processes that IT workers are performing could be and therefore should be made easy enough to delegate to the user community in the organization rather than being held behind the curtain of "IT". All but the most incompetent IT workers are naturally following a progression that closely matches this process, but after the process has been documented and automated, they still hold onto the process. Why? If it's automated sufficiently, why not hand it over to the users and let the department most closely responsible for the task incorporate it into their processes. IT has plenty of projects in the pipeline that are more interesting than running running reports for accounting or creating new users in the LDAP directory. IT must get rid of those repetitious tasks and focus on the tasks that truly cannot be passed on to the users; software engineering, troubleshooting, database engineering, etc....

Document, Automate, Delegate in a Nutshell

Here is the nutshell approach to Document, Automate, Delegate (DAD):

1. First, document what you do.
2. Once you have the process documented, automate it.
3. Delegate the task to the organizational unit most likely to feel empowered by inclusion.

The first two (document and automate), everyone should be doing as much as possible if they are being smart with their workflow. The new part of this philosophy is that tasks are "delegated to the organizational unit most likely to feel empowered by inclusion".

Yes, it's simple. Yes, it works!

About Document, Automate, Delegate

This is not a new idea. I've never considered doing something that I didn't plan for someone less skilled than me to take over at some point. In fact, one could say that most of my career has been spent simplifying interfaces to a level where lay-persons can do my job for me (and waiting for a computer to tell me what I did wrong, but I don't like to talk about that). Since I started on my M.S. in Engineering Management, I've been spending a lot of time thinking about how IT compares to other engineering disciplines and I have to admit I'm a little disappointed. I really expected IT to be smarter than the other engineering disciplines, mostly because some of the smartest people I know are in IT, they are passionate about their work, and the field is young (we don't have decades of overhead built into our processes). I still believe IT has some of the smartest, most passionate people working in the field, but IT management is a near complete failure in most companies. Either the company is treating the IT workers as if they were assembly line employees and applying metrics that insure no challenging projects are undertaken, or IT departments are completely segregated from the organization's strategies.

This isn't terribly surprising, since most IT managers are old IT workers that either knew too much or became too important to lose. Since most companies can't figure out how to pay someone enough money to keep them from leaving the company without promotion them (another huge problem in technology), people without managerial skill are promoted to managerial positions and completely fall on their faces as managers. The IT managers who do not have managerial skills will honestly try to do the right thing, but without any guidance, they will continue to leave many tasks in IT, because, "the users will just screw it up" or the always ambiguous "SECURITY ISSUES!!!!".

If we re-adjust our thinking in such a way that the role of IT is to make advanced technological feats routine and to put the power of the technology in the hands of the company's users, we see that the D.A.D. philosophy has merit.

First, Document It!

No one likes to document their work. This goes double for IT staff. The interesting and challenging part of the IT person's day is solving the puzzle, not writing down how they did it. However, the IT worker also hates repetitive problems. By leveraging hatred of the mundane, vehicles can be adopted to share information and thus keep the most skilled IT workers working on more interesting projects, it is a win-win situation.

This, of course, assumes a certain level of guruship exists in your organization. Maybe your IT staff is barely able to keep the network running and the servers up. Maybe you should be finding ways to attract better IT staff to your company.

Guidelines

1. If you do it more than once or twice, it should be documented
2. Document IMMEDIATELY before you forget what you did
3. If possible, document while figuring out the solution
4. Make documentation searchable by other IT staff

How it Works

Documenting will take more time than just solving the problem, but it will yield vast rewards quickly. The organization will have to be somewhat patient with an overloaded unit while they bring their processes into the current century, but it will be valuable. The best value is that the organization as a whole now has a repository of searchable solutions to common problems and guidelines for common tasks.

I like to have multiple monitors. In one monitor, I'm solving problems. In the other, I'm documenting what I'm doing, so I don't forget. This sometimes poses a problem when working with really complex problems that I have to make several tries at, but UNIX has a great command history that permits you to remember what commands you have been running. Just about any systems administrator should be able to reconstruct what they did using their memory, log files, and their shell history.

Additionally, the IT manager will have an amazing new performance monitoring tool to use when evaluating the performance of an IT worker. For years it has been difficult to impossible to effectively evaluate technology workers. Partially because they are experts in their specialized areas (which the manager may not be familiar with) and partly because the quality of their work can be a bit ethereal. However, a worker can be effectively evaluated based upon the quantity and the quality of the documentation they produce. This may not work for very small teams, but when dealing with a department of workers, it should be apparent who the rising stars are and who is holding the unit back. Note that I would not advocate this as the sole criteria for evaluation, but as one component that could help the IT manager understand who the rock stars really are.

The idea of documenting their work may threaten IT workers who are too busy, do not fully understand their solutions, or who feel threatened when others have access to information that previously resided in their heads only. This is to be expected and the IT manager must work diligently to convince the IT staff that they will become more valuable when they have a repository of information at their fingertips.

There will continue to be some hold-outs that are too stubborn and set in their ways to change. It shouldn't happen in IT. IT workers are supposed to be amongst the most flexible in the organization, but sometimes it happens. There could be several reasons for this:

1. The hold-out does not trust the organization
2. The hold-out feels that by keeping secrets, they are more valuable
3. The hold-out is too busy to update documentation and do their job

These feelings may be completely justified, so give the hold-outs time to adapt and adjust to the new system. If they do not trust the organization, that is a BIG problem. Work on that one and the IT team will fix itself.

If they are truly too busy (gurus are frequently interrupted), give them a documentation week to document every problem that they encounter. This can be accomplished best by getting them out of the office and to someplace with NO PHONES. However, part of the bargain has to be that once some of the documentation is done, others will be able to handle the problems. This means that your hold-out guru has more time to document solutions and should lead to a cycle of increased knowledge by others in the unit and decreased emergency work by your most knowledgeable staff member.

The overall trend I expect when working with these hold-outs is that they will contribute to the documentation pool roughly equivalently to how much they use it. If they aren't using it, they aren't going to be nearly as effective at performing their duties as other team members, This is a self-rectifying problem, just give it time and they will either fall into line or need to seek employment elsewhere.

It does not matter what you use for your documentation, just get it written down. I have used wikis, blogs, hand-coded HTML, TexInfo, Emacs (outline mode -- like I'm using to write this document), spreadsheets, and many more I'm probably not thinking about right now. Once you have a number of documents, it would be wise to make the repository search-able. It really doesn't matter how you do this, but blogs or wikis seem like a great way to do it and many of them have built-in search and comment features.

Then, Automate It!

Once the task has been documented, the document starts reading a lot like a shell script (or batch file for my OS-impaired colleagues). Take advantage of this and automate the task. If you know what needs to be done, there is no reason for a person to read commands and re-type them. They are just going to mistype them anyway. It's better to just type the named of the file that contains all the commands, rather than each command individually.

Guidelines

1. Automate using the simplest tool that completely solves the problem
2. Plan for the automation scripts to be run by the least competent members of the team
3. Use version control to incrementally improve the automation tools

How it Works

Automating takes even more time than documenting, but for perilously long procedures, there is no option. Automating allows the person performing the work to start a job and walk away while it runs. This is a great place to be. Once the team is focused on automation, rather than documentation and performing tasks manually (ick!), they will be more relaxed and more confident in their solutions.

One of the great benefits of having a shared automation library is that it takes some of the ego out of the work being performed. Imagine a situation where one staffer installs a workstation and forgets a step, that worker is blamed and possibly reprimanded. But who could perform every step in a 3000 step document flawlessly? By automating the system, the team begins looking at the automation system as a separate entity that belongs to the entire team. When an error is found, the person finding it reports it and hopefully suggests a patch to repair the issue.

To get started, find your most time consuming procedure and write a simple script that automates the longest part of this process. When looking for a time consuming procedure, do not always look for the process that takes the longest to perform. A 12 hour process that is performed annually is not nearly as problematic as one that takes 1 hour per day. Shoot for the 90% cases at first. If a process can be 90% automated and save 1 hour of staff time each day, ample time will be available to find and automate the first 90% of all the major time consuming activities. Over time, those activities will become less intensive from the perspective of staff man-hours and the remaining 10% can be automated.

The primary goal at first is to introduce the team to automation. The team must see results quickly to support the process, so it is vital that the initial automation cases are those that truly save the team a great deal of time in their daily or weekly workflow. Do not start an automation initiative with a process that is only performed once per quarter or once per year.

Once the team has seen the value (light?) in automation, a greater deal of time will be available to complete any further automation tasks that remain. Over time, if the team is working cooperatively, the automated techniques will become more trusted than the old manual processes.

Once the automation tools are well tested and trusted, why not go ahead and make them user-friendly too? If the tool can be made user friendly, why not proceed to the next step? Delegate it!

Finally, Delegate It!

What is better than a problem that is not your problem? Nothing, that's what! Highly skilled IT workers have been working as data-entry clerks for too long. Running reports, generating statistics, etc... These are not IT tasks in the modern business environment. There was a time when those tasks were virtually impossible for normal business users to perform, but that time was long ago and employees no longer need physical access to the servers to perform computing tasks.

Guidelines

1. Do NOT delegate until it is well tested
2. Find the right unit to which to delegate the process
3. Work with the team being delegated the new task to insure they understand it

How it Works

The delegation process starts long before the task is delegated outside of the IT unit. In fact, the IT unit should be delegating the routine work of running automated processes to more junior staff with each iteration of the automation tools. Once the task has been delegated to the least technically savvy member of the IT team, it is safe to assume that there is not a great deal of guruship required to perform the task. If there is not a great deal of technical knowledge required to complete the work, it shouldn't be in the IT unit. It should be performed by the users who perform the business processes surrounding the task so that they may integrate this additional task (that no longer requires a great deal of tech savvy) into their existing processes.

It is important to select the correct unit when delegating an IT task such as this. Especially if this idea is new to the company, it is essential that the division or department selected understands that this software was developed in-house and that feedback is helpful. The first time a company attempts to integrate activities traditionally performed by IT into the processes of another part of the organization, it is essential that that the project is pursued as a partnership. The division receiving the new work is receiving the ability to work more quickly (without waiting on IT), but they will have to re-engineer some of their processes to include the activity. This requires a certain tolerance for change that may be difficult to find in companies where a great many of the managers are not interested in improving the processes, but rather are waiting for retirement.

Finding the business unit which is most likely to be empowered by performing a task is key. If the unit is already overworked or they do not have staff who are open to change, the delegation will be rejected or sabotaged. This rejection or sabotage may make it politically difficult to perform future delegations, so it is vital that, especially for the first few, the users truly feel empowered by receiving the new technical capability.

Once a somewhat technically savvy business unit has been selected, you must consider what value this brings to their unit by taking over one of the tasks IT has traditionally performed. With luck, members of the unit are already frustrated because they see IT as a bottleneck which keeps them from being able to complete their work as quickly as they could if IT weren't in their way. In a situation like this, the absolute best thing for IT to do is to empower the users to perform some of the work themselves. If the users think IT is not giving them proper priority and IT has a well-defined, highly automated technique for performing the work, problem solved!

Now, the problem is not entirely solved. IT must work with the business users to train them on the use of the automation suite (or their portion of it) if the delegation is to be successful. It is to be expected that the first few times the automated task is performed, the users will have questions. These questions are opportunities to make the automation suite better. If the users have common questions, then it should be added to the documentation for the suite. Maybe start a Wiki or a FAQ for the automation suite, which allows users to share their expertise on using the software. Since this really puts IT in a mindset of developing software for end-users, it can be treated much more like a software project, where the users of the software are the customers.

One of the great benefits of doing these integration activities is that it gets IT staff and the business users talking. To avoid conflict, many companies, keep the two pulled back to their separate corners, where all they hear from each other are complaints, which creates greater stress on everyone involved. Once IT staff are helping to train the users of the automation suite, they will see many new opportunities to improve the software and automate other processes. Does Linda from accounting have a 3000 word cheat sheet for processing accounts receivable? Can IT help automate part of that process or is it already automated as part of another process that was designed for another department, but could easily be tweaked to meet Linda's needs?

Good things happen when IT integrates with the rest of the organization...

What Next?

When the IT infrastructure management is integrated with the rest of the company everyone wins. There are many opportunities for these processes to be integrated and they should work diligently to integrate their activities, but the IT unit will have to take the lead. No one else has the power to refine the processes to a level where they can be performed by non-IT staff.

When integrating, it is essential that the users feel empowered by the addition of this work to their daily routines and those workers who are not receptive to the concept may have legitimate concerns about performing the work themselves, versus having IT do it for them. The IT unit must be aware of those concerns and work to mitigate those problems.

I came to rely on this process during my days as a Systems Administrator and I believe it works. It seems to be a good fit for many companies, especially those for whom Six Sigma (or whatever the current quality management flavor of the month is) is overkill. In many companies, simply getting to a place where the IT tasks are documented and automated is a great improvement, but actually putting the burden of performing the routine work on the users can simultaneously increase the satisfaction of the non-IT workers and relieve some of the burden on the IT staff.

Before You Go

I will leave you with a popular analogy. Many times we hear software compared to how automobiles are built. It's not a totally accurate analogy, but it probably applies here.

It usually goes something like this:

If cars were built the way software is constructed, the car would crash twice a day for no reason whatsoever, you would have to turn the car off and back on again to change the radio stations, etc...

Usually, there is a rebuttal, such as:

If cars were built the way software is constructed, the car would cost $25.00, get 1,000 miles to the gallon, etc...

In this spirit, I submit:

If cars were built the way technology infrastructure is managed:

• All cars would have to be hot-wired to start
• Only mechanics could be licensed to drive
• Shifting gears would require powering off the vehicle
• You would have to call the manufacturer to find your current speed