Software production Support (DevOps, before we called it DevOps)

In a conversation with a friend once, they jokingly described their inability to play racquetball against other seasoned players as "They are playing racquetball, while I am just hitting a ball around the room."

I'll borrow that reference and apply it to Software Production Support.

Is your Software Production Support group "playing racquetball," or are they "just hitting a ball around the room?"

From a distance they can appear like the same activities.  On closer inspection however, one is much more organized, elegant, patterned, and proactive--while the other is only reactive.

Finding the order from all the choas separates the effective from the ineffective.

There are three particular areas your Software Production Support team should be focus on.  These three areas are:

1. Maintaining Systems
2. Managing Customer Expectations
3. Become a Quick-Reaction Force

1. Maintaining Systems:

Think of your production servers like a fleet of cars.  In a fleet plan, the company sends every car to get an oil change after x number of miles, a tire rotation after y number of miles, and a general tune-up, fluid change, etc. after z number of miles.  This pattern repeats itself for the life of the car that is serviced by the fleet manager.

How often are your server hard drives defragmented?  How often are the transaction-logs backed up?  How often are the indexes reindexed, and the statistics updated?

How often are memory settings adjusted for performance? Latest patches applied? How often are your servers checked to see if there any impending disk space issues?

To maximize system performance, create a "fleet plan" for your servers which checks all of these items at regular intervals.

2. Managing Customer Expectations:

If a server fails, do you know which systems depend on it? If a database goes corrupt, do you know which applications need it, and which corresponding business units will be impacted when that happens?

Do you have a way to communicate to those groups immediately?

Create a dependency map for your products.  A dependency map illustrates which servers host which databases, and then which databases are used by which applications, and finally the names, numbers, and email groups of the business users that are affected by that server/database failure.  This will enable your team to proactively manage your customers expectations.  You can notify them before they have to notify you.

3. Become a Quick-Reaction Force:

The SWAT team, the FireStation, and the Ambulance services all have something in common: they are ready to take action at a moment's notice.

They have the information they need available to them, and additional services available with a simple call.

Do your products have support information organized and readily available?  Do you have the names and numbers of your account representative for each third-party product or tool you support?  Do you have the product-support phone numbers and your support plan credentials readily available?

Do you know who knows what about each application in your enterprise?  Who programmed it originally?  Who has supported it lately?  Which business units use it?  Where is the source code located?

Keeping information about each system updated in a central location should also be part of your "fleet plan."

Another effective tool for a Quick-Response group is a monitoring system.  Something that indicates the overall attitude of each of your production servers?  Disk Space available? Will the system reply to a ping?  Is SQL Agent running? Is that required Windows Service up and running?  Monitoring tools like Nagios can do this for you.

Another great idea is to keep a lessons-learned log for each component you support.  Track problems, fixes to problems, assumptions to be confirmed, and ways to test if the component is functioning properly.

All of these pieces in place will make your production support much more effective.

So, think about it...is your Software Production Support team playing racquetball, or are they just hitting a ball d a room?

Mike J. Berry
www.RedRockResearch.com

What Does it Mean to Be a Professional?

Decades ago I had a friend tell me this question was posed to their High School class. I never found out what the class concluded.

Over the years I have thought often about the answer to this question.

My earlier conclusion was that professionalism meant a separation of work and personal life.  This is something that I think the older generation is better at.  The younger generation seems more transparent about personal matters in the workplace.

As the years go by, however, my experience doesn't support this conclusion as a definition of professionalism.  I find many professionals are actually quite personable.

This has caused me to re-evaluate the answer to this question.

I think the answer I would give now is that professionalism means ownership.  It means responsibility and accountability for producing the appropriate results.

I walked into a CostCo last week looking for a large household item.  I found a smiling attentive employee with whom I asked where I might find the item I was looking for.  He said "I'm new here," and shrugged his shoulders.

There was this moment of pregnant miscommunication.

No doubt he was unable to help me due to his present unfamiliarity with the store layout, but as a customer I felt neglected.

I thought to myself, "Well, are you going to get someone for me who knows where this item is?" And then I realized I had, perhaps, misaligned expectations for customer service from a new employee at a wholesale warehouse selling everything from car tires to margarine.

Then the light bulb went on---a more professional employee would have "owned" my problem.  They would have found someone who did know where my item was and would have walked with me until my problem was solved.

Suddenly I realized I had the answer to my decades-old question: Professionalism means ownership.   Ownership of issues.  Ownership of assignments.  Ownership of tasks.

My thanks go out to the anonymous clueless employee.  After several decades, I finally have my answer.

How would you answer this question?
Mike J. Berry
www.RedRockResearch.com

The Three P's of a Quality Management System

A Quality Management System, sometimes referred to as a Total Quality Management (TQM) System, is a simple concept that will dramatically improve software production quality over time.

Companies that don't have a quality system are commonly reacting to production and support issues due to omissive events.

A simple rule of thumb is to ask yourself how many fires your development team has put out this month.  If any come to mind, then chances are you don't have a proper quality management system in place, and should read on...

I remember early in my career I struggled to get my employees to follow our procedures.  Whenever we'd encounter a production problem with our software, it would inevitably be a result of someone not having completely followed an established procedure.

We would have a big discussion about what should have happened, and about how "we can't forget to do that next time," yet we'd experience the same omission later.

I would get frustrated because I could never seem to find a way to get my team accountable for following our established procedures--until I discovered the "Quality Management System."

A Quality Management System has the following three elements (the Three P's!):


  1. Process (documented--most of us have processes or procedures we are supposed to follow.)

  2. Proof (a separate checklist, or "receipt" that the process was followed for each software release.)

  3. Process-Improvement (a discussion, and then an addition or adjustment to the documented process.)


Most companies have an established--and hopefully documented--software development process.  (If you don't you can download one from my website for Waterfall, or Agile here.)  This is the first 'P' and should be in place at every established development shop.

A great question to ask the team is "How do you know the process was followed for each release?"  This is where you may get the deer in the headlights response.  This is the second 'P' and is the piece missing from most software development shops.

Think of this 'Proof' document as a checklist accompanying each software release.  The checklist would include every major step in the documented process, names of team members performing specific functions, and locations of final source code, test scripts, install files, etc.  The checklist would also require a series of quality checks.  Ie: Were requirements signed off by the customer, stakeholder, tester, and developer?  Was the help file updated with the new release number and appropriate functionality?  Was the source code checked in?  Where is it located?

As problems occur, the checklist would be added to so that the product would be protected against a similar failure in the future.

The governing driver considered here is that one particular problem might broadside the development team once, but after the process is improved, that problem should never occur again.

For example, you might have a stored procedure that goes into production without a "Go" statement at the end.  After the error is discovered, and fixed in production, your team should have a discussion and conclude that a checkbox needs to be added to the quality document stating "All Stored Procedures Confirmed to have 'Go' at the end."

From that point on, whenever a stored procedure is moved into production, the developer presenting it must check for 'Go' statements at the end and then sign their name at the bottom of the checklist.

This is the difference between process improvement, and hope.  Many companies view process improvement as a discussion and some verbal affirmations.  What they are really doing is "hoping."

Actually, the "act" of process improvement is physically altering a written process or procedure.  This is the real definition of process improvement--the third 'P.'

The final endpoint of a quality management system is to achieve excellence.  I've heard excellence defined once as "Crisp execution of established procedures."

You can't have excellence without procedures, proof, and process-improvement.

Mike J. Berry www.RedRockResearch.com

The Bat Phone

Do you have one of those executives that harasses you with status updates to projects, yet never attends the status update meetings?

Perhaps they call you, email you, stop in to your office, and want to know what the latest on project X is?

Is the behavior effecient?  What suggestions do you have about how to convey project status communication within your organization?

Mike J. Berry  www.RedRockResearch.com

Anti-Values

I was sitting in a KFC eating lunch, reading the slogans muraled on the wall.  This particular KFC is supposedly the first KFC in America.  Yes, it's in Utah.  Along with some chicken legs and a drink, you can enjoy a small exhibit showing Colonel Sander's original briefcase, white suite, shoes, etc.

One mural read, "Somehow we'll do it, by the principles of thrift, honor, integrity, and charity."

I thought for a moment.  Some of the financial service companies I've worked with would fail if they valued charity.  Then I thought about how trust is a wonderful interpersonal dynamic, but the companies I've worked with in the medical field allow no latitude for trust.  Everything must be written down and authorized by a credentialed physician.  Walk into a pharmacy and you'll need a signature on piece of paper to get a prescription filled.

Hmmm, just like charity is an anti-value in the financial services industry, trust is an anti-value in the medical industry.

I spent the day thinking about this new concept.  I owe the title of 'Anti-Value' to the Discovery-Channel documentary about Anti-Matter I was watching the night before.  I  guess I'm coining the phrase here, but it makes a lot of sense to me.  Normally, a value is something our society charish's, yet in a particular situation, or line of business--it becomes the wrong thing to do.

I started seeing how this concept can be applied all over to help clarify the decision making process.

I remembered taking third place instead of second in a Maryland school-district programming competition in high school because I let the guy from our rival high school cut in line in front of me to turn in his test.  When the results were announced we had both scored the same grade, but because he handed his paper in first, he won second place and I won third. (I beat him in the State programming competition the following month.)

I've never forgotten this experience, and actually now that I think about it, offering your competitor any leeway is an anti-value.

Some business meetings I've been involved in are a collage of participants cutting other participants off mid-sentence to make their point known.  Rude? Yes.  But, in fact, politeness may be considered an anti-value in these types of situations.

I think the concept is fascinating.  Just as a good value system should be in place to help an organization, department, team, or individual govern their decisions, an anti-value system can compliment a value-system by providing additional clarity for the decision making process.

One example of this is the U.S. government's policy on dealing with terrorists.  The government values having a "no negotiating with terrorists" policy.  As a disincentive to future terrorism, they have an additional policy to provide or produce exactly the opposite of what the terrorists are demanding.  The notion--to give them what they want--really becomes an anti-value, and is an additional input to the decision-making process.  So, in fact, their policy is set by values, and anti-values.

I hope you find this concept as fascinating as I do.  It was the best $7.79 I've spent on lunch in a while.

Mike J. Berry www.RedRockResearch.com

Book Review: What Got You Here Won't Get You There

Marshall Goldsmith's New York Times Bestseller, What Got You Here Won't Get You There: How Successful People Become Even More Successful is an excellent self-help book for executives and managers wishing to improve their "soft skills" and other interpersonal traits.

Goldsmith is an executive coach who has worked with more than 80 of the worlds foremost CEO's.  As a symbol of his influence,  Alliant International University recently renamed their school of management after him.  With these credentials, probably anything he writes is worth reading.

In his book, Goldsmith lists twenty-one common "soft-skill" dysfunctions he has encountered while coaching top executives.   He explains that the higher you go in executive management, the more your problems are behavioral.  A few of these behavioral problems are as follows:

 

    1. The need to win too much

 

    1. Making destructive comments

 

    1. Starting sentences with "No," "But," or "However"

 

    1. Telling the world how smart you are

 

    1. Speaking when angry

 

    1. Withholding information

 

    1. Clinging to the past

 

    1. Playing favorites among direct-reports

 

    1. An excessive need to be "Me" (or, "I can't change, that's just how I am")

 

    1. Goal obsession

 


To get the whole list, you need to read his book.  The first half of the book details these ten and the other eleven common issues at length.

One of the primary challenges Goldsmith writes about is getting executives to understand how they are perceived by others in their work environments, and at home.  He separates our personal "perception" into four categories:

 

    1. Public Knowledge (Traits known to others and self)

 

    1. Private Knowledge (Traits known to self but not to others)

 

    1. Blind Spots (Traits known to others but not to self)

 

    1. Unknowable (Traits unknown to others, and not know to self)

 


Goldsmith says that the most interesting traits to examine and study are #3, the blind spots known to others but not to ourselves.  He provides a formula for detecting these traits, examining them, and fixing any negative discoveries.  The formula is:

 

    1. Collect feedback from everyone around us, using both deliberate and subtle tactics.

 

    1. Apologize to everyone for any negative traits.

 

    1. Advertise that you are beginning a personal campaign to improve and that you would like their feedback periodically as you work on improvement.

 

    1. Listen to feedback in terms of "what can I do in the future to improve" and not "what did I do wrong in the past" (one is positive, one is negative)

 

    1. Thank people for their suggestions, and don't disagree with them.

 

    1. Follow-up relentlessly.  This is the key to the improvement process taking shape.

 


He explains, for example,  that as a professional coach, he and a colleague call each other each evening and report to each other on the progress of their goals.  This simple practice enables them to metric their performance over time--the same thing effective executives do to examine trends in their departmental interests.

Goldsmith discusses several other topics in his book.  One interesting aside is a list of common reasons why goal setting can fail:

 

    1. Time: It takes longer than expected, so it couldn't be completed.

 

    1. Effort: It's harder than was expected.

 

    1. Distractions: Nobody expected a "crisis" to emerge that took resources or time away.

 

    1. Lack of Rewards: After they see some improvement, they don't get enough positive response from others, so they give up.

 

    1. Maintenance: Once a goal is met, there is no fortitude to stick with the pattern that brought success.

 


One of the closing thoughts in Goldsmith's book struck me as quite novel.  As one of his executive coaching tools, he sometimes asks executives to produce a "How to Handle Me" guide for his staff.  This is a short memo detailing behavior, values, lessons from past experience, and input from past and present coworkers and direct reports.

As new hires are on-boarded, part of their welcome packet is the "How to Handle Me" guide from their manager.

I found the most valuable part of Goldsmith's book to be his formula for collecting feedback about others' perceptions of us, and how we can affect change within ourselves where needed.  I appreciated Goldsmiths continuous transcendent theme that all of these tools and dynamics also have value at home to improve our family lives and social relationships.  This was a reoccurring theme in his book.

I recommend Goldsmith's book for middle to senior level management, and to any husband or wife.

Mike J. Berry www.RedRockResearch.com

Your First Week as a Software Development Manager

Wether you are starting a new job, or you just got promoted, the first week as a Software Development Manger, VP, Director, etc, can be a dizzying experience.

Depending on your particular situation, you'll likely have to meet many new people, learn about new systems, and remember to smile often.

A good starting point is the be sure the following items are in place:


  1. Make a contact list of everyone in your department, your peers, you manager.  Include their desk phones, mobile phones, and email addresses.  Keep this list updated.  You will use it for a long time.

  2. Find or Create the 'Development Procedures Manual.'  Include in it the following:

    1. Corporate Mission/Vision Statement & Values

    2. Department Mission/Vision Statement & Values

    3. New Employee Hire checklist

    4. Development Workstation Setup checklist

    5. Software Development Procedures

    6. Coding Standards

    7. VPN Setup Instructions

    8. Weekly Meeting Schedules


  3. Create a 'Development Managers Log' containing the following:

    1. Employee Time Off Log

    2. Observed holiday list

    3. 3rd Party Software Licensing information

    4. Historical Release Log


  4. Be sure you have a source code repository

  5. Be sure you have an issue tracking system

  6. Review/Create the Disaster Recovery plan for all of your critical systems:

    1. Source Code Repository

    2. 3rd Party Code libraries

    3. Issue Tracking System & DB


  7. Make a 'projects list' containing an ever-updating list of projects and their status.

  8. Have a 'welcome meeting' with the group you oversee to tell them something about you.  Whomever interviewed you knows about you, but chances are the group you are now managing doesn't.  Tell them your past work history, your management style, communication plan, and something fun and personable about yourself.

  9. Ask your group what would make their jobs more rewarding.  Ask this question a lot at first because they won't believe you mean it until you have asked the question many times.


Good Luck!  You're off to a good !

Mike J. Berry
www.RedRockResearch.com

What to Look For When Interviewing a Candidate

My sister was recently promoted to manage a team of software project managers for a large bank on the East coast.  She told me she gets to hire someone for the first time in her career.

I told her that hiring is always a bit of a dice roll, but I offered her some advice after having hired about 15 people at various times in my career:

1. The most important indicator of future success is past success.  Good interviewers know this.  Dig into people's past work experience and try to find out if they have been generally successful, or not.  Some indicators of this are whether they have changed jobs often.  If they jumped jobs on their way up the ladder of responsibility, this is OK.  If they jumped sideways, or sometimes down, this is a red flag.  Drill them about each job change.  You will get interesting results.  People will say they were fired, or had fights with their boss or coworkers.  These are usually not your desirable candidates.  If they fought with their previous peers and managers, chances are they will fight with your group also.

2. Look for enthusiasm.  Enthusiasm is a great sign of a star employee.

3. Examine their personal lives (you really can't do this in an interview).  But whatever they tell you can be a clue as to how they respond to accountability, pressure, authority, and responsibility.  If they hate police, or the government, or have been divorced five times, then they may have issues with authority or responsibility.  You have to be careful here because you cannot descriminate.

4. Call their references and ask their references if that person was successful, and if they would re-hire that person.  Ask how socially distracting they were inside the workplace, and what time they came in the morning and what time they went home.  Ask if they were a good team-member, and if they were typically dependable enough to get things done.  Ask why they left and compare their answers to the candidate's explanation.

5. Because software development is not always a 9-to-5 job, a good question to ask is if they have any extra-curricular activity that would prohibit them from staying late if needed.  I have hired people to discover that every day at 5:30 they need to pick up their kid from daycare.  This obligation makes them incompatible with leading a team that may require them to stay late and fix a critical problem.  This is a good thing to find out before you hire someone for a position like that.

6. Try to get them to express an opinion about something business related and that they are passionate about.  Pay attention to how they express their opinion.  Do they express themselves dogmatically, as if their opinion is fact and you must argue with them to object, or do they express their opinion in a collaborative way, where they would be more of an asset in a group discussion where others may disagree.

7. Pay attention to how they show up for the interview.  Are they on time, and dressed for the part.  Did they bring with them a copy of their resume? Are their shoes shiny?

8. Ask them several obvious question about your company to see if they did any research before the interview.  Find something on your website homepage that they would know if they looked there before the interview. This is a clue as to their proactive abilities.

9. Pay attention to how they describe their previous workplace, management, and executive staff.  This will likely be an indicator of what they will think of your staff.

10. If you sense an extreme level of dissatisfaction, high-maintenance, or lots of questions about what's in it for them---beware!  This is an employee that will likely perform the bare minimum and be unnecessarily needy.

There are lots of books and tips about how to be the interviewee, but not so much is written about how to interview.  I wish I could have read these tips years ago when I began hiring people.  I hope this helps others and I would be interested in hearing what readers have to add.
< ike J. Berry
www.RedRockResearch.com

Book Review: The 360 Degree Leader

John C. Maxwell's book,  The 360 Degree Leader, is an excellent field-guide for navigating the challenges of leadership at all levels of an organization.

Maxwell starts his book by dispelling many common dysfunctional myths that are found at line-level, or middle-level management.  Ideas such as "When I get to the top, I'll be in control," and "If I were on top, then people would follow me" are inaccurate adolescent attempts to understand the true nature of leadership--which is influence.

Maxwell continues by explaining the characteristics of influence:


  1. Position - Influence because people have to follow you.

  2. Permission - Influence because people want to follow you.

  3. Production - Influence because of what you have done for the organization.

  4. People Development -Influence because of what you have done for them.

  5. Personhood - Influence because of who you are and what you represent.


Maxwell gives examples of effective leadership in all directions: up, across and down.

To lead up well, he suggests you lighten your leaders load, anticipate your leaders needs and use their time wisely, and invest in Relational Chemistry--get to know what makes your leaders tick.

To lead across, Maxwell suggests you focus on completing your fellow leaders, instead of competing with them.   Be a friend, don't pretend you're perfect, and avoid office politics.

To lead down, Maxwell suggest you develop each team member, place people in their strength zones, model the behavior you desire, transfer the vision from above, and reward the results you desire.

Overall this is a good book worth reading and re-reading every so often.  I recommend it for managers at all levels.

Mike J. Berr >www.RedRockResearch.com

Three Dimensional Value Systems

What is a value system?

As of late, corporations have discovered that mission-statements are only somewhat helpful in providing direction to a company.  Being strategic in nature, they don't provide enough detail to govern tactical decisions made by the corporate employees on a daily basis.

To answer this need, value-statements, and value-systems have come into vogue.  Many companies have value-statements to underscore their mission statements.

Just as some mission statements are more effective than others, some value-systems are more effective than others.

The simple approach to establishing corporate, department, or team values is to get everyone together in a room and have them suggest values the team should adopt.  Voting happens, and the group committs to their agree-upon values.

After one of these sessions, the group might come up with a list like:


  • respect

  • trust

  • excellance

  • high performance


This list is a start, but only representative of a one-dimentional value system.  These values, by themselves, realy don't project any context or weight.

A more effective approach would be a two-dimensional value system.  A two dimensional value-system provides a greater context fabric.  For example, you could say your group values:


  • respect over cynicism

  • trust over hope

  • excellence over heroics

  • high-performance over sub-optimization


These comparison value statements proved direction and context.  This represents a two-dimensional value system, and is more effective that a simple list of values.

A three-dimensional value system is a prioritized list of these comparison statements.  For example, you could say your group values these statements in this order:


  1. trust over hope

  2. excellence over heroics

  3. high-performance over sub-optimization

  4. respect over cynicism


This list shows that trust is the highest factor in inter-departmental dynamics.  It shows that excellence is more important than high-performance (so no cutting corners!), and that the group values trust, excellence, and high-performance more than respect.

Every group will have their own values and differences in priorioties, but putting a three-dimensional value-system in place with your team is a great step forward in building functional team cohesion.

Once in place, a reward-systems can be built around your value system to promote it' ectivness.

Mike J Berry
www.RedRockResearch.com