Jim Clemmer's Leader Letter

Jay, an Improvement Points subscriber, sent me an e-mail after reading this February 17, 2006 Improvement Point:

"A key contributor to ineffective training is weak evaluation. "Happy Sheets" (rating of the training program, instructor, facilities, lunch, etc.) don't tell us if the training was any good. Instead, they measure behavior change and impact on service/quality levels, process performance, leadership ratings by those being served, innovation, productivity, costs, or progress in reaching improvement goals."

From Jim Clemmer's article, Organizational Skill Development Pathways and Pitfalls
Read the full article now!
http://www.clemmer.net/excerpts/org_skills.shtml

My response follows Jay's question:

Hi Jim,

I am currently planning a training session, and you hit on my concerns for an effective evaluation form. Is there one that you would recommend for me, so as to get the most from my training deliverable?

Thank you,

Jay

Hi Jay,

That particular Improvement Point and article excerpt come from my experiences at Achieve/Zenger Miller in the eighties and early nineties. Wayyy back in the day, our colleague and leader, Jack Zenger, wrote an article with Kenneth Hargis entitled "Assessing Training Results: It's Time to Take the Plunge" (Training and Development Journal, January 1982). In this article they outlined five ways to evaluate the payback of any leadership-development effort:

1. Assess participants' reactions to a program (happy sheets)
2. Collect anecdotes, incidents, or testimonials
3. Measure knowledge gains
4. Assess behavior change
5. Collect data about organizational performance

Happy Sheets or evaluation forms (#1) can give useful immediate participant impressions and flag any potential problems or improvement opportunities. However, these give no indication of whether anything has changed on the job. Just because participants are entertained or stimulated doesn't mean that translates into lasting training results.

Anecdotes and testimonials (#2) help spread enthusiasm, but they offer no real measurement of changes back on the job.

Like school exams, knowledge testing (#3) will show if participants gained new insights, perspectives, or information. But knowing isn't doing. Understanding leadership theories or using popular new jargon doesn't mean a participant has learned the skill of how to address a key issue or how to focus and energize a team.

Monitoring organizational performance (#4) such as customer perceived service levels, defect/error rates, quality levels, productivity, absenteeism, or turnover takes us more clearly into objective measurements showing whether leadership training has a real and lasting impact. However, these performance indicators are moved by a wide variety of factors. Confidently isolating a training program as the key cause of improvements can be quite difficult.

What Jack Zenger and Kenneth Hargis were arguing for was more rigorous use of behavior change assessment (#4) for leadership training. So in those Achieve/Zenger Miller days, we used survey instruments that today would be called "360 degree assessments." We would have participants score their own behaviors (usually using 7 point scales) on a list of skills or leadership competencies before the training, immediately after the training, and again months after the training. Variations of the same survey (before, after, and again later) were also given to the people each participant led, his or her peers, and his or her manager. This provided the most rigorous evaluation of the effectiveness of the training. Those behaviors could then be correlated to organizational performance (#5).

Today there are many variations of 360 degree feedback tools/instruments that can show fairly clearly if training has had an impact.

So, Jay, it all depends upon what the ultimate objective of your training is. If it's strictly to assess participant reaction to the training, then fine-tuning your evaluation form is where you need to focus your attention (how to do that depends upon what reactions you want feedback on). If you want to assess on-the-job or organizational impact of the training, I suggest you look for ways to combine #4 and #5.

Jim

Jim,

I enjoy your articles and insight but I admit you lost me on this one (How Many Companies Lose That Loving Feeling).

Are you suggesting that the weak management and processes are okay as long as everyone has passion?

You also note that you know of 'dozens' of people who have made the switch to other careers...considering the amount of people you would interact with 'dozens' would hardly constitute a grass roots rebellion. I believe that you need to have some of those things that you seem less keen on. The trick is how do you continue to find passion in everything that everybody does?

I'm currently employed (last 2 years) with a company that I'm guessing went through these changes. It started out with a person who had a vision and a technical skill who hooked up with a few others to work out of his garage. Thirty-five years later and 140 employees on board, the garage is long gone and the passion may not be as thick. Challenge and chances have been replaced with accountability, borderless competition, and demanding customers (who also grew).

Passion, drive, and entrepreneurial spirit are all great, and if you have them, you're in a better position than those who don't. That said I wouldn't be willing to give up good management, solid processes and accountability to customer, shareholder and employee.

Regards,

Chris Moote, Lean Manufacturing Coordinator, BC Instruments

Hi Chris,

I can see how you could get lost in this excerpt from Pathways to Performance: A Guide to Transforming Yourself, Your Team, and Your Organization. It is a bit out of context here on its own.

I am not at all suggesting that weak management and processes are OK as long as everyone has passion. I am a very strong believer in balancing strong technical know-how and technology (what we do) with disciplined processes and systems (how we do it) on a base of strong energy, passion and people focus (who we do it for and with – our customers and staff). That's the "High Performance Balance" and you can read about it at  http://www.clemmer.net/models/hperflship.shtml. This was the focus of an earlier foundation chapter in Pathways to Performance before the reader would get to this excerpt that you're questioning (I expand further on this performance triangle in Chapter One of The Leader's Digest – you can read it online at http://www.clemmer.net/books/tld_ch1.shtml).

The main point of How Many Companies Lose Their Loving Feeling is that too many companies start out with high passion and strong leadership and allow this spirit to be squeezed out of the company by swinging the pendulum too far toward bureaucratic processes and systems. A stream of ongoing research around 100 Best Companies to Work For, national quality prize winners, Fortune's Most Admired Companies, Jim Collin's work on Good to Great and Built to Last, Gallup's worldwide studies on best practices, and the like show clearly that that a small group of top performing organizations are able to balance passion and spirit, systems and processes, with technical skills and technology. So it's not a matter of choosing between passion, drive, entrepreneurial spirit and good management, solid processes, and accountability to the customer. The best leaders/managers and companies do both.

Jim

Thanks for clarifying. I agree passion + strong management and processes will beat the alternative.

Compare it to a hockey team...players with skills and players with passion. Those with both will be your best. The coach's role is to develop and draw out these skills and traits.

Chris

Hi Jim,

I really enjoyed your article "A Call to Action" (at http://www.clemmer.net/excerpts/call_action.shtml (highlighted as an Improvement Point on March 3, 2006). 

My experiences have shown that fear is usually the underlying factor in a life of mediocrity and regrets.  Fear of change, fear of being wrong, fear of the unknown, and fear of being successful; believe it or not. 

Fear usually stems from surrounding oneself with negative people and naysayers who try hard to convince you that you won't succeed because they are more comfortable with you at or below their perceived level.  If one is not strong enough to remove themselves from this influence they will soon believe this nonsense and incorporate it into their own self talk.  Negative self talk can and will have a crippling effect if it is not recognized and corrected.

That being said, some of the most important early influences in my life were the ones who said, "you can't."  Having a bit of a rebellious streak in me turned their negativity into a driving force for me to show them that I could and I did. 

Setting goals and building a roadmap to success will surely leave fewer regrets when we look back on our lives.  I would rather try and fail than not try at all and when I fail I will continue on my path until I get it right. 

Instead of "why me" try "why not me."

Thanks again for doing what you do and being one of those people we should be surrounding ourselves with.

Ken Chisholm, Corporate Sales and Marketing Manager, Great Western Containers Inc.

"There ain't no answer. There ain't gonna be any answer. There never has been an answer. There's the answer!"
Gertrude Stein, American expatriate writer, poet, feminist, and playwright

"It is good to have an end to journey toward; but it is the journey that matters, in the end."
Ursula Le Guin, author

"Of course life is bizarre. The more bizarre it gets, the more interesting it is. The only way to approach it is to make yourself some popcorn and enjoy the show."
David Gerrold, author

"Security is when everything is settled. When nothing can happen to you. Security is the denial of life."
Germaine Greer, Australian feminist and writer

"I've learned that everyone wants to live on top of the mountain, but all the happiness and growth occurs while you're climbing it."
Andy Rooney, TV commentator

"In a marathon everyone who completes the course is a winner.  While some run faster than others and some compete with others up at the front, most of the runners are running against themselves, seeking to better the standards which they set themselves.  Life is more like a marathon for most of us.  We choose which races to enter, and what pace to run at, seeking, most of the time, to better ourselves.  There is ultimately no winning and losing, only the taking part, and the getting better."
Charles Handy, The Hungry Spirit: Beyond Capitalism - A Quest For Purpose in the Modern World

Improvement Points is a free service providing a key thought or quotation from one of my articles, provided three times per week, directly to your e-mail inbox. Each complimentary Improvement Point links directly into the full article on our web site that spawned it. If you'd like to read more about that day's Improvement Point, you can choose to click through to the short article for a quick five-minute read. This is your opportunity for a short pause that refreshes, is an inspirational vitamin, or a quick performance boost. You can circulate especially relevant or timely articles or Improvement Points to your team, Clients, or colleagues for further discussion or action.

Here are my personal three choices of the Improvement Points we sent out in March:

"Leaders don't motivate with rewards and punishments. Whether at home or in the workplace, they energize people to motivate themselves."
- from Jim Clemmer's article, Growing the Leader in Us
Read the full article now!
http://www.clemmer.net/excerpts/growing_leader.shtml

"Within an organization, attention to context and culture might involve keeping everyone in touch with, and connected to, whomever the organization serves. It could mean keeping the long-term vision front and center, especially when problems and obstacles look insurmountable. It might involve clarifying core values and using them as a fixed framework to guide and reinforce everyone's behavior."
- from Jim Clemmer's article, Leaders Shape Focus and Context
Read the full article now!
http://www.clemmer.net/excerpts/leaders_shape.shtml

"Effective change and improvement processes evolve and change to fit the shifting environment and what's being learned about what works and what doesn't. Both top-down and local, or bottom-up, approaches are needed. The challenge is finding the right balance."
- from Jim Clemmer's article, Balancing Top-Down and Bottom-Up Change Processes
Read the full article now!
http://www.clemmer.net/excerpts/balancing_top.shtml

A major challenge for editors of various periodicals is finding fresh, relevant content for their newsletters, magazines, e-zines, or web site. We are seeing a steady increase in editors who are finding their way to our web site to select from over 300 articles and columns freely available. The main index of this rich source of content is at www.clemmer.net/articles.shtml. Clicking on any one of the twenty-six topic areas (which follows the outline of our major models and approaches) takes you to a list of articles with a short description of each one. Click on any that look interesting to read the entire article.

If you know any editors, you could help make their continual search for content easier by recommending this section of our web site to them.

We're also hearing of many managers who print or e-mail their team a copy of a particularly relevant article. They then have a discussion around the concept or approach outlined in the article. Feel free to use this "ten minute tune-up" with your team!

I am always delighted to hear from readers of the Leader Letter with feedback, reflections, suggestions, or differing points of view. Nobody is ever identified in the Leader Letter without their permission.

I am also happy to explore customized, in-house adaptations of any of my material for your team or organization. Drop me an e-mail at [email protected].

Keep learning, laughing, loving, and leading -- living life just for the L of it!

Jim

The CLEMMER Group
10 Pioneer Drive, Suite 105, Kitchener, ON N2P 2A4
Phone: (519) 748-1044 ~ Fax: (519) 748-5813
E-mail: [email protected]
http://www.clemmer.net