Page Revision: 6/3/2011 1:07:48 PM

Training Evaluation

What is Evaluation?

• We examine the assumptions upon which an existing or proposed program is based.

• We study the goals and objectives of the program.

• We collect information about a program’s inputs and outcomes.

• We compare it to some pre-set standards.

• We make a value judgment about the program.

• We report findings in a manner that facilitates their use.

Why Evaluate?

• Planning

• Analysis of program effectiveness or quality

• Direct decision-making

• Maintain accountability

• Program impact assessment

 

• Is the program important or significant enough to warrant evaluation?

   

 

• Is there a legal requirement to carry out an evaluation?

   

 

• Will the results of the evaluation influence decision-making about the program? Will the evaluation answer questions posed by your stakeholders or those interested in the evaluation?

   

 

• Are sufficient funds available to carry out the evaluation?

   

 

• Is there enough time to complete the evaluation?

Solve Problems (What are some low cost training options available to me?)

On-the-job training

Hands on learning for the employee utilizing the required equipment and resources as they are learning the job. This method of training would be the most effective learning environment for an employee due to the fact that the employee is demonstrating transfer of learning simultaneously as they are gaining the knowledge and skill through instruction.

Train the trainer programs

Due to expensive price tags on training events, organizations will send one or two employees to be trained on a particular topic. The employees will return back to the organization and be expected to train other employees that need the same particular skill or knowledge. The employees are labeled as the trainer with subject matter expertise for the particular topic in the organization, and will train other employees in the future.

Cross-training (e.g., details, developmental assignments, special projects)

Training that covers several tasks within a department or office. Employees in a particular department will master their own tasks of responsibility and the tasks of their co-workers in the event that extra help may be needed and different areas require coverage. Cross-training can typically be accomplished internally with the more senior employees being the trainers. Cross-training is appropriate when turnover is high, and headcount cannot be filled immediately due to budget cuts.

Coaching

Coaching is a thought provoking and creative process that inspires the client to maximize their personal and professional potential. This is a method of instruction where the desired outcome is to obtain a certain level of knowledge or skills.

In-house training or brown bag lunches

In-house training is an excellent way to provide training to a large portion of the workforce without a hefty price tag. Facilitators for in-house training must be the subject matter expert of the training topic. The majority of organizations identify employees as candidates who have demonstrated outstanding knowledge and skill on the training topic through work tasks and competency levels. Brown-bag lunches are a good way to recruit individuals to training events where they can come voluntarily for one hour during lunch time, and bring their lunch to eat while participating in the training.

External Subject Matter Expert Training Opportunities
Externally, many different low-cost training methods can be utilized from the employee's home if they telecommute or if they are on-site at the organization. Elimination of travel to training sites saves organizations time and expense. Externally, organizations can seek webinars, podcasts, e-learning events, and videoconferencing opportunities. In addition, organizations may be able to unite with other organizations seeking a particular training to obtain a group training discount from the vendor due to high demand of the training.

LinkedIn Events

Search the social networking site LinkedIn (www.linkedin.com) for events by subject matter or location to find seminars, speaker presentations, and workshops. Some event organizers will charge a fee for admission but many won’t. Similar to vendor training, you may need to sift through sales pitches but you can glean some very useful and practical information from these sources.

Lrnchat (Learn Chat on Twitter)

Learn Chat is an online chat over the social messaging service Twitter that now happens twice every Thursday: first at 11:30-1pm EDT/4:30-6pm BST/5:30-7pm CET and then again at 8:30-10pm EDT/5:30-7pm PDT. Participants are people interested in the topic of learning from one another and who want to discuss how to help other people learn. The official Twitter account for Learn Chat is http://twitter.com/lrnchat.

Toastmasters
Join your local Toastmasters International (http://www.toastmasters.org/) group to help you or your employees develop presentation skills, which can be essential to briefing executives, presenting important material and conveying the value of your training programs to stakeholders. There is a minimal fee for this group.

Learning management system (LMS)

LMS may be linked under corporate universities and widely used. An LMS is a computer software program to deliver, track and manage training. A number of courses maybe housed in the LMS with easy access by the employee to register for the courses, schedule the courses and take the courses. The type of training an LMS can provide is synchronous (self-instruction), asynchronous (collaboration with others), blended and classroom based training.

E-Learning

Becoming a widely popular training delivery method in organizations where participants take courses electronically via the computer. Time and resources may be demanding to set up the e-learning courses initially; however, the organization saves costs on facilitator and logistical needs long-term. Many agencies offer e-learning courses through a learning management system to 1) allow easy access, and 2) create a record of the course being taken.

Webinar

Meetings and presentations conducted via the internet where the presenter is on-line presenting the information, and the participants are sitting at their desks or conference room receiving the training. During webinars, participants are given the chance to ask the trainer questions through the phone line that is required for operation of the webinar. The trainer also takes mini-polls at various points during the training getting participant feedback on issues. Often, the webinar is archived, so that it is available on demand around the clock. Many training based organizations offer free webinars to industry-based organizations.

Podcasts

A series of digital audio or video media files available for download through websites. Podcasts are great tools to have for the workforce to attend at various times due to shift work or difficult availability. Podcasts can be available on demand for anytime learning by employees – in the office, during the commute, or at home. Podcasts, which offer an alternative to reading that appeals to employees who prefer learning modes, enable employees to learn about issues, innovations, and best practices.

Here are examples of on-demand professional and personal development podcasts:

• I.T. Conversations
• Learn Out Loud Podcasts
• Congressional Internet Caucus Advisory Committee
• Life Habits Mentoring
• Tips for Conducting Webinars
• HBR Ideacast
• SpokenWord.org
• Research Channel
• TEDTalks
• John Udell’s Interviews with Innovators
• AMA Podcasts
• ASTD Podcasts

Videoconferencing

This training method allows for multiple sites within organizations to connect together at the same time to participate in training events, meetings or presentations. The training is conducted through video and audio transmissions simultaneously. Videoconferencing is extremely helpful in organizations with global sites that also need regulatory training.

Federal Executive Boards

Agencies should coordinate with their local Federal Executive Board (FEB) for cost-effective training opportunities and pre-retirement seminars in their area. Historically, the Federal Executive Boards have offered low cost and sometimes no-cost training courses and seminars for federal employees. Fees arranged through the FEB are usually considerably less than published rates or tuitions from commercial sources.

Peace Corps posts a listing of no- and low-cost training resources, based on searching sites on the Internet (only as possible resources to consider, not sponsored or evaluated by PC) and make the list available to their employees (with other resources) on their Staff Development and Training page in addition to an Individual Development Plan (IDP) form.

Stay Current What can I learn that will help me refresh my knowledge base and add value?)

Read Trade magazines/Journals

Trade magazines are a good way for employees to learn about issues, innovations, and best practices. Subscription costs continue to decline; many publications are now free. A few are perks of association memberships. Often, a paper subscription comes with online access to the same content. Online access can be shared by several people, who can print out the articles that interest them. Professional reading can be done during the commute or lunch hour. Here are examples of robust content:

· For learning & development and human resources professionals:

o Chief Learning Officer (free)

o Talent Management (free)

o Diversity Executive (free)

o HR Magazine and Staffing Management Magazine (paid)

· For project managers:

o PM Network (available with paid membership in the Project Management Institute)

· For I.T. professionals:

o Federal Computer Week (free)

o Nextgov (free)

· For education technology professionals:

o Converge (free)

· For budget and program analysts:

o Public Budgeting & Finance (available with paid membership in the American Association For Budget And Program Analysis)

· For supervisors, managers, and executives:

o Government Executive (free)

o The Public Manager (subscription)

o Governing (State & Local) (free)

o PA Times (available with paid membership in the American Society of Public Administration)

Nearly every discipline has trade journals. Training departments can help employees stay current by posting hyperlinked lists, such as the above, to agency intranet sites. Facilitating brown bag discussions of articles that touch on hot topics at the agency are another idea.

Department of Justice, Alcohol, Tobacco, and Firearms:
The Stepping up to Leadership Program (SUTL) is a twelve-month aspiring leaders program for non-supervisory GS-9s, 11s, and 12s. The program includes a mix of formal coursework, leadership competency assessments, and personally assigned activities aimed at developing the leadership competencies of program participants. Participants spend approximately eight hours a month on program activities, which include monthly (web-based) training sessions, online team meetings, and individualized learning using online courses available through the agency’s Learning Management System (LMS). The SUTL program is evaluated using monthly training surveys and a final leadership assessment measuring the change in leadership competency scores for each participant. Costs are limited to the purchase of leadership books. All other costs of the program are offset using existing training resources within the agency.

For further information please contact Bill Thimmesch, Program Manager, at 202-648-7293.

Create Opportunities (What can be done to create low cost learning experiences?)

Agencies employ experts in many different fields. Create opportunities for them to convene, with interns and journeyman employees, to discuss cases or program challenges. Leaders abound in Washington and other cities where there are large concentrations of government employees. Invite them to come and speak. People like to talk about their work. Many will say “yes” at no cost to the agency in order to have the opportunity to conduct a discussion with interested employees. For example,

• Invite the Federal Executive Board or DHS Fusion Center Director to visit, so prospective leaders can find out how agencies are collaborating in the community.
• Invite a manager or executive in, to explain how a huge success was accomplished.
• Organize a cross-agency meeting of people who do similar jobs in different bureaus, such as grants management, so that they can share what works.
• Invite employees to organize a gathering in order to listen to a webinar together, and then to lead a discussion afterward.
• Invite an author to come speak about their newest book. Some vendors like Hooks Book Events will identify authors that fit your requirements. Contact information is Perry Pidgeon Hooks, Hooks Book Events, perry@hooksbookevents.com, 301 229 1128 or Loretta Yenson, loretta@hooksbookevents.com, 301-229-1128. They charge a nominal fee.

Consider making some events interagency – that can lead to sharing of cost and effort, while it promotes dialogue and openness to future collaborative efforts.

As mentioned above, one way to make an opportunity for low cost training is to bring in a book author to discuss their book. For example, the Treasury Executive Institute hosts 2-hour book discussions with well known authors on topics related to Leadership and other areas of interest. Other agencies can participate for a nominal fee.

Agencies can easily create a book club by simply following a few steps:

How to Implement a Book Club at Work:[i]

1. Determine if employees are interested in a book club. Send out an email to gauge employee interest in reading a book on their own time and then meeting at lunch once a week to discuss the book.
2. Sometimes organization leaders and other employees may have a book in mind to suggest. (Perhaps an employee recently read a book they'd recommend.) Other times, a small team is recruited to pick a book, or to provide several choices. This step can also depend on who the volunteer readers are. If the majority represents a certain function (the training function for example), you may want to decide upon a recent learning and development book. If readers are from across the organization, you will want a broader or more agency oriented book.
3. Allow the voluntary participants to vote to select the book to read.
4. If possible the agency should purchase the copies of the book. It's a small price to pay for knowledge generation.
5. Hold a quick organizational meeting to determine the number of chapters the group wants to read each week and to pass out the books. Select a volunteer to lead the book discussion at this meeting. Select a volunteer to lead the relevance discussion, too. Select a regular meeting time.
6. Read, meet, discuss.
7. When the group completes the book, select the next book. Send an email announcing the next book and soliciting members for the next round of the book club.
8. Cross functional book club members for agency team building and the cross functional viewpoint is ideal. However, you can also reap benefits when department members, as an example, read together on a book of interest to the department.

Tips:

1. Do invite new members to the book club each time a new book is started. You don't want the group turning into an exclusive team.
2. Select books that have broad appeal. Several books that have been popular in recent years in work book clubs include:

• First Break All the Rules: What the World's Greatest Managers do Differently, by Marcus Buckingham & Curt Coffman
• Good to Great by Jim Collins
• Outliers, by Malcolm Gladwell
• Freakonomics, by Stephen Dubner & Steven Levitt

What You Need:

• One Book per Person (sharing a book is not recommended)
• Conference Room
• Flip Chart or White Board and Markers