New Supervisor Project

and they even gave suggestions what would more interesting for them to learn about if more classes were offered at variety of times. We found this question to be good in terms of collecting needed information. The data collected from this question can be helpful in offering a better and revised program.

How did your project change your work environment?

Many of the questions both from the online survey and follow up interviews concentrated on the project, indicating the importance of this facet of the training. Question seven from the online survey results is focused on feedback received from supervisor, dean or staff. It is clear the respondents all received feedback and most received at least minimal

feedback, the last

In the chart above

are the results from question seven of the online survey results
two columns represent no feedback or much feedback, which were equal.
It appears that this area needs improvement.

The chart below represents results from questions three and four of online survey

Questions three and four ask if the project was challenging and meaningful, indicating the importance of the project. Of the respondents, 7/11 strongly agreed the project was meaningful while 4/11 agreed the project was meaningful, giving this aspect of the training high marks.
The question from the interviews, “How did your project change your work environment?” had a variety of responses. Some of the responders felt indifferent and slightly unsure if there was any impact. For example, one person said, “Unsure-mainly for others-doesn’t know if they are actually using it”. Few others were either indecisive or provided minimal feedback. We believe that the structure of the question asked should be revised. These types of questions are made to collect direct feedback on the meaningfulness and usefulness of the project, and if useful data cannot be collected than something needs to be changed. The third level of Kirkpatrick model is a behavior study, it is used to analyze “To what degree participants apply what they learned during training when they are back on the job” (Kirkpatrick, 2007). Most of the responders provided well-structured and detailed feedback of what they learned, proving the interview question above just needs to be slightly changed to collect better behavioral data.
Analysis from Bosses Feedback
Analyzing feedback from the bosses we can conclude that the bosses are overall happy with the program New Supervisor Series and believe it has been effective in training supervisors.
The bosses mentioned seeing significant improvements from their supervisors in three areas: increased staff recognition, require less assistance from bosses and knowledge of human resource policies and procedures.
Four out of the six bosses saw supervisors placing a greater emphasis on improving staff recognition. Additionally, bosses saw an increase in communication between supervisors and employees, which helped improve employee relations. Supervisors have begun to make recognition a part of their work culture. When asked to give examples of their supervisors effectively using and understanding employee relations procedures and documents it generated several examples.
In resolving problems and planning, observations showed supervisors needed little, if any, assistance from their bosses.

Supervisors showed independence and increased confidence in themselves and their work. With their newly gained confidence supervisors are more assertive and initiated both tasks and projects. Bosses one, two, three and four all agree that their supervisors are applying what they learned from the classes. For example, boss four can attest to their supervisors “enhance their problem solving skills giving them the tools and resources to effectively problem solve.” Similarly, boss five recognized that as a result of the New Supervisor Series their supervisor took on extra duties and is more confident working independently. The New Supervisor Series placed emphasis on human resources policies and procedures helping to sharpen the supervisor’s knowledge and focus attention on best practices. In their responses, the bosses frequently commented on the supervisor’s awareness to different policies. Comments relating to Mason policies similarly stated, “better understanding of these policies”, “knowing best practices” and “understanding the process.” Learning and understanding the A-Z hiring process was the most

improved.
Although the all the bosses seem to collectively agree the program is great they make two distinctive suggestions. The first suggestion is the optional classes become part of the training in the New Supervisor Series. Unfortunately, they do not go into detail as to which class or classes should be considered. Second, another boss suggests more opportunity for supervisors to network and incorporate more group interactions. If the New Supervisor Series involved more group work supervisors are better prepared in their new leadership role after training is complete.
One of the questions asked of the bosses included, “Was there any particular class that the employee learned the most?” and, “Is there a knowledge gap or skill that the employee did not learn in NSS?” We believe those two questions taken from question number eight are inefficient in generating the best responses as they are lumped into one category there is no way to tell which is a knowledge gap and which is a class where the employee learned the most. Perhaps a better question is, “In what class did the employee learn the most?” For example, to measure supervisor's experiences, we need to collect direct examples of instances or use probe questions to dig deeper. Concluding the overall analysis from the notes of the boss’s interview, good questions were asked eliciting responses linked to the main objective of the New Supervisors Series. that speak to the effectiveness in training the supervisors to improve employee retention, encourage supervisors to participate in the hiring process and to learn the responsibilities and duties of a mason supervisor. As boss one stated:
“In three years since I have been working in Admissions, I require staff to participate in New Supervisor Series have not had to fire anyone. Speak volumes to the effectiveness of this program and the information to what is provided by the New Supervisor Series” (Notes from Interview, Boss 1).
Even though some of the questions require improvements to gather better data from the interviews overall we believe this interview was useful in collecting needed information. Though to be certain of our findings we need a bigger pool of candidates.

Conclusions/ Evaluations

While the tools used to measure the effectiveness of the program are adequate, we believe that there are some techniques that can be used to improve the collection of data. We would like to recommend rephrasing some of the questions that elicit more than a yes or no answer; and if those questions do not generate good responses we think adding probe questions is necessary to collect information. Additionally, the interns might benefit from additional training when asking the follow-up questions to utilize probe questions effectively.

As a team we suggest questions like these:

• How did your project change your work environment? - Please identify specific skills learned and provide examples

• What about classes that you took would you like to change? Provide specific answers (Schedule, material, lecture type etc.).

• In addition, if some questions generated short and uninformative answers- probe questions need to be asked to get better data.

• We also suggest to eliminating or replacing the question from the follow up interview: “How did you feel when you graduate?” We believe that this type of question is not useful and does not provide any valuable information.

• Our suggestions for the interview for bosses: Rephrase questions that ask bosses to describe and measure supervisor experiences, the wording of the question was too vague and generate poor quality responses. The last question from the interview notes can either be eliminated or rephrased. The question does not give clear responses nor can the reader differentiate what the supervisors learned or which answers pertain to which question. The second part asked if there was a knowledge gap or skill that the employee did not learned in NSS? This question is not beneficial as there may or may not be a knowledge gap, and sometimes a person does not know what they do not know.