Evaluation Model
I used the Kirkpatrick Model to evaluate student learning in the IDT Technical Development course. The first level is reaction. I used formative evaluation including: anecdotal notes per student comments, emails, and observations of body language to assess the degree of participants reactions in the course.
Individual quizzes demonstrated student content acquisition. This is the second level of evaluation (learning). The scores will be a part of the final assessment.
The third level, behavior is identified within the simulations. Students show behavior change when they take the time to incorporate new content into their daily practice. All simulations were created to provide opportunities for this to occur.
The final level are the results of learning. This will be found within the course scoring guide and e-Portfolio submission.
Artifact Background Information
In Team Based Learning, part of the evaluation is culmination of scores accrued on individual quizzes and assignments while the second part of the evaluation is based on collaborative team problem-solving and simulations. The evaluation for learners enrolled in IDT Technology Development has both of these components. The quizzes and individual assignments demonstrate accrual of specific content while the team-based assignments show the ability to evaluate and create using the new content.
IBSTPI Competencies
Evaluation and Implementation
17. Evaluate instructional and non-instructional interventions. (advanced)
a) Design evaluation plans (advanced)
I created the evaluation plan for the IDT Technology Development as described within the background information of this entry.
b) Implement formative evaluation plans (essential)
The quizzes and the team based assessments were used as formative assessment. The problems and simulations were created based on student learning in these areas.
c) Implement summative evaluation plans (essential)
The e-portfolio entry and justification of competencies contained within along with the scoring guide will allow for a summative review the program at large. These pieces of data will impact the changes and refinement of this course for future uses.
Artifact
The following is an example of the IBSTPI Competencies and Course Learning Objectives in a scoring guide. The intent of this evaluation is for learners to choose artifacts that best represent their learning within the course and explain how the work best meets the competency. The course evaluator will review justifications, artifact, and level of competency for each student. Comments are provided to help the learner best understand the relationship between what was presented and evaluator correlation to the objectives and competencies.
The following is an artifact to support the creation of an evaluation plan.
Professional Foundations and Planning & Analysis Competencies/Learning Outcomes
| Number | Competency | Rating/Artifact | Comments |
| PF-3 | Update and improve one’s skills, knowledge and attitudes pertaining to instructional design and related fields. | AccomplishedCompetentDevelopingBeginningNot Competent | |
| PF-4 | Apply data collection and analysis skills in instructional design projects | AccomplishedCompetentDevelopingBeginningNot Competent | |
| PF-5 | Identify and respond to ethical and legal implications of design in the workplace. | AccomplishedCompetentDevelopingBeginningNot Competent | |
| PA-6 | Conduct a needs assessment in order to recommend appropriate design solutions and strategies. | AccomplishedCompetentDevelopingBeginningNot Competent | |
| PA-7 | Identify and describe target population and environmental characteristics. | AccomplishedCompetentDevelopingBeginningNot Competent | |
| PA-8 | Select and use analysis techniques for determining instructional content. | AccomplishedCompetentDevelopingBeginningNot Competent | |
| LO-1 | Identify three key components of Planning and Analysis | ||
| L0-2 | Solve business problems based on implementation of the planning and analysis standards. | ||
| L0-3 | Apply each component through simulation experiences. |
Reflection
As of this post, this particular evaluation tool has not been implemented, thus I am unable to reflect on the effectiveness of the components within the given structure. The intent is to use this evaluation tool in the Avila Advantage IDT certificate program starting Fall of 2014. I will then return to this post and share what I have learned and the adjustments I will need to make to ensure this tool is valuable.
Reimagining Education 