Brooke Marton's
ePortfolio
EVALUATION
Definition
"Evaluation is the process of determining the adequacy of instruction and learning" (Seels & Richey, 1994, p. 54). ECIT candidates demonstrate their understanding of the domain of evaluation through a variety of activities including problem analysis, criterion-referenced measurement, formative evaluation, and summative evaluation (AECT Standards). Evaluation is a process that is ongoing throughout the design process that is used to determine the adequacy of the instructional program or product. This standard includes 4 subdomains: problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.
Reflection
When thinking back to before I started the IDT program, I had no experience creating or conducting an evaluation. I never even knew it was a part of the design process. The only experience I had in evaluation was when I participated in the course/instructor surveys that were given out at the end of the semester.
After taking the evaluation course in the IDT program, I understand the process of formative and summative evaluations. Prior to taking the course, I thought assessment and evaluation were synonymous, but I have come to realize that the two are different in that an assessment collects information on performance and achievements of the students while an evaluation is essentially a judgement as to whether the instruction was effective. Basically, assessments relate to the student while evaluations effect the program/product/instruction itself.
Artifacts
5.1 Problem Analysis
"Problem analysis involves determining the nature and parameters of the problem by using information-gathering and decision-making strategies" (Seels & Richey, 1994, p. 56). ECIT candidates exhibit technology competencies defined in the knowledge base. Candidates collect, analyze, and interpret data to modify and improve instruction and ECIT projects (AECT Standards).
In this artifact, I evaluated a faculty workshop program for my Product and Program Evaluation course. I had to analyze the program in order to determine the effectiveness of the program. I offered some recommendations to the program as a result of the evaluation findings.
This artifact reflects the needs assessment we created for the Design for Learning course. We analyzed both the learners and instructional needs and developed our instructional strategy based upon those findings.
5.2 Criterion-Referenced Measurement
"Criterion-referenced measurement involves techniques for determining learner mastery of pre-specified content" (Seels & Richey, 1994, p. 56). ECIT candidates utilize criterion-referenced performance indicators in the assessment of instruction and ECIT projects (AECT Standards).
Performance objectives Scratch Jr
In this artifact I cited before, we had to create an objectives checklist as part of our project. This artifact is a list of the performance objectives for the Scratch Jr project.
This is a survey I created for my project in the Program and Product Evaluation course. I used mostly Likert scale questions with room to elaborate upon their response while others were short answer. I designed the items on the survey as the criteria to evaluate the program's effectiveness.
5.3 Formative and Summative Evaluation
"Formative evaluation involves gathering information on adequacy and using this information as a basis for further development. Summative evaluation involves gathering information on adequacy and using this information to make decisions about utilization" (Seels & Richey, 1994, p. 57). ECIT candidates integrate formative and summative evaluation strategies and analyses into the development and modification of instruction, ECIT projects, and ECITprograms (AECT Standards).
In this artifact, as listed above, I evaluated a faculty cohort program in order to help determine its effectiveness. I gathered data and made some suggestions based upon that data. The results were presented to the client to help them make appropriate decisions and modify the program as they felt necessary.
Summative Evaluation Scratch Jr
The two above artifacts are for the project I created for Design for Learning. We developed a summative evaluation through a parent survey form as well as a short simple 1 question survey for the young learners. The formative evaluation entailed meeting with the lead of the children's museum in order to get feedback and gain insight into the feasibility of our program being appropriate for the children's museum context. This feedback was used to make revisions to our program.
Rubrics Evaluation Survey
This is the survey instrument used to evaluate the rubrics online module. The survey examined several domains: content, course expectations, structure, multimedia, interaction, and overall experience. The data was then used to make modifications as needed.
5.4 Long-Range Planning
"Focusing on the organization as a whole is strategic planning ... usually defined as a future period of about three to five years or longer. During strategic planning, managers are trying to decide in the present what must be done to ensure organization success in the future" (Certo, et. al., 1990, p. 168). ECIT candidates demonstrate formal efforts to address the future of this highly dynamic field including the systematic review and implementation of current ECIT developments and innovations (AECT Standards).
This artifact demonstrates the beginning of a project that will evolve over the coming years. The artifact is the very first in a series of workshops that will be developed and implemented in the Fall of 2017. This is part of the next generation face-to- face workshops.
Again, I included this artifact because it also reflects the beginning of a new generation of Canvas online workshops that will be available to faculty to engage with online. Five other online workshops were implemented, but this is the one that I designed using DC&C model, developed using storyboarding and Captivate, formatively assessed by peers and supervisor, implemented using Canvas LMS platform, managed using Google Docs and Google Drive, and evaluated using a participant survey.
Seels, B. B., & Richey, R. C. (1994). Instructional technology: The definition and domains of the field. Washington, DC: Association for Educational Communications and Technology.