| |
Program Evaluation
Both the Partnership and Media Workshop specifically, use a variety of methods
and multi-level approaches for evaluating current programs and services. Both
internal and external evaluative methods are used, which helps us to continuously
improve upon our offerings. Operating for nearly ten years, these evaluation
methodologies -- some qualitative and some quantitative -- have shown us the
tremendous value our programs bring to the educational community in New York
City and beyond. External evaluations
were conducted by Dale Mann of Interactive Designs (2002) and Dr. Aimee Dorr
of UCLA (1994-1996) to assess the overall effectiveness of Media Workshop programs.
External Evaluations
| Internal Evaluations | Quotes
Internal Evaluation
| Skill level Assessment | Collaborative
Reflections | General Workskhop
Evaluation |
Peer Review and Observations | Student Assessment
| Evaluation Tools
Internally, Media Workshop conducts its own evaluation regarding
the effectiveness of our professional development programs. Our
constructivist philosophy for integrating technology into curricula
is an ambitious pursuit for a number of reasons. Working in urban
New York City (NYC) public schools which educate over one million
students and employ more than 80,000 teachers, poses questions concerning
issues such as equity, staff development, and communication. Additionally,
the broad spectrum of populations that encompass NYC public schools
inserts a mammoth amount of variables into equations that attempt
to make decisions for the entire system. Due to these variables,
a “one size fits all” approach to the decisions affecting
learning and instruction often fall short of success. Media Workshop
takes a different approach to learning and instruction. By forming
a collaborative partnership, our staff developers accommodate the
individual needs of the teachers and other educators with whom we
work, by taking what educators already know, and together developing
ways to infuse technology into curricula.
As the use of technology in schools expands, so will evolve a systems of “best
practices.” Media Workshop’s pursuits are in many ways experimental;
staff developers learn by trial and error – implementing theories and bodies
of knowledge that drive our approaches. In fact, one strength of our program
is the diverse staff whose skills and knowledge levels vary to different degrees
and focus on a variety of issues and concentrations. This diversity allows our
staff to draw upon one another’s expertise and continuously learn through
peer review and observation.
Media Workshop conducts ongoing evaluations using internal resources, namely
our own staff and questionnaires developed in-house, with the assistance of
graduate student researchers at Columbia University. A number of methodologies
are used, depending upon the context in which we work with an individual, school
or organization. These methods are outlined below. In general, results from
these evaluations unequivocally show the significant and positive impact of
Media Workshop staff and services.
Skill-level Pre and Post Assessments
|
Using both quantitative and qualitative questionnaires, prior to beginning
our work with an educator, and following the completion of a project,
staff assess the skill level of each participant, relative to their perspectives
and individual experiences with using instructional technology. This helps
our staff to assess the impact that we may or may not have on a specific
individual, in helping him or her to reach their curriculum and instructional
technology goals, based on their level and ability.
|
“ I truly liked being able to work one-on-on-one,
and having my questions and needs met as occurred, which usually
is not the case in group workshops.”
“[Staff developer] was very eager and available to make
me comfortable with what I was doing – her encouragement and
research was very helpful
Middle school teachers, IS252 |
|
Collaborative Reflection of Process & Product
|
In scenarios where staff developers collaborate with teachers on classroom
projects, we use a method of reflection and evaluation that allows for
staff developers and educators with whom we work, to continuously monitor
the ongoing process of designing, creating and implementing a project.
Using questions that encourage thoughtful reflections, this method encourages
the staff developer-teacher team to consider the entire process of working
together to develop a lesson or learning activity, by first considering
the overall goals of the project, and reflecting upon the successes and
challenges of all stages, from design to implementation. The reflective
process is used two or three times when collaborating on a project, depending
upon the duration of the collaboration.
|
“The demonstrations were
invaluable and the hands-on experience were even better. The instructors
of this workshop were excellent. I will use this to enhance my curriculum.
Working with the Media Workshop has helped me feel comfortable using technology
to enhance my curriculum. I was able to plan and execute interesting projects
with MWNY's support. I'm looking forward to implementing these new strategies
in my classroom on my own next year.” Aileen
Vourliotis, 1st grade teacher, PS 7 |
General Workshop or Seminar Evaluation
| This evaluation is typically used
when we conduct a group workshop or seminar addressing a very specific topic
(i.e. Creating Effective Multimedia Presentations). Often, these evaluations
are used when we conduct workshops on staff development days, or when schools
or districts contract our services to provide a series of workshops for
selected teachers. |
“ The workshop educated me immensely about the
different approaches on how to use technology in the classroom.”
Teacher, Trinity School
“This experience was excellent! I expected less
progress because most teachers don’t know how to use computers or
the Internet.” High School Teacher, High
School for Environmental Studies
|
| "Hands-on experience was the most valuable,
complimented with guidance and support of the instructor as well as my peers."
High School Teacher, Forest Hills High School
"The training sessions we had over the last 3
months have been fabulous. The staff has been extremely motivated
about what they have learned and most importantly have been applying it
to their classrooms and projects...The [Media Workshop staff developer]
managed to keep the group focused and engaged while tending to the varied
needs of the group. It has been a great way of bringing the day and afternoon
staff together....This has been the BEST site-based training we have had."
Wendy Edge,after-school program coordinator, American Museum
of Natural History
|
Peer Reviews and Observations
Using collegial observations, this method is used for a Media Workshop staff
developer’s own professional development as a means to learn from one another.
On a monthly basis, staff developers are encouraged to visit a colleague to
both support and observe how he or she works within their school context. Since
every scenario where we conduct staff development is unique, this process allows
staff to understand other effective methods of staff development, while giving
individuals an opportunity to lend support and expertise to their peers. Using
a “Collegial Observation” checklist as a guide, staff can informally
assess their peers while also learning from other approaches and methods that
individuals use.
Student Assessment
|
Similar to the Teacher's skill level assessment, this evaluation
tool allows us to measure students'perspectives and individual
experiences with using instructional technology. This helps
our us to assess the impact that we may or may not have on
a specific individual, in helping him or her to reach their
personal and academic goals with using technology.
|
"I loved Cyber 18. I learned
how to type quicker and I also learned how to use the computer more. I learned
how people make a magazine. I felt like I was really one of those people
that make magazines. I felt very proud.”
Diana Castillo, 5th grade student, PS8, after school program |
Evaluation Tools
Pre-Skill level Assessment
Post-Skill level Assessment
Collaborative Reflections (I)
Collaborative Reflections (II)
Collaborative Reflections (III)
General Workskhop Evaluation
Peer Review and Observations
Student Assessment
|
|
