Introduction
There is no happiness if the things we believe in are
different than
the things we do . -- Albert Camus, Philosopher, Writer
I remember my school days
in a structured classroom, where the teacher presented the class with a
predictable series of lessons in lecture format, followed by uninspired
written, independent practice assignments. Of course, on Fridays I took
the paper and pencil test to assess our acquisition of the knowledge
and comprehension-based concepts. I felt relieved to have gotten
through the test and the material, thinking I wouldn’t have
to worry about that anymore. I often asked myself, “Why is
this important to know?” When I remember school this way, I
am part of the generations of American students whose learning was
influenced by 19 th century ideas on the standardization of public
schools. The 1970’s ‘back to basics’
movement supported the study of relevant core subject matter, student
self-discipline and standardized tests. Perhaps
my parents agreed that time spent on non-academic activities was time
wasted, and supported the decline of whole-language, experiential, and
social change instruction based on the theories of progressivist
thinkers such as Francis Parker.
Colonel Francis Parker,
founder of the ‘Quincy system’, believed in a
child-centered curriculum. Parker’s successor in leadership
of the progressivist movement was pragmatist John Dewey. Dewey
supported instruction that did not teach concepts in isolation, but
learning through experiences specifically designed to assist students
in acquiring knowledge of the conditions and resources of their
communities. Progressivists wanted students to be able to use these
resources to acquire skills to affect social change.
Progressivists did not
use paper and pencil tests exclusively as a way to measure learning.
They liked the idea of evaluating student progress using alternative
methods. The idea of using portfolios can be found in the beginnings of
the progressive education movement in the late 19 th and early 20 th
century. While portfolios were thought to be time consuming, they
placed emphasis on students sharpening research skills and creative
thinking, rather than just building a broad base of knowledge. Present
day Progressivists view portfolios as a more meaningful, in-depth
method for assessing student progress than scores on standardized
tests.
At the end of a typical
school week, elementary school students bring home all the work they
are proud to show to parents. They want to share every page, explain
how they completed each task, and receive praise for papers that show
their best work. This collection of work is a simple type of portfolio
that allows the teacher and student to share academic progress with the
parent, and acts as a summative assessment for the week. I remember a
fourth-grade social studies project that assessed our knowledge of the
physical Earth. Our teacher demonstrated the technique of blowing up a
balloon, layering paper mache strips around it to form a somewhat
spherical form, then instructed us to recall our knowledge of the
oceans and continents and paint the surface accordingly. I can still
see that classroom, and our blue and green models arranged on the
table. The most memorable moments I have and the concepts that stayed
with me were those when the teacher included a performance-based or
authentic assessment.
Authentic assessments
(AAs) incorporate a wide variety of techniques "designed to correspond
as closely as possible to `real world' student experiences" (Custer
1994, p. 66). An assessment is authentic when it measures the
ability of the student to apply an acquired skill or concept in new
situations and calls upon students to create responses instead of
choosing answers from a limited selection of items. Authentic
assessments require students to use higher order thinking skills.
Teachers design authentic, appropriate assessments to reflect the
actual concepts presented and learned in the classroom.
What authentic assessment
does not do is compare or rank students. In this educational climate
requiring benchmarks, universal standards, and accountability for all,
authentic assessments are seen as too time-intensive, varying greatly
in their design and purpose from teacher to teacher, and are too
subjective. Wiggins argues that what is really subjective is the
creation and scoring of standardized tests. “Though the
scoring of standardized tests is not subject to significant error, the
procedure by which items are chosen, and the manner in which norms or
cut-scores are established is often quite subjective--and typically
immune from public scrutiny and oversight.” (Wiggins, 1990,
#26).
Ryan and Miyasaka (1995)
considered performance assessment and portfolio assessment to be types
of authentic assessment. Wiggins provides two descriptors for measuring
assessment authenticity: “an assessment reflects the
challenges, work, and standards engaging practicing professionals, and
that it involves a student with opportunities for dialogue,
explanations, and inquiry” (Wiggins, 1989, p. 703). A
portfolio holds selected samples of student or teacher work, with
reflections generated by the portfolio developer, which
“transforms the artifacts into ‘evidence’
of achievement,” (Barrett, 2001, p.2). Portfolios are a type
of authentic assessment that some say can be used as an alternative to
standardized tests in determining Adequate Yearly Progress of schools
under the No Child Left Behind Law (Harris, 2004)
Much of the available
research on the impact on students involved in portfolio development
addresses pre-service teachers, adult learners, and the use of
portfolios in higher educational institutions to assess student
academic work. Not since the early 1990’s when the states of
Vermont, Connecticut, Pennsylvania and Kentucky initiated statewide
programs for elementary student portfolio development, has any
significant attempt been made to study the effect of student electronic
portfolio development in elementary school populations, as an approach
to “promote a high degree of learner activity and increase
metacognitive awareness.” (Hattie, Biggs, & Purdie,
1996, p.131). Many institutions of higher learning utilize portfolio
development to obtain clear evidence of student learning. If portfolio
creation, and more specifically electronic portfolio development, is
accepted as authentic measures of academic mastery of concepts and
skills by these accredited institutions, then isn’t this
instructional practice appropriate for student assessment at all
educational levels?
The purpose of this study
is to record elementary school student perceptions of how increased use
of technological tools, specifically electronic portfolio development,
increases their level of participation in the learning process and in
self-assessment of their own achievement. Researchers, who have focused
on integration of technology, alternative and authentic assessment, and
student portfolio development in institutions
of higher learning, agree
that further study is needed to determine whether portfolio development
is a beneficial instructional practice at the K-12 level (Harris, 2004,
Barrett, 2005).
What is missing from the
literature is current research on the phenomena that occurs in the
elementary school classroom. Specifically, how do 4 th grade students
respond to electronic portfolio development? What seem to be the
benefits for 4 th grade students engaged in developing electronic
portfolios? I will focus on student perceptions of the portfolio
development process, how it seems to affect their level of engagement,
and how the reflection element of portfolio development improves their
learning.
I intend to collect data
on the student reactions and perceptions from student self-assessment
and reflection of self-selected artifacts for individual electronic
learning-process portfolios, in two suburban 4th grade elementary
classrooms, in a public school with a diverse population. The study
will include a collection of student, and parent survey responses,
student interviews, as well as students’ reflective
artifacts, addressing the results of the portfolio development
experience.
For the purposes of
this study a portfolio is defined as “a purposeful collection
of student work that exhibits the student’s efforts, progress
and achievements in one or more areas. The collection must include
student participation in selecting contents, the criteria for
selection; the criteria for judging merit, and evidence of student
self-reflection” (Northwest Evaluation Association
Conference, 1990, definition developed by Pacific Northwest Educators).
“An electronic portfolio uses electronic technologies,
allowing the portfolio developer to collect and organize portfolio
artifacts in many media types (audio, video, graphics, text). An
electronic portfolio is not a haphazard collection of artifacts (i.e.,
a digital scrapbook or a multimedia presentation) but rather a
reflective tool that demonstrates growth over time.”
(Barrett, 2001, p. 3).
Literature Review
Introduction
As America becomes
increasingly dependent on the mathematical and technological skill sets
of citizens of other countries, discussion over American
students’ lack of engagement and interest in schoolwork
prompts several important questions. If lack of student engagement is a
barrier to student attainment of basic and essential skills for
increased student achievement (U.S. DOE, 1994), which educator and
student behaviors must change to improve engagement, and which
instructional method(s) will be most effective in generating
self-motivated learners? Which instructional practices, that research
indicates lend themselves to technology integration strategies, seem to
support student attainment of skills that will transfer to real-world
applications?
With ever-increasing
expectations that students must be able to interpret and apply ideas
from complicated information, from various technological formats, to
real-world situations, it is imperative that educators identify
effective instructional strategies that integrate technology.
Technology can provide support for differentiated student learning,
when the teacher is proficient in the use of the chosen technology. One
meta-analytic study, which included students from kindergarten through
higher education, revealed that students who used computer based
instruction (CBI) performed better on achievement tests and in the
classroom, taking less time to learn concepts and were more likely to
exhibit a positive attitude toward learning (Kulik,C, 1994, p. 4).
Computers, when used to
engage students in higher-order thinking activities, have improved
academic achievement in mathematics among elementary and middle school
math students. Indeed, researchers in the twentieth century, who have
studied cognition in learners, describe the learning experience as
being most effective, when certain characteristics are present.
The research indicates
these characteristics are active engagement, collaborative groups,
frequent interaction between students and teacher with meaningful
feedback, and instruction that is connected to real-world contexts
(Roschelle, Pea, Hoadley, Gordin, Means, 2000).
Studies by researchers
seeking to determine the cause of progressive decreased intrinsic
motivation toward school work completion in adolescents focus on the
change in environment from elementary to middle school. Their findings
indicate that in the middle school culture there is a prevalence of
greater teacher control of the learning environment and decreased
participation of students in decisions concerning the learning process.
Several studies ( Brophy & Evertson, 1976; Midgley &
Feldlaufer, 1987; Midgley, Feldlaufer, & Eccles, 1988; Moos,
1979), as cited in Eccles & Midgley, 1993), revealed that the
motivating elements of choice and self-management for students making
the transition to a new learning environment, are greatly diminished.
It is well known that
the middle school years are a time when students struggle between
fitting in with their peers and figuring out who they are as
individuals. If this is also a time when they lose some of the control
over the learning process that they were accustomed to having in
elementary school, then it is no wonder some students lose interest in
working to their highest potential. When choice is removed and the
purpose for instruction becomes mastery of concepts for the sake of
assessment, students cannot make a connection between their academic
and personal contexts (Eccles &
Midgley,
1993, p.90). Students at the upper elementary and middle school levels
demonstrate more intrinsic motivation when given an opportunity to
analyze and reflect on the results of their academic efforts, and then
determine the necessary course of action needed to improve their
performance. The previous statement can be generalized for adults in
the workplace. The ability to analyze and improve work performance is
required of any employee no matter the job description.
The increased use of
electronic portfolios in classrooms in K-12 and university learning
environments and in the workplace suggests that portfolios may be an
effective tool, capable of bridging the gap between classroom behaviors
and workplace expectations. Just as portfolios in the school
environment reveal the intellectual, mathematical, artistic strengths
of the developer, so does the professional portfolio provide evidence
of talents to the employer. A study of the impact of experiential
portfolios on adult students, conducted by Judith Brown, Barry
University associate dean and director of The Portfolio Program, School
of Adult Learning and Continuing Education reported the following
outcomes: a) a marked increase in the participants’
self-knowledge after portfolio development; b)a greater recognition of
the value of learning from work and from mentors; and c) improved
communication and organization skills, and greater appreciation of the
role of reflection in recognizing learning (Brown, 2002, p. 234 ).
Research Question
Do students
recognize electronic process-portfolio development as an effective
method for self-assessment and for improving their learning process?
If adult learner outcomes
are positively shown to impact communication and organizational skills
in the work setting, what will be the perceptions of 4th grade students
of the impact on their communication and organizational skills as
students develop?In this paper I have attempted to spotlight the
literature that studies the effects on student learning as a result of
electronic portfolio creation. My search constantly required me to
refocus on my mission as article after article presented advice on how
to create or facilitate the creation of electronic portfolios, select
portfolio creation software, or guidance on assessment of ongoing and
completed student products. There is an abundance of literature that
defines the types and purposes of portfolios and more that tout the
usefulness of this tool as a repository of student work and as a
meaningful use of the available technology, but finding that golden
nugget that provided research-based support for using electronic
portfolios as an effective instructional tool for increasing student
engagement, and transferable skills proved to be more difficult.
Portfolio
Research: Purpose of Portfolios
Portfolios have been
described as “a purposeful collection of student work that
tells the story of a student’s effort, progress and /or
achievement in one or more areas” (Artel & Spandel,
1992, p. 36). Assessing student progress using portfolios became more
widespread during the 1980’s, but received negative press in
1994, when Daniel Koretz, former RAND Corporation researcher, reported
portfolios as lacking usefulness in evaluating student work. Koretz,
(1992) when evaluating Vermont’s statewide implementation
portfolio development in fourth and eighth grade classrooms, found that
a lack of uniformity of the development process from teacher to teacher
and school to school created a non-standardized form of assessment.
Other problems reported in The Vermont Portfolio Assessment
Program: Interim Report on Implementation and Impact,
(1991-92) were inconsistent training of teachers in the first year of
implementation. Teachers expressed the reality that change takes time,
and that they were not given specific guidelines for implementation.
During the second year of implementation, more effective and frequent
training opportunities for teachers were provided.
Some researchers hold the
positivist view that portfolios are assessment tools to obtain
documentation of standards met (Wolf, 1996, p. 40). Others see the
value of portfolios as a vehicle for self-assessment. “Self
assessment, including both self-reflection and self-evaluation, allows
students to develop feelings of ownership and responsibility for
personal learning” (Wagner & Lily, 1999, p.31).
Reflection is a critical component of student learning and portfolio
development (Wilcox 1996; Wolf and Dietz 1998, p.2).
These proponents of
Learning/Process portfolios (Barrett, 2005, p. 5) see portfolio
development as a transformative experience during which the
learner’s critical self-reflection ” results in the
reformulation of a meaning perspective to allow a more inclusive,
discriminating, and integrative understanding of the
learner’s experience” (Brown, 2002, p. 241). Still
one more purpose for portfolio creation is as a showcase to highlight
exceptional student work, the more traditionally recognized model. It
is the Learning/Process portfolio that has received the most attention
in recent professional articles and the model that when studied by
researchers, seems to provide the strongest support for educators to
implement instructional design which utilizes portfolio development.
Learning Process
portfolios that provide scaffolding for cognitive reflection are also a
key factor in helping students develop self-regulated learning. As a
learner progresses from being assisted by others to a self-assisted
state, the learner needs to determine what skills they have acquired
and what they have yet to acquire to progress in a specific context. As
a first step in this diagnostic process, the learner needs to be able
to reflect upon his/her progress in a meaningful way. Portfolios can
serve as a scaffolding technique to help the learner make the
transition from being dependent on others for direction in his/her
writings to becoming an independent writer.
A closer examination of
the literature on portfolio development reveals potential, major
obstacles to obtaining desired results from EP development.
Implementation of an instructional initiative, even if within a small
segment of an institution, requires the acknowledged relevance of the
goals, objectives and planned activities of the individuals involved,
by segments of the institution not directly involved in the initiative.
No member of an institution can act in isolation. Actions or failure to
act has an impact on others or other systems within the institution.
For student electronic portfolio development to take place, all
students and staff who have an interest in the academic and personal
development of each student involved, must at least be aware of the
specific, unique, strategic plan of the primary educator in improving
student achievement levels. A common vision for applying proven,
research-based strategies must be present in the learning community.
For example, if the school’s technology coordinator has not been consulted in advance
on a plan to utilize the school’s mobile lab on a
thrice-weekly basis, for an extended period of time, she may plan to
allow other educators access to the same equipment. Also the
coordinator may need to schedule tutorials for both teacher and
students on the use of hardware or software in order that EP work
sessions are as productive and trouble-free as possible. Open
communication with administration about goals may not only garner
approval for or access to additional resources known to administrators,
but also the moral support and encouragement that may make the
difference when things don’t go as planned.
Another major
consideration when facilitating EP creation is being clear of the
purpose before beginning. Is the portfolio to be a tool for assessment
and if so for formative or summative assessment? Will the portfolio
serve as a collection of artifacts that demonstrate mastery according
to previously set criteria and the completion of a universal checklist?
Or, will the portfolio be part of a balanced system of assessment for
learning that includes a reflective component to ensure the focus is on
process and not product, individual growth and goal attainment
(Barrett, 2004)?
If the
portfolio’s purpose is to provide a framework for the
learning process and opportunities for student reflection, then how can
the teacher be sure of the quality of reflection? (Barak, L. 2005).
Students may not have actually had any direct instruction and guided
practice with reflective writing. Teachers must not assume students
have the experience of reflecting deeply on their progress. Students
may produce reflections that are a narrative of actions or behaviors
that are not truly a self-examination of behaviors and attitudes toward
learning (Farr, 2001).
The expense of the kinds
of up-to-date, digital recording devices, web-authoring software, file
accessibility and storage capabilities must be considered as well as
providing time and personnel resources for training in their use.
These issues must be
discussed and solutions found before considering or implementing a plan
for student electronic portfolio creation.
Portfolios: Research by
Organizations
Several articles I
discovered support electronic student portfolios that move beyond
skills checklists. Dr. Helen Barrett advocates a combination of three
purposes, including “an archive of student work, an
assessment management system to document achievement standards, and an
authoring environment where students can construct their own electronic
portfolios and reflective, digital stories of learning (Barrett, 2004,
p. 1). Dr. Barrett’s recently released White
Paper-Researching Electronic Portfolios and Learner Engagement
(2005) is an excellent place to begin a review of electronic portfolio
research. It provides an overview of the available research on student
portfolios in K-12 classrooms,the difficulties of conducting precise
research due to multiple definitions, and purposes for portfolios. This
paper is a part of ongoing research by The REFLECT Initiative; an
initiative directed by Barrett and dedicated to Researching Electronic
portFolios: Learning, Engagement and Collaboration through Technology.
Dr. Barrett cites Joanne Carney’s (2001) dissertation, as she
noted that little has been added to research literature since the
Herman and Winters’ (1994) paper, “Portfolio
Research: A Slim Collection.” Barrett also notes a Consumer
Guide published by the U.S. Department of Education (1993) that
promotes portfolio development as beneficial in creating student
awareness of the processes and strategies involved in writing, problem-solving, research, analysis and
reflection. If the available research on the effects of student
portfolio creation is supportive of continued implementation of this
practice, then any endeavor to conduct further research that would
substantiate previous findings is needed.
Barrett (2005, 2004)
emphasizes the difference between Assessment of Learning and Assessment
for Learning, another reminder that the selection of a purpose of the
portfolio is the most important decision made by the creator. Dr.
Barrett’s web site provides additional resources for
reviewing teaching and professional portfolios (http://electronicportfolios.com/portfolios/howto/tutorials.html).
Helen Barrett has
recently agreed to assist TaskStream, a producer of ePortfolio
software, in researching “ the effectiveness of the portfolio
development process in secondary student learning, motivation and
engagement... By using a single tool that doesn't require a lot of
technical skill, we can focus on the real goal of the project: student
learning, engagement and reflection, not HTML coding, hyperlinking and
design. I am hoping that TaskStream will add more options for
creativity in design to their tools; but our goal is to get students to
collect (create their digital archive), select the key pieces, reflect
on their growth over time, project their future goals, and respect
their work through sharing with a wider audience.” (Barrett, http://electronicportfolios.org/blog/2005_01_01_eportfolios_archive.html,
#3). Barrett says, “I am hoping that this project provides a
seed for more future serious research about portfolios for learning
(not just for accountability) and that we can show how the development
process can lead to enhanced student self-esteem. (Of course, how to
research that outcome will be a challenge!)”. (Barrett, #4)
The National Coalition
for Electronic Portfolio Research, an alliance of twenty institutions
including the University of Georgia, brings together teams to share the
intricacies of e-portfolio projects, and to research the impact of
ePortfolios. The new NCEPR website, launched in September, 2005, allows
members to communicate and share findings in chat rooms, blogs, an
aggregator, wiki, email lists, and news.
Each participating NCEPR
institution, by implementation and evaluation of ePortfolios, is
addressing key issues toward answering the question of impact on
student learning. In a presentation to The EIFEL (the European
Institute for E-Learning) Conference in October of 2005, Barbara
Cambridge stated that the institutional goal of the University of
Georgia in the area of rhetoric and composition is, “ How do
successful and unsuccessful revisers in first-year composition courses
articulate their revision choices in their e-portfolio reflections?
What does analysis of reflections tell about the effect of revision on
final products?” Other institutional goals include:
Professional identity
-How do preservice teacher professionals identify and develop over the
course of a programmatic e-portfolio experience? University of Texas,
San Antonio
Identity with
educational institution- How is student representation of research in
the university, which has been shown to affect students’
institutional identities and participation, enhanced through
e-portfolios? University of Illinois, Champaign Urbana
Cultural identity - How
does Hawaiian identity influence student self-reflection on learning
experiences and academic and career goals in e-portfolios?
Kapi’olani Community College
Issues in the discipline
- To what extent do discipline-related issues shape e-portfolio
implementation? Washington State University
Psychology - How is
progress in the ten dimensions of understanding within psychology
supported by e–portfolio use? Clemson University
Connections between
academic study and professional reality - How does the use of
e-portfolios, blogs, and wikis influence students’ making of
explicit connections among their aptitudes, knowledge, and skills and
the real work of engineering? Stanford University
Integration of learning
across and beyond academic study - How does the practice of integrative
thinking supported by e-portfolios transfer with students across their
years of undergraduate education and into their personal and
professional lives?
George Mason University
Integration through
linking - Why do students create links and what kinds of materials do
they link? Through links are students reflecting, integrating, building
community, and/or providing context?
St. Olaf College
(Cambridge, B., Presentation, EIFEL Conference, 2005)
http://www.eifel.org/publications/eportfolio/proceedings/ep2005
The collaborative
question for all participants in this endeavor is: How does reflection,
as supported by electronic portfolios, influence student learning? All
participating institutions have a vested interest in supporting the
search for answers to all the above questions. The answers to these
questions, I suspect, will strengthen the viewpoint that when
educational institutions at all levels provide opportunities for
students to create, select and reflect on artifacts that demonstrate
their understanding of content, then not only will a greater number of
students reach self-actualization, the quality of instruction at these
institutions will also improve. I cannot help but think that by finding
answers to these institutional questions, researchers at these
universities will be providing answers to the effects of electronic
portfolio development on the learning of primary and secondary
students.
Though the practice of
implementing authentic assessment to gauge student learning in
elementary classrooms has been studied and widely held as an effective
means for assessing students in rich and meaningful ways, I could not
locate research on the development of perceptions of elementary-level
students engaged in electronic portfolio development in the literature.
Although electronic
portfolios are used widely in elementary and secondary schools,
researchers have devoted more time to studying the results of the use
of portfolios to assess instructional practices of preservice teachers.
Institutions have provided evidence of undergraduate and graduate
student learning, and highlighted institutional portfolios to provide support for accreditation.
Wieseman’s (2005) meta-analysis examined studies and
conceptual papers on the nature of reflection in electronic portfolios
of preservice teachers. The result was the creation of a framework on
three kinds of reflection for three different aspects of education that
includes technical, practical, and critical reflection.
As I guide students in
their portfolio creation, I will place emphasis on the meta-cognitive
aspect of students’ experience. I contend that consistent
attention to critical reflection will increase individual student
awareness of their growth and could precipitate increased academic
gains and motivation in goal setting.
Conclusion
The literature reveals
evidence of several collective efforts by higher education
internationally (NCEPR, The REFLECT Initiative 2006, EIFEL, National
Learning Infrastructure Initiative), and of collaborative efforts
between educators and software creators (i.e., Barrett and TaskStream),
in determining the most effective methods for guiding portfolio
development, assessing portfolios, and eliminating variables in
implementation The research questions posed by these groups might also
determine the true impact of portfolios on meta-cognition in students
and lasting effects from the experience.
There is a need for the
continued efforts of both experts in the field and organizations
mentioned previously to complete qualitative and quantitative research
on the progressive, transformative and transferable effects of the
electronic portfolio creation process. Researchers will need to
consider the lack of uniformity in electronic portfolio design, the
software utilized, and the differences in instruments used to determine
performance. Another major, and arguably the most important,
consideration is the importance of the self-reflective factor and
documenting student growth and change in motivation, attitude, and not
only conceptual knowledge, but knowledge of self as observed through
their reflections. The showcase portfolio and the assessment portfolio
have their place in documenting exceptional work or achievement over
time, but current research has focused on the metacognitive effect of
learning/process electronic portfolios. Students remain the ultimate
beneficiaries of research in this direction.
Methods
What is not
known given the recent change in the educational climate as a result of
the No Child Left Behind Law, is how electronic portfolio development,
as a means of authentic assessment, impacts the learning process in the
4th grade classroom compared to more time-efficient, less open-ended
strategies for determining yearly progress. Given the increased
availability and variety of digital tools for use in the classroom, and
ease and eagerness with which 4th grade students approach technological
tools, how can technology be integrated into instruction to facilitate
real learning? Specifically, how do 4th grade students respond
meta-cognitively to electronic portfolio development? What are the
benefits for 4th grade students engaged in developing electronic
portfolios? I will focus on student perceptions of the portfolio
development process, perceptions of their level of engagement, and how
the reflection element of portfolio development improves the learning.
The student portfolio
development process as an instructional framework for student
self-assessment, self-regulation, and self-reflection has proven to be
a powerful phenomenon with students at the higher education level. Why
then is there a dirth of current research on the impact of this form of
authentic assessment for learning in the place that nurtures the
foundational skills of all learners? Do students recognize electronic
process-portfolio development as an effective method for
self-assessment and for improving their learning process? This study of
this question deserves the attention of educators at all levels.
Participants in this
study include students in two suburban 4th grade elementary classrooms,
in a public school with a diverse population. One class was the control
group that did not receive instruction or utilize the electronic
portfolio guidelines. The other 4th grade class experienced teacher
designed instruction and facilitation in the content, process, and
product development of individual electronic portfolios. The
qualitative methods included a collection of student and parent survey
responses, student interviews, as well as students’
reflective artifacts addressing the benefits of the portfolio
development experience
Twenty two 4th grade
students from two regular education classrooms, and their parents, in
one elementary school participated in the study to determine
perceptions of the electronic portfolio development process. Each
participant completed a pre-portfolio development survey questionnaire
for their views on traditional and authentic assessments
they’ve taken, or reviewed with a student. The 4th grade
students completed The Computer Attitude Questionnaire,
a Likert instrument for grades fourth through eighth, to establish
baseline data for student enjoyment and comfort with computers.
During the electronic
portfolio development process, instruments for documenting observable
learner behaviors and engagement during each instructional phase
included a checklist for teachers and weekly student reflections. The
teacher checklist lists specific higher-order thinking skills, enhanced
problem-solving skills, and documents evidence of increased learner
motivation and occurrences of student engagement. I believe the
checklist format will make it easier for teachers to make observations
on the fly that they can create anecdotal notes from when they have
more time.
These artifacts will
reveal a pattern in student perceptions that the portfolio development
process creates in students of the same age, with diverse backgrounds.
Student perceptions of
their own level of motivation, engagement, and academic progress, as
revealed in their reflections, addresses the question, “Do
4th grade students recognize electronic portfolio development as a
means for self-assessment and for improving their learning
process?”
At the end of the
eighteen week implementation period, all student and teacher
participants will retake the Computer Attitude and Teacher Attitude
Questionnaires to identify changes in individual perspectives and to
establish any patterns that have developed. Student interview questions
at the end of a eighteen-week implementation period will probe
reactions concerning their ability to choose the content of student
portfolios, their views on portfolios as an assessment tool, and
student thoughts on the opportunity to use technology tools to create
and self-regulate a device for determining their own learning. Teacher
reflection at the end of the nine-week implementation period will
elicit views on portfolios as an assessment tool, the uses to which
portfolios are put within their classroom, and views concerning
portfolio-based learning.
To further examine for
patterns of portfolio use, one-on-one interviews of all participants
focus on the perceptions that students, and parents hold about the
impact of the electronic portfolio development process on students.
Interview questions cover perceived positive and negative aspects of
the provided guidelines for portfolio development, and teacher ability
to provide technological support throughout the process. Specific
interview questions address the use of electronic portfolio development
as an assessment for learning compared to more traditional formative,
summative and standardized assessment instruments.
This qualitative study
will provide evidence of the impact of the student electronic portfolio
development from the perspective of upper elementary students.
Educators and administrators at the elementary school level may find
electronic portfolio development to be an alternative method for
determining the true academic progress of their students that federally
required standardized tests do not measure. Educators seeking to
maintain student engagement at a high level will be able to offer proof
that integrating a variety of technological tools in instruction
captures student interest. This study will provide evidence that
including students in facilitating the learning process, and providing
a mechanism for increasing students’ intrinsic motivation, in
turn, increases time on task and improved academic progress.
This study, while
important to the academic community, only includes a small sample of
4th grade students in a specific setting, so results, whether positive
or negative, cannot be generalized to all 4th grade populations. The
difference in each classroom teachers’ teaching experience,
and competency and attitude toward technology will be factors in the
outcomes of the data collected, as each teacher exerts influence on
their students during portfolio development implementation. For the
participants in this specific study, student exposure, or lack thereof,
to various technological tools was a major factor in lesson planning.
The school’s master schedule and availability of the
schools’ two permanent computer labs and two mobile labs (16
laptops each) had a major effect on student access for artifact
incorporation and retrieval on demand. The transient nature of a
percentage of the 4th grade population might normally have affected the
continuity of instruction as the teacher and students in the classroom
receiving the treatment and would have to take additional time to
provide support for new
students; however, this did not become an issue as no new students
enrolled in the classes. I expect that a positive effect would come
from this variable, as students who verbalize and demonstrate the task
expectations to assist others would internalize and improve their own
execution of content selection and reflection upon their own
performance in the portfolio development process.
Instrument Development
As I began this study to
determine how students perceive the impact of electronic portfolio
development on their learning, the question of how students would
compare electronic portfolio (EP) development to other forms of
assessment of their learning was foremost in my mind. I had an
expectation that students would prefer EP development to other forms of
assessment of their learning. My thinking was based upon prior
observation of student engagement and enjoyment in previous class
sessions and long term units of study when computer or other digital
tools were integrated into the lesson. I set about formulating
open-ended interview questions that would allow students to express
their honest reactions to assessment in general, their past experience
with assessments and the type and frequency of the assessments their
teachers have used. The questions I created for the
preportfolio-creation interviews could be placed into two categories;
those that asked students to describe assessment, identify the types of
assessments they were most familiar with and that their teachers have
used, and those questions which asked students to define a portfolio,
and a portfolio’s purpose.
I selected the Computer
Attitude Questionnaire developed by the Texas Center for Educational
Technology to record student attitudes toward computers, their
importance and level of enjoyment based on student responses in these
subscales. The reliability for the six attitude measures ranges from
0.80 to 0.86 (80 to 86% consistency in scores produced). The Computer
Attitude Questionnaire (CAQ) produces seven sub-scale scores, including
Computer Importance, Computer Enjoyment, Study Habits,
Motivation/Persisitence, Empathy, Creative Tendencies, School, and
Anxiety. For the purposed of this study I was most interested in the
Computer Importance, Computer Enjoyment, and School subscales.
“The Computer Attitude Questionnaire (CAQ v5.14) is a
65-item, 4-point Likert-type self-report questionnaire to be used with
students in the fourth through eight grades. The CAQ is designed to
measure attitudes (feelings toward a person, or thing) and prevailing
attitudes (dispositions), rather than achievement. Students record
their own perceptions of the extent to which they agree or disagree
with each item, under the supervision of a teacher in the classroom
environment, or a parent in the home.” (Knezek &
Miyashita, 1993). The CAQ also incorporates three paired-comparison
sets of 4 activities (writing, watching television, reading a book, or
using a computer) asking students their preferences for, difficulty
with, and from which they would learn most.
Results
Using the preportfolio
interview questions in one-on-one sessions, I recorded student
responses to the following questions:
What is an
assessment?
Why do teachers
give assessments?
What kinds of
assessment do your teachers give you?
What type of
assessment do your teachers give you most often?
How do
teachers figure out what students have learned?
How
do your teachers allow you to show what you’ve learned?
What is a
portfolio?
What are
portfolios used for?
Results and Discussion
It was important in this
study to determine student knowledge and awareness of assessments, to
record their personal definitions of assessments, and how their
teachers use assessments. Surprisingly 50% of students were not sure
what the term ‘assessment’ meant. Students called
it
“a
piece of writing”,
“a
review for a test”,
“a
special project”.
Others gave a more
expected, traditional definition stating it is
“like
a test”
“you
take a certain subject for a period of time and the teacher wants to
see if you’ve completed the subject”,
“It’s
a test you take only on Fridays to see what class you’re
supposed to go to for Team Time”.
When asked why teachers
give assessments, students responded that
“teachers
give you important tests to pass to the next grade level”
“to
see if you’ve completed the subject, to see what
you’ve learned”
“to
see if you were paying attention”
“to
see how good you are in that subject”
“to
see where you are on that specific thing”
“they
want us to be able to do them”
When asked what kinds of
assessments their teachers give most students provided a list of
subject area tests. One student stated they take “tests with
one-word answers”. One student responded “she gives
me assessments like ‘write a summary’ of what
I’ve learned”. Other responses included
“she usually gives us written tests, multiple
choice”. In answer to the question ‘How does your
teacher allow you to show what you’ve learned I received a
variety of responses:
“by
calling on you and you answer out loud”
“letting
us write something about it or maybe make a diorama”
“write
your own questions and give them to your partner then the partner
checks to see if
you’ve given enough detail”
“summary
questions to give to everybody”
“Do
problems from the book”
“I
could write a story”
“make
a quiz about it ”
When asked
“What is a portfolio?”, three students responded
‘I don’t know”. Other responses were:
“Sort
of like a file, something to organize. I’m guessing, but
I’m not so sure what it is”
“It’s
like an organizer (binder)”
“I’m
guessing that its a record or something you type down that’s
information”
“I’m
guessing it’s like somewhere where you can go to find
information you need”
After, portfolio
development (the goal setting cycle, artifact selection, and reflective
practice) has become a natural part of the school day and
students’ reflections have reached a level beyond the
narrative of behaviors, can illustrate what they’ve learned
and use portfolios as opportunities for self-assessment and growth, I
will revisit the interview questions on student definitions of
assessments, how they are used, the various types, and definitions of
portfolios, to record any changes in conceptual knowledge and evidence of awareness of
portfolio creation and reflection as a method of assessment of their
learning.
CAQ
Documenting student
opinions about their own computer use, the importance of computers in
completing their school work and general attitudes toward school in
general would provide baseline data prior to the EP creation process.
Students had not yet been told that they would soon begin utilizing
computers in an integrated, personal way as part of their daily
learning. Of the 21 4 th graders who completed the CAQ, all agree or
strongly agree that they “enjoy doing things on the
computer” and all but one student responded that they are not
tired of using a computer. Ninety-one percent of students responded
that they feel comfortable using a computer (see Table 1, p.32).
In the three-paired
comparison activity sets that included read a book, write, watch
television, and use a computer, students chose first,the activities
they would rather do, in the second set, the activities they thought
were more difficult, and in the third set activities from which they
thought they learned more. Although 90% of students would rather use a
computer than watch television, 62% of students would rather read a
book than use a computer, while 52% would rather use a computer than
write. Student responses indicated that 66% found it more difficult to
use a computer than to watch television while 77% found it more
difficult to read a book than to use a computer. Almost all students,
95%, thought they would learn more from computers than watching TV, 52%
thought they would learn more from computers than reading a book and
81% thought they would learn more from a computer than writing.
Table 1
Computer Attitude Questionnaire – Computer
Importance/Computer Enjoyment Subscales
|
Item
Note . Scale ranged from 1 - strongly disagree to 4 - strongly agree. a
N = 21
|
Strongly Disagree No.(%)
|
Disagree
No. (%)
|
Agree
No.(%)
|
Strongly Agree No.(%)
|
|
I Enjoy using
computers.
|
|
|
6(28%)
|
15(71%)
|
|
I am tired of using
computers.
|
13(62%)
|
7(33%)
|
1(5%)
|
|
|
I will be able to
get a good job if I learn how to use a computer.
|
|
2(10%)
|
9(42%)
|
10(48%)
|
|
I concentrate on a
computer when I use one.
|
|
|
10(48%)
|
11(52%)
|
|
I enjoy computer
games very much.
|
|
3(14%)
|
3(14%)
|
15(71%)
|
|
I would work harder
if I could use computers more often.
|
3(14%)
|
1(5%)
|
8(38%)
|
9(42 %)
|
|
I know that
computers give me opportunities to learn many new things.
|
|
|
8(38%)
|
13(62%)
|
|
I can learn many
things when I use a computer.
|
|
|
10(48%)
|
11(52%)
|
|
I enjoy lessons on
the computer.
|
|
2(10%)
|
11(52%)
|
8(38%)
|
|
I believe that the
more often teachers use computers, the more I will enjoy school.
|
1(5%)
|
4(19%)
|
10(48%)
|
6(29%)
|
|
I believe that it
is very important for me to learn how to use a computer.
|
|
|
7(33%)
|
14(66%)
|
|
I feel comfortable
working with a computer.
|
|
2(10%)
|
9(42%)
|
10(48%)
|
|
I get a sinking
feeling when I think of trying to use a computer.
|
13(62%)
|
8(38%)
|
|
|
|
I think it takes a
long time to finish when I use a computer.
|
10(48%)
|
6(29%)
|
5(23%)
|
|
|
Computers do not
scare me at all.
|
1(5%)
|
|
4(19%)
|
15(71%)
|
|
Working with a
computer makes me nervous.
|
13(62%)
|
7(33%)
|
1(5%)
|
|
|
Using a computer is
very frustrating.
|
13(62%)
|
6(28%)
|
2(10%)
|
|
|
I will do as little
work with computers as possible.
|
12(57%)
|
5(23%)
|
1(5%)
|
3(14%)
|
|
Computers are
difficult to use.
|
12(57%)
|
7(33%)
|
2(10%)
|
|
|
I can learn more
from books than from a computer.
|
4(19%)
|
8(38%)
|
9(43%)
|
|
Electronic Portfolio
Development
Students began the
development of their electronic portfolios by first recording their
academic and personal goals, so as to have a basis for artifact
selection and specific objectives to be met and reflected upon. I
showed students how to create a folder in their personal drives to
house all files to be collected or created. The goals documents,
created in Microsoft Word, were then linked to a simple, main web page,
containing a table that incorporates four main categories for
organizing linked pages and artifacts.. The four category labels that
students are required to use are ‘Academic and Personal
Goals’, Writer’s Corner, Projects, and Hobbies. The
students themselves agreed upon these labels as titles under which any
artifact could be placed. Students were already proficient in using
Format functions, so were allowed to choose color and graphics to
enhance their pages. One brief but important tutorial I gave students
was on file naming, impressing on them the importance of not including
spaces or symbols within the file name as sometimes the server would
‘get lost’ trying to locate a file. Students are
now involved in choosing artifacts based on their opinion that the
artifacts demonstrate some level of growth towards a goal or that is an
example of their best work to date. Every artifact incorporated into
the portfolio must include a reflection. I have often reminded students
that if they cannot defend their selection and importance of an
artifact to their learning or goal achievement, it is not to be
included in their EP. Emphasis on EP creation has been on a maintaining
a balance between three important components of the learning portfolio,
as modeled in Zubizarreta’s impressive examination of
learning portfolios (Barrett’s term is Learning/Process).
This model of portfolio development moves beyond the showpiece and
product for the sake of completing a checklist types of portfolios.
Students must think critically during the development process using
reflection component as well as collaborating with peers or mentors to
assess their learning beyond simple documentation. “By
providing a structure that is essentially an act of communication--that
is an investment in learning as community—the learning
portfolio concept, primarily through the powerful agency of reviewed
work, facilitates students’ active engagement of the three
crucial domains: 1) reflections on learning, 2) evidence and outcomes
of learning, and 3) collaboration and mentoring (Zubizarreta, 2004, p.
21)
Created
by John Zubizaretta, (2004)
Conclusions
At this point in the study students require
further guidance by me in reflective writing. Their selection of
artifacts is thoughtful and consistent with their academic goals, but
have the feel of writing that sounds like the right thing to say, or is
a narrative of how the student went about accomplishing
their goal without that deep reflection on the personal journey they
took to get there. Time, much practice, and me incorporating
example/non-example strategy will assist students in improving this
skill. What is promising is the eagerness with which students want to
use the laptop computers. I believe this is due to the opportunity to
work with material that matters to them, while at the same time
increasing their motivation to improve skills addressed in the
curriculum, and the involvement they themselves have had in identifying
challenges and highlighting progress made. Electronic portfolios, when
created with a balanced approach have a way of crystallizing, bringing
into focus for students what students have learned, how they learned
it, and what it took to get them there. My plan is to continue
facilitating students in EP creation throughout the school year, so
that reflection becomes not a collection of segmented captions, but a
series of personal realizations about the self as learner.
References
Arter, J.; And Others,
(1995). Portfolios for Assessment and Instruction,
Retrieved
March19, 2006 from ERIC Digest database.
http://searcheric.org/digests/ed388890.html
Barrett, H. (2004).
Electronic Portfolios as Digital Stories of Deep Learning. Retrieved
February 27, 2006 from http://electronicportfolios.com/digistory/epstory.html.
Barrett, H. (2005). Eportfolios
for Learning Blog, Researching Electronic Portfolios,
Retrieved March 19, 2006 from http://electronicportfolios.com/blog/2005/01/researchingelectronicpor_110703008938538317.html.
Black, P. &
Wiliam, D. (1998). Inside the black box: raising standards through
classroom assessment. Phi Delta Kappan, 80(2),
139-144, 146-148.
Brown, J., (2002). Know
Thyself: the impact of portfolio development on adult learning, Adult
Education Quarterly, 52(3), 228-245.
Cambridge,
Barbara: National Coalition on Electronic Portfolio Research
(USA). Asking important questions: research on the influence
of electronic portfolios on student learning. Slideshow,
Retrieved March 11, 2006 from http://www.eifel.org/publications/eportfolio/proceedings/ep2005
Camp, R. (1998).
Portfolio reflection the basis for dialogue Clearing House72(1),
p10.
Carney, J. (2001).
Setting an agenda for electronic portfolio research: A framework for
evaluating portfolio literature. Retrieved February 27, 2006 from http://it.wce.wwu.edu/carney/Presentations/AERA04/AERAresearchlit.pdf.
Carliner, S. (2005).
Commentary: assessing the current status of electronic portfolios. The
Canadian Journal of Learning and Technology, 31(3).
Carney, J., (2005). What
Kind of Electronic Portfolio Research Do We Need? Retrieved
March 11, 2006 from http://it.wce.wwu.edu/carney/Presentations/SITE05/ResearchWeNeed.pdf.
Custer, R. L., 1994, Performance-based
education implementation handbook. Columbia: Instructional
Materials Lab, University of Missouri.
Eccles, J., Midgley, C.,
(1993). Development during adolescence: The impact of Stage-environment
fit on young adolescents’ experiences.
American
Psychologist ; 48(2), p90.Fernsten,
L., Fernsten, J. (2005). Portfolio assessment and reflection: Enhancing
learning through reflective practice. Reflective Practice,
6(2), p303-309.
Gibson, D. &
Barrett, H. (2003). Directions in electronic portfoliodevelopment. Contemporary
Issues in Technology and Teacher Education, [Online serial],
2(4). Retrieved March 11, 2006 from http://www.citejournal.org/vol2/iss4/general/article3.cfm
.
Hall, B. W. &
Hewitt-Gervais, C. M. (2000). The application of student portfolios in
primary-intermediate and self contained-multiage team classroom
environments: Implications for instruction, learning, andassessment. Applied
Measurement in Education, 13(2), 209-228.
Harris, D., (2004). Using
portfolio assessment to improve K-12 schools and meet the
accountability requirements of the no child left behind legislation. Soapbox,
April, 2004. The Institute for the Advancement of Emerging Technologies
in Education at AEL. Retrieved July 7, 2006 from http://www.iaete.org/soapbox/summary.cfm?&tid=Usin6580.
Hartlep, K. Forsyth, A.
(2000). The effect of self-reference on learning and retention. Teaching
of Psychology; Autumn, 27(4),
p269-271.
Herman, J. &
Winters, L. (1994). Portfolio research: A slim collection. Educational
Leadership, 52(2) 48-56. Retrieved March 11, 2006 from
EBSCOhost database.
Heath, M. (2005). Are
you ready to go digital? The pros and cons of electronic portfolio
development. Library Media Connection, 23(7), p66.
Hewett, Stephenie M.
(2004). Linking research & practice to improve learning.
TechTrends , 48 (5), p26-30, ( AN
14939353 ), Retrieved March 11, 2006 EBSCOhost database.
Jones, J. E. (1994).
Portfolio assessment as a strategy for self-direction in learning.
New Directions for Adult and Continuing Education64, p23.
Kish, Cheryl K.,
& Sheehan, Janet K. (1997). Portfolios in the classroom: A
vehicle for developing reflective thinking. High School
Journal , 80 (4), p254.
Kulik, C. (1994).
Commentary on research: Computer-based learning and integrated learning
systems. p. 4. Retrieved on July 18, 2006 from
www.hull.ac.uk/php/edskas/cbl_commentary.doc.
Murphy, S. (1998).
Reflection: in portfolios and beyond. Clearing House, 00098655,
72 (1) p. 7. Database: Academic Search Premier.
Niguidula, D. (2005).
Documenting learning with digital portfolios. Educational
Leadership , 63 (3), p44-47.
Roach, V. E. (2001).
Literature review on portfolios: literature, theory, applications and
such, Retrieved March 11, 2006 from
http://webfolios.home.netcom.com/Literature_Review/Literature_Review.html.
EBSCOhost database.
Rogers, G. M., &
Chow, T. (2000). Electronic portfolios and the assessment of student
learning. Assessment Update, 12(1) 4.
Rossi, R. &
Montgomery, A. (1994). Educational reforms and students at risk: a
review of the current state of the art, Editors American Institutes for
Research.
Wagner, L. &
Lilly, D (1999). Asking the Experts: Engaging students in
self-assessment and goal setting through the use of portfolios. Diagnostique25(1),
31-43.
Wenglinsky, H., (1998)
Does it compute? The relationship between educational technology and
student achievement in mathematics. Princeton, NJ: Educational Testing
Service, September 1998.
Wieseman, K. (2005).
Nature of Reflection in Electronic Portfolios: Elementary and Secondary
Science Preservice Teachers.
Wiggins, G. (1989).
Teaching to the (Authentic) Test. Educational Leadership,
46(7), p41.
Wiggins, G. (1989). A
true test: Toward more authentic and equitable assessment. Phi Delta
Kappan, 70(9) 703-713.
Wiggins, Grant (1990).
The case for authentic assessment. Practical Assessment,
Research & Evaluation, 2(2).
Wilcox, B. L. (1996).
Smart portfolios for teachers in training. Journal of
Adolescent & Adult Literacy40(3): 172-79.
Wolf, K., and M. Dietz.
1998. Teaching portfolios: Purposes. Journal of Adolescent
& Adult Literacy40(3): 172-79.
Wolf, K., &
Siu-Runyan, Y. (1996). Portfolio purposes and possibilities.
Journal of
Adolescent & Adult Literacy , 40(1),
p30.
Wright, V.H. Challenges
of electronic portfolios: student perceptions and experiences. Journal
of Technology and Teacher Education, 10(1), p49-61.Zubizaretta, J. (2004). The
learning portfolio: Reflective practice for improving student learning.
Bolton, MA: Anker Publishing.
