I honestly can't really remember significantly learning from a previous submission - being honest I guess we were too focused on the score and the approximate 'grade band'. On some occasions, I may have engaged with the demonstrator to enquire about my mark and what I could do to improve it. Otherwise, it was more 'one-way traffic', a process not realising its learning potential. Traditionally, everything was summative, and that was the approach instilled across the board at that time.
Joining Higher Education & Initial Feedback goals.
When I joined the academic community in 2009, one of my early personal goals was to help my students to learn at every opportunity, and develop a feedback-centric, always-improving mindset. The potential was too immense to ignore. My own experience outlined above was a key driver for me with this goal. I approached with a focused gusto, providing copious amounts of feedback to each students, handwritten - with hours of enthusiastic time being put in. However, when I reflect now, I probably did what many do in the early stages of a teaching career, and go all in with an assumption you are doing the right thing. So what happened? Looking back, I recall hours and hours of feedback generation not fulfilling its purpose. The same mistakes were appearing week-on-week, by the same students despite my sustained and repeated efforts. Feedback uptake, and future actions from it were not working. I had to consider some sort of a complete systemic change around feedback.
I worked with students in a partnership approach to learn from their experiences. All of my research around this space were evaluated via online surveys or focus groups, to ensure I was hearing anonymous feedback. It was the only way I could improve what I was doing. With ethical approval acquired in advance, I was able to disseminate my findings at conferences and in publications.
In general, reviewing Figure 5, I have worked on individual approaches developing personalised feedback sheets for example, group approaches to generalise feedback for large groups, while starting to consider digital approaches, e.g. audio or video feedback, which I will detail and reflect on below.
As part of a National Forum supported project (the TEAM project I was involved in) aimed to develop technology enhanced assessed methods in practicals across science and health, it is worth noting one of the project's four themes centred specifically on 'digital feedback', while some of the others had feedback elements built in to them (such as electronic lab notebooks, pre-practical quizzes combined with app-based quizzes, rubrics etc.). When we reflected on the project, we realised that while we were focusing on assessment at the beginning, all our work was engaging with student feedback provision in parallel.
My other app of choice has been 'notability'. It's primarily a note taking app that works well on the iPad with the Apple pencil. As well as annotating PDF files submitted from assignments, I started recording audio comments on the app that integrated on the annotated PDF file. It gave me an opportunity to explain and expand on my comments - something that was concerning me with written feedback, i.e. was I explaining it enough with text? The audio and video help so much in this regard. However, one issue with this can become time. Over explaining can make feedback too long, and this can mean students won't engage. Turnitin has limited its feedback to 3 minutes, a feature that ensures you explain concisely while importantly controlling both your, and the student's, workload. Vital tip there! I will expand on my digital approaches further below.
Option 2: Engaging with Microsoft OneNote, I was able to design assessments and provide students with collaborative spaces for peer learning. A key element of this software package was the opportunities it provided around feedback. For example, I could click, record and place my audio comment at the relevant part of the assignment; I could annotate the submission with comments and with the iPad and stylus, could handwrite comments also. I found this helpful to link and connect comments with relevant aspects of the submission. Equally, the software allows typed text comments also. In addition to this, I worked on integrating rubric feedback in to the OneNote reports, to support learner feedback corresponding to specific criteria. Option 3: Another element I commenced was recording standalone audio files that I could send via e mail to students. I found this helpful for group project submissions as it supported the dialogic process I had established with the groups. After sharing audio feedback with teams, I could use class time to engage and discuss it further, continuing the conversation. Option 4: Engaging with our institution's virtual learning environment (Moodle), I developed a set of rubrics around assessing a skill or competency. Here, I was able to assess while providing immediate feedback via the rubric. Using an iPad and stylus, i was able to 'check' the relevant rubric boxes on each student's virtual learning environment based rubric. Before the student left the room, they could access their rubric feedback and grade. While very beneficial, I find rubrics alone, without personalised comments, can cause a disconnect in the feedback process. Don't get me wrong, they can work very well especially if co-created with students, but I feel they are complemented and enhanced by the additional incorporation of open comments from the educator. I found taking that approach more beneficial for the learner, as the rubric may not capture aspects that could support that specific individual. Option 5: I became a big fan of screencasts relatively early in my teaching career. I like various aspects of 'tech' and was keen to engage where it supported learning. I was able to make screencasts to complement my teaching. Over the years, I got more in to the video and audio quality of these, as well as finding new platforms for sharing my content. But I often found I needed to make my videos more engaging, to help the viewer. I was successful in applying for funding to support the development of customised sketch diagrams, and this brought my feedback videos to a new level. Equally, I began to invest in various software packages which facilitated mouse tracking. These simple approaches were very effective and supported ways I was providing feedback. I was able to engage with new ways to engage my students with the feedback I was providing. As we have now entered a sphere of hybrid/hyfelx/blended/online/remote learning etc., the expertise around feedback in addition to the grounding in literature, an evidence base, around feedback, I feel I am very aware of the importance and potential of feedback and hopefully can continue to build on my work and continue to improve how I support my students.
But what is feedback without uptake and action?
At this stage I have described my chronological journey enhancing feedback for learners. However, as I have often mentioned to date, how a student engages with feedback, and develops an action from it, is equally important to consider. My incremental marking system engaged learners with the mindset of improving based on engaging with, and acting on, the feedback received. The self-assessment worksheets engaged the students with reflection on their work, meaning they are considering their work as they submit. I found this change again impacts their mindset; they seemed to care more about their submission and were looking forward to seeing what I thought of it, reading through the comments. Positioning feedback review sessions and feedback dialogue opportunities in to class time can enhance this uptake and engagement further, while asking students to mention an element of feedback on a previous submission they acted on gets them in to the mindset of recognising how feedback can support their future work. It's worth knowing that all the effort you put in to feedback needs to be complemented by processes and systems to support its uptake. Engage with your learners, speak to them and most importantly listen to what they are saying about feedback. Together, both groups can ensure your feedback makes a big impact.
In my reflection piece below, I will review and consider some elements of where this feedback journey has taken me so far.
Supporting Feedback Enhancement with System Changes
Feedback needs attention and supports in place. For example, for first year students transitioning to higher education, explaining what feedback is and what its purpose is should be considered. We then need to ensure we develop consistent approaches across programmes to embed familiarity for learners. Reflecting, I feel a key approach that advanced uptake of my feedback was the incremental marking scheme, as it propelled students to seek ways to improve each submission - with the realisation that feedback facilitates this. Does Feedback Actually Work? Reflecting on my experiences, I feel feedback on its own only truly resonates with a small proportion of a class. Feedback needs to be more of a culture than a simple process. It needs a framework of support, consistent across modules and programmes. For example, first years transitioning in to Higher Level may need feedback to be explained to them in simple terms, with opportunities to realise its role and potential for learning. Educators need to ensure feedback is more facilitator or action focused, as opposed to content focused. How educators consider their feedback is important - is what is provided supporting future learning and improvement, or is it just fixing the mistakes, grammar or spelling errors? Feedback uptake needs to be promoted, and monitored, ensuring engagement and synthesis of the learning takes place. I found building in feedback review times helpful, as well as providing a task to students to highlight an aspect of the previous feedback received which they have taken on board. It makes them reflect, think and identify one element they are taking forward to improve. If this happens across the board, they identify several improvement areas. I am proud of my incremental marking system, and the impact it had on feedback engagement and uptake. Since its inception, I stopped seeing the same mistakes being made on subsequent assignments. So yes, reflecting, I still feel feedback can work, but for a better reach of uptake and action, elements and structures around it need to be considered. Pilot Power & Partnering with Students Advising the changes and innovations I put in place were pilots and collaborations with students to inform, reflect and evaluate processes. I learned quite quickly the power and importance of designing pilots initially with small groups. It built my confidence in what I was doing, while allowing an opportunity to trial and make improvements to elements too, changes that are student-informed and enhancing their learning. Dialogic Approaches to Feedback Like many elements of teaching, didactic or one-way traffic approaches can often restrain learning potential. Similarly with feedback, I have found building in opportunities to reflect and discuss feedback helpful. We often hear of feedback loops, and the term 'closing the loop' but I do like David Carless's description of a feedback spiral (Carless, 2018), promoting the life long learning element and the role feedback can play in this. I think this takes things to another, continuous learning, level and something that really impacted my way of thinking around feedback. Does Technology Actually Help Students with Feedback? Of course with technology, it's important not to just blindly follow a new trend. Remembering pedagogies and identifying an evidence base for interventions can support the success around technology. I feel technology should support the learning, not drive it. With regard to feedback, technology does provide so many additional avenues, that can benefit learners. We can ensure that accessible approaches to feedback can be provided, for example, video and audio feedback can be viewed online, with accompanying text transcripts allowing further engagement. For international learners, translation opportunities may support further. Personally, I've found feedback with technologies has enhanced my practice, allowing me to communicate and explain with my students in a clear way. I can use subsequent class or practical time to engage in feedback reflection and dialogue. In saying that, technology does have to be truly considered by programme teams. For example, students receiving feedback by various software platforms or apps across their modules, could become overwhelmed or bombarded by well-meaning educators. I really dream of developing some sort of consistency around this, as a learner overwhelmed by excessive feedback may actually avoid it. If I had to be critical of technology for feedback, or putting it a more positive way, aspects to ensure you consider in this space, it is important to ensure feedback also remains 'personalised'. With technology, there is the potential of 'colder' matter of fact feedback, even automated feedback, without personal elaboration and guidance. For example, a VLE rubric can provide feedback based on criteria along a scale, however the text is often generic and many don't add personalised comments on top of the rubric to support future actions around improvement. Technology has the potential to succeed in this space, but you have to make it work for you and your students. Putting yourself in their shoes can really ensure the benefits are truly met. Beating the bombardment? Answer = FEATS; bringing it all together - what an amazing model! While I am proud of my achievements around feedback, I still feel I have so much to yet improve on. Digital feedback sounds fantastic for students, but many don't put themselves in the position of the student being 'bombarded' by all these various types of feedback. While meant as a support, learners may find this overwhelming, unable to synthesise and act on the feedback across modules. It has to be brought together - it has to! I attended a session on feedback in our Institution, led by Dr. Naomi Winstone and Dr. Rob Nash. As part of this, they presented the University of Surrey's FEATS platform, co-created with students working with Dr. Winstone (see Figure 8 above). Here, students are able to categorise the feedback they receive in one place, meaning they can learn the aspects of their work requiring further attention and those in which they are excelling. It provides a platform to help synthesise the feedback, and allow learners to see the gaps they need to work on and fill. A superb innovation, so student centred yet staff can also use this to support their students further and even monitor implementation of feedback category areas. So, you can see, I still have a lot to accomplish! Workload One aspect of an educator's workload that has significantly changed in recent years has been around providing feedback. Looking back, it is an area that I spend a huge amount of time on. I provide feedback on all continuous assignments, projects and laboratory exercises, activities and reports. An immense amount of effort and time. It is important for me, and my colleagues, to manage this feedback workload yet still achieve the role of feedback in enhancing our students' learning experience and associated work. Could tweaking assessment be the key, across programmes? Can we have programme feedback? A lot to ponder on. Have I it All Mastered? Can I improve? While I feel I have made a significant contribution to enhancing feedback with my students, I still feel I have a lot to learn and implement. I would like to work more on my feedback becoming more actionable, and track its implementation across future assignment submissions. Ultimately, a big issue for me is the 'bringing together' of feedback across stages, or programmes, for students. It is a big initiative to consider though, one I feel needs cross-departmental buy-in. With many academics having their own approach to feedback, it may be difficult to bring it all together. However, with the new digital approaches and increased digital confidence amongst staff, perhaps getting a programme platform for tracking feedback approaches may be more feasible now. I'm currently working with the MyFeedback Moodle plugin to further investigate its potential in this space. There is more 'digital' feedback than ever at the moment, however I still get drawn to the potential for feedback bombardment to overwhelm students. It will be so important to put the learner at the centre of everything as we navigate a post-COVID future, and while assessment will be a major discussion point, feedback has to be part of these considerations also. I do see the approach to assessment can influence feedback directly. As we enter a post-COVID world, final exams could even become a thing of the past, with a more continuous approach to assessment. However, this has to be approached with caution as over-assessment of students could become an unsustainable reality. Synoptic, integrated assessment could represent the way to move forward, and in situations like this, an approach of collating feedback in a holistic way - it might end up being the only way forward yet! Walk the Talk! Finally, I commenced this post with a reflection on personal experience, and I will end with the same. In the summer of 2018, I did an Associate Certificate in Graphic Design as a PD opportunity. Interestingly, I was specifically praised by the course instructors for how I received, listened and acted on the feedback they provided. It was highlighted on a few occasions. So as a learner, I recognise the potential of feedback - and showing how much I value it, practice what I preach!
Citations mentioned in the text:
In recent days, I attended a session on ePortfolios (at ALT’s online winter conference), obviously a topic I feel close to at the moment considering my PD PACT experience. Having created three separate T&L ePortfolios now, each for different purposes, I was interested in hearing some other viewpoints. Shane Sutherland (Founder and CEO at PebblePad) mentioned a certain term around ePortfolios that I hadn't heard before: ‘DAPPER’. This term actually outlines their framework of six portfolio-like dimensions for people to consider the degree to which their ePortfolio demonstrates them:
I since found an excellent post by the presenter online that goes in to a bit more detail around each one of these posts, as well as in a Linkedin post he authored. Highly recommend reviewing the six dimensions and reflecting on your approach to your ePortfolio. I found the points might be worth considering when both starting an ePortfolio adventure, or even as you begin to reflect on your current one. Some other points mentioned were the concept that ePortfolios may be designed with different foci – I guess this depends on the role and goal of the ePortfolio, for example based around the person (Shane refers to these as Me-Portfolios), or about an activity or experience (Task-portfolios). Stay DAPPER! As illustrated in the figure above (from the PD framework), four types of learning are associated with PD learning activities.
Reflecting, I feel the PACT PD course activities that I have engaged with are helping me develop across a combination of the Framework’s ‘types of learning’. For example, I am learning from everyone each week, but particularly from the ‘mentoring’ and support from facilitators and my fellow triad members. Course participants have such a diverse skillset, primary discipline and range of expertise, yet the course feels more and more like a team taking a shared approach to learning, with everyone encouraging and supporting each other in a constructive way. The forum debates, and break-out rooms, have provided me with some ‘new learning’ while our triad activities and discussions have brought up new ways of approaching things, which has been a benefit for me to learn from. I feel many of the activities have allowed me to engage with ‘consolidated learning’ too; working through the domain self-assessment exercise and sitting to compose a reflection post on a trial blog reengaged the ‘reflective practice’ concept. Also, generating the ePortfolio, and discussions around this, has brought a range of previous learnings back to the fore. To conclude, I think the final ‘leading’ aspect is evident in all of us, in our PACT commitment. I feel each of us are learning by ‘leading’ too, with everyone developing innovative ways to reflect and capture our unique PD journeys, experimenting with various platforms and sharing experiences, all while doing plenty soul-searching reflections along the way! Identify a professional event that had an immediate significant effect on you. Select one that either encouraged or forced you to react immediately and/or to subsequently reflect on your practice. A few years ago, I attended a visiting speaker present in DkIT around assessment, a leader in the field who was in Ireland at the time (Prof. Chris Rust). He gave a lunchtime seminar to staff associated with the MA in Teaching and Learning. I have been to many seminars, events etc. over the years, but this one inspired me to reflect....and most importantly....act on my reflection. Describe how you felt when this event occurred. What immediate thoughts occurred to you? Immediately, I drew on the inspiration and motivation one can draw from attending a visiting speaker seminar. When I was working in science research, at NUIG, we had visiting national and international speakers almost on a weekly basis. This allowed networking, sharing of ideas and most importantly collaborations to form. It was almost part of the weekly schedule. So I did think back to that a bit, and perhaps how I missed this element (although now, with the way things are, and events such as the NF seminar series, we do have opportunities to virtually attend 'guest speaker' sessions). Referring to the event outlined in Q1, the value of self-assessment, and including this approach in assessment strategies, was discussed. My immediate thoughts were around the potential benefit for my students (of this approach being integrated), and how I could work on including an element of this in my assessment of practicals (a project I was working on at the time, introducing more formative approaches etc.). What action, if any, did you take when this event occurred? I designed a self-assessment sheet, and included it in the laboratory practical manual. Here, students had to self assess against a rubric and then identify certain features they were happy/not happy with regarding their submission. This form would be submitted with their lab reports. Examining this further, students fed back that it made them reflect and think about their work more - with some saying it made them feel guilty as they knew they could do better. Hearing these aspects, and knowing that students were reflecting and considering their work, and its quality, before submitting was great to hear. After the event was over, how did you reflect on the event and any actions you took? Would you do things differently now? What would you do? Overall, this event led to a benefit for my students, so that was the main aspect, i.e. that it had a positive impact. They were considering their own work before submitting, even identifying areas they would work to improve on. I still reflect on this event a lot, and consider it a trigger point for me with this element of assessment. With the introduction of the self-assessment forms, and other assessment aspects, I surveyed the class group and engaged with focus group discussions for feedback on it. This led to further iterations, but it is an element still used in the manual currently. I was proud I took an active approach to something from the seminar I attended, and am grateful to Prof. Rust for how he broke it down in to a simple approach to consider, with the self-assessment aspect really resonating with me. If I was starting again with this process from scratch, i would include a student partner group in developing the rubric and forms...this is something I am considering doing in the near future with the next iteration. “Life can only be understood backwards; but it must be lived forwards” Søren Kierkegaard Reflective practice is something I engage with regularly, in fact I realise I have actually been living the process for many years - unknowingly! I spent nearly ten years as a research scientist where you design experiments, carry them out before analysing - and reflecting - on the data/outcome. This reflection leads to the basis for designing the next steps to take. When I commenced my MA in T&L, I became much more in tune with reflective practice. We were asked to spend our commutes reflecting on how the day had gone, keep a reflective journal and discuss our progress with critical friends. These initiatives all facilitated the process of reflection....and from this, you determine a baseline from where you can improve. Reflect backwards to move forward! One interesting point in a zoom break out room made by a fellow PACT participant was that in the current global situation, without commutes, and falling from one online meeting in to another, there is less time to reflect in the way we used to. Perhaps it needs to become a small time slot we input in our schedules and actively engage with! |
Ronan BreeEducation Developer,Science Lecturer, Archives
March 2023
Categories
All
Any opinions expressed here represent my own and not those of my employer.
|