I honestly can't really remember significantly learning from a previous submission - being honest I guess we were too focused on the score and the approximate 'grade band'. On some occasions, I may have engaged with the demonstrator to enquire about my mark and what I could do to improve it. Otherwise, it was more 'one-way traffic', a process not realising its learning potential. Traditionally, everything was summative, and that was the approach instilled across the board at that time.
Joining Higher Education & Initial Feedback goals.
When I joined the academic community in 2009, one of my early personal goals was to help my students to learn at every opportunity, and develop a feedback-centric, always-improving mindset. The potential was too immense to ignore. My own experience outlined above was a key driver for me with this goal. I approached with a focused gusto, providing copious amounts of feedback to each students, handwritten - with hours of enthusiastic time being put in. However, when I reflect now, I probably did what many do in the early stages of a teaching career, and go all in with an assumption you are doing the right thing. So what happened? Looking back, I recall hours and hours of feedback generation not fulfilling its purpose. The same mistakes were appearing week-on-week, by the same students despite my sustained and repeated efforts. Feedback uptake, and future actions from it were not working. I had to consider some sort of a complete systemic change around feedback.
I worked with students in a partnership approach to learn from their experiences. All of my research around this space were evaluated via online surveys or focus groups, to ensure I was hearing anonymous feedback. It was the only way I could improve what I was doing. With ethical approval acquired in advance, I was able to disseminate my findings at conferences and in publications.
In general, reviewing Figure 5, I have worked on individual approaches developing personalised feedback sheets for example, group approaches to generalise feedback for large groups, while starting to consider digital approaches, e.g. audio or video feedback, which I will detail and reflect on below.
As part of a National Forum supported project (the TEAM project I was involved in) aimed to develop technology enhanced assessed methods in practicals across science and health, it is worth noting one of the project's four themes centred specifically on 'digital feedback', while some of the others had feedback elements built in to them (such as electronic lab notebooks, pre-practical quizzes combined with app-based quizzes, rubrics etc.). When we reflected on the project, we realised that while we were focusing on assessment at the beginning, all our work was engaging with student feedback provision in parallel.
My other app of choice has been 'notability'. It's primarily a note taking app that works well on the iPad with the Apple pencil. As well as annotating PDF files submitted from assignments, I started recording audio comments on the app that integrated on the annotated PDF file. It gave me an opportunity to explain and expand on my comments - something that was concerning me with written feedback, i.e. was I explaining it enough with text? The audio and video help so much in this regard. However, one issue with this can become time. Over explaining can make feedback too long, and this can mean students won't engage. Turnitin has limited its feedback to 3 minutes, a feature that ensures you explain concisely while importantly controlling both your, and the student's, workload. Vital tip there! I will expand on my digital approaches further below.
Option 2: Engaging with Microsoft OneNote, I was able to design assessments and provide students with collaborative spaces for peer learning. A key element of this software package was the opportunities it provided around feedback. For example, I could click, record and place my audio comment at the relevant part of the assignment; I could annotate the submission with comments and with the iPad and stylus, could handwrite comments also. I found this helpful to link and connect comments with relevant aspects of the submission. Equally, the software allows typed text comments also. In addition to this, I worked on integrating rubric feedback in to the OneNote reports, to support learner feedback corresponding to specific criteria. Option 3: Another element I commenced was recording standalone audio files that I could send via e mail to students. I found this helpful for group project submissions as it supported the dialogic process I had established with the groups. After sharing audio feedback with teams, I could use class time to engage and discuss it further, continuing the conversation. Option 4: Engaging with our institution's virtual learning environment (Moodle), I developed a set of rubrics around assessing a skill or competency. Here, I was able to assess while providing immediate feedback via the rubric. Using an iPad and stylus, i was able to 'check' the relevant rubric boxes on each student's virtual learning environment based rubric. Before the student left the room, they could access their rubric feedback and grade. While very beneficial, I find rubrics alone, without personalised comments, can cause a disconnect in the feedback process. Don't get me wrong, they can work very well especially if co-created with students, but I feel they are complemented and enhanced by the additional incorporation of open comments from the educator. I found taking that approach more beneficial for the learner, as the rubric may not capture aspects that could support that specific individual. Option 5: I became a big fan of screencasts relatively early in my teaching career. I like various aspects of 'tech' and was keen to engage where it supported learning. I was able to make screencasts to complement my teaching. Over the years, I got more in to the video and audio quality of these, as well as finding new platforms for sharing my content. But I often found I needed to make my videos more engaging, to help the viewer. I was successful in applying for funding to support the development of customised sketch diagrams, and this brought my feedback videos to a new level. Equally, I began to invest in various software packages which facilitated mouse tracking. These simple approaches were very effective and supported ways I was providing feedback. I was able to engage with new ways to engage my students with the feedback I was providing. As we have now entered a sphere of hybrid/hyfelx/blended/online/remote learning etc., the expertise around feedback in addition to the grounding in literature, an evidence base, around feedback, I feel I am very aware of the importance and potential of feedback and hopefully can continue to build on my work and continue to improve how I support my students.
But what is feedback without uptake and action?
At this stage I have described my chronological journey enhancing feedback for learners. However, as I have often mentioned to date, how a student engages with feedback, and develops an action from it, is equally important to consider. My incremental marking system engaged learners with the mindset of improving based on engaging with, and acting on, the feedback received. The self-assessment worksheets engaged the students with reflection on their work, meaning they are considering their work as they submit. I found this change again impacts their mindset; they seemed to care more about their submission and were looking forward to seeing what I thought of it, reading through the comments. Positioning feedback review sessions and feedback dialogue opportunities in to class time can enhance this uptake and engagement further, while asking students to mention an element of feedback on a previous submission they acted on gets them in to the mindset of recognising how feedback can support their future work. It's worth knowing that all the effort you put in to feedback needs to be complemented by processes and systems to support its uptake. Engage with your learners, speak to them and most importantly listen to what they are saying about feedback. Together, both groups can ensure your feedback makes a big impact.
In my reflection piece below, I will review and consider some elements of where this feedback journey has taken me so far.
Supporting Feedback Enhancement with System Changes
Feedback needs attention and supports in place. For example, for first year students transitioning to higher education, explaining what feedback is and what its purpose is should be considered. We then need to ensure we develop consistent approaches across programmes to embed familiarity for learners. Reflecting, I feel a key approach that advanced uptake of my feedback was the incremental marking scheme, as it propelled students to seek ways to improve each submission - with the realisation that feedback facilitates this. Does Feedback Actually Work? Reflecting on my experiences, I feel feedback on its own only truly resonates with a small proportion of a class. Feedback needs to be more of a culture than a simple process. It needs a framework of support, consistent across modules and programmes. For example, first years transitioning in to Higher Level may need feedback to be explained to them in simple terms, with opportunities to realise its role and potential for learning. Educators need to ensure feedback is more facilitator or action focused, as opposed to content focused. How educators consider their feedback is important - is what is provided supporting future learning and improvement, or is it just fixing the mistakes, grammar or spelling errors? Feedback uptake needs to be promoted, and monitored, ensuring engagement and synthesis of the learning takes place. I found building in feedback review times helpful, as well as providing a task to students to highlight an aspect of the previous feedback received which they have taken on board. It makes them reflect, think and identify one element they are taking forward to improve. If this happens across the board, they identify several improvement areas. I am proud of my incremental marking system, and the impact it had on feedback engagement and uptake. Since its inception, I stopped seeing the same mistakes being made on subsequent assignments. So yes, reflecting, I still feel feedback can work, but for a better reach of uptake and action, elements and structures around it need to be considered. Pilot Power & Partnering with Students Advising the changes and innovations I put in place were pilots and collaborations with students to inform, reflect and evaluate processes. I learned quite quickly the power and importance of designing pilots initially with small groups. It built my confidence in what I was doing, while allowing an opportunity to trial and make improvements to elements too, changes that are student-informed and enhancing their learning. Dialogic Approaches to Feedback Like many elements of teaching, didactic or one-way traffic approaches can often restrain learning potential. Similarly with feedback, I have found building in opportunities to reflect and discuss feedback helpful. We often hear of feedback loops, and the term 'closing the loop' but I do like David Carless's description of a feedback spiral (Carless, 2018), promoting the life long learning element and the role feedback can play in this. I think this takes things to another, continuous learning, level and something that really impacted my way of thinking around feedback. Does Technology Actually Help Students with Feedback? Of course with technology, it's important not to just blindly follow a new trend. Remembering pedagogies and identifying an evidence base for interventions can support the success around technology. I feel technology should support the learning, not drive it. With regard to feedback, technology does provide so many additional avenues, that can benefit learners. We can ensure that accessible approaches to feedback can be provided, for example, video and audio feedback can be viewed online, with accompanying text transcripts allowing further engagement. For international learners, translation opportunities may support further. Personally, I've found feedback with technologies has enhanced my practice, allowing me to communicate and explain with my students in a clear way. I can use subsequent class or practical time to engage in feedback reflection and dialogue. In saying that, technology does have to be truly considered by programme teams. For example, students receiving feedback by various software platforms or apps across their modules, could become overwhelmed or bombarded by well-meaning educators. I really dream of developing some sort of consistency around this, as a learner overwhelmed by excessive feedback may actually avoid it. If I had to be critical of technology for feedback, or putting it a more positive way, aspects to ensure you consider in this space, it is important to ensure feedback also remains 'personalised'. With technology, there is the potential of 'colder' matter of fact feedback, even automated feedback, without personal elaboration and guidance. For example, a VLE rubric can provide feedback based on criteria along a scale, however the text is often generic and many don't add personalised comments on top of the rubric to support future actions around improvement. Technology has the potential to succeed in this space, but you have to make it work for you and your students. Putting yourself in their shoes can really ensure the benefits are truly met. Beating the bombardment? Answer = FEATS; bringing it all together - what an amazing model! While I am proud of my achievements around feedback, I still feel I have so much to yet improve on. Digital feedback sounds fantastic for students, but many don't put themselves in the position of the student being 'bombarded' by all these various types of feedback. While meant as a support, learners may find this overwhelming, unable to synthesise and act on the feedback across modules. It has to be brought together - it has to! I attended a session on feedback in our Institution, led by Dr. Naomi Winstone and Dr. Rob Nash. As part of this, they presented the University of Surrey's FEATS platform, co-created with students working with Dr. Winstone (see Figure 8 above). Here, students are able to categorise the feedback they receive in one place, meaning they can learn the aspects of their work requiring further attention and those in which they are excelling. It provides a platform to help synthesise the feedback, and allow learners to see the gaps they need to work on and fill. A superb innovation, so student centred yet staff can also use this to support their students further and even monitor implementation of feedback category areas. So, you can see, I still have a lot to accomplish! Workload One aspect of an educator's workload that has significantly changed in recent years has been around providing feedback. Looking back, it is an area that I spend a huge amount of time on. I provide feedback on all continuous assignments, projects and laboratory exercises, activities and reports. An immense amount of effort and time. It is important for me, and my colleagues, to manage this feedback workload yet still achieve the role of feedback in enhancing our students' learning experience and associated work. Could tweaking assessment be the key, across programmes? Can we have programme feedback? A lot to ponder on. Have I it All Mastered? Can I improve? While I feel I have made a significant contribution to enhancing feedback with my students, I still feel I have a lot to learn and implement. I would like to work more on my feedback becoming more actionable, and track its implementation across future assignment submissions. Ultimately, a big issue for me is the 'bringing together' of feedback across stages, or programmes, for students. It is a big initiative to consider though, one I feel needs cross-departmental buy-in. With many academics having their own approach to feedback, it may be difficult to bring it all together. However, with the new digital approaches and increased digital confidence amongst staff, perhaps getting a programme platform for tracking feedback approaches may be more feasible now. I'm currently working with the MyFeedback Moodle plugin to further investigate its potential in this space. There is more 'digital' feedback than ever at the moment, however I still get drawn to the potential for feedback bombardment to overwhelm students. It will be so important to put the learner at the centre of everything as we navigate a post-COVID future, and while assessment will be a major discussion point, feedback has to be part of these considerations also. I do see the approach to assessment can influence feedback directly. As we enter a post-COVID world, final exams could even become a thing of the past, with a more continuous approach to assessment. However, this has to be approached with caution as over-assessment of students could become an unsustainable reality. Synoptic, integrated assessment could represent the way to move forward, and in situations like this, an approach of collating feedback in a holistic way - it might end up being the only way forward yet! Walk the Talk! Finally, I commenced this post with a reflection on personal experience, and I will end with the same. In the summer of 2018, I did an Associate Certificate in Graphic Design as a PD opportunity. Interestingly, I was specifically praised by the course instructors for how I received, listened and acted on the feedback they provided. It was highlighted on a few occasions. So as a learner, I recognise the potential of feedback - and showing how much I value it, practice what I preach!
Citations mentioned in the text:
In my case, I was the facilitator for working group 5 of cohort 1, which was comprised of five Universities spread around Europe, each coming to the project with a different context yet similar priority areas and challenges, eager to share, collaborate and learn. I consider myself extremely fortunate to have worked on the EUA's LOTUS project, and to have met so may amazing leaders in the space of leading teaching and learning enhancement and change. During the process, I learned a great deal on several levels with one being around facilitating group meetings. Hence, I wanted to reflect and identify ten learnings around this that may be of help to others. You can find these outlined below! 1. The mood of the group is important. Establish, promote and sustain a collaborative, collegial and fun tone. Depending on the level of their role, people can spend their day jumping from meeting to meeting. Hence, when they come to your meeting, it's vitally important to realise you can set the tone and a mood based on the way you engage the group members. So throughout your meetings, consider how you can make this one count, make it stand out from normal meetings and to engage and connect your group members. Remember, in your role as facilitator, you can create and impact the tone of the meeting, and the mood in the camp. I prepared for my initial meeting with my international University colleagues by learning how to pronounce my 'hello and welcome' message in each of their languages. Lots of smiles all round that morning! Another aspect is humour - keep a fun tone, make your meeting one that is remembered (just to clarify I mean small moments of light humour, not a stand up routine!). 2. Ensure balance across all members Working groups are often made up of various members, many of whom won't know each other and may each have a wide variation of roles and experience. You can also add in all the various personalities to the mix. As a facilitator, it's good to be aware of this, as it's vital to ensure everyone has an equal opportunity to engage and contribute. If you allow one voice to dominate, it may cause others to disengage, even sub-consciously. In the case of my group, I was fortunate as we consistently maintained a very equal contribution throughout, but it's something I was tuned in to from the onset. 3. Use technology to support brainstorming that in turn can fuel discussions; e.g. engage with menti, jamboard, miro, google docs etc. When facilitating a virtual meeting, there is an onus to mix up the content, engage group members and perhaps create ways to capture thoughts and reflections from the group, on top of open conversation. With advances in technology, there are many tools to support engagement, and if used in the right away, it can give everyone a voice, fuel subsequent discussions, capture points that may otherwise may not be mentioned, include all 'voices', and provide time for deep reflection and more. There are so many potential advantages. In my meetings, I regularly used shared google docs, google slides and menti and in each case I feel they added something extra to the meeting, and supported the acquisition of thoughts, opinions and reflections. The outputs provided a valuable platform to dissect and discuss amongst the group. Important not to over-do the technology also, ensure you use it where it truly supports the group and discussion, but not for the sake of having technology present. 4. Have a structure to meetings - especially if online ; consistently keep to time I was once told by my father that there are two key things to a meeting - the time it starts, and the time it finishes! Wise words! As a facilitator, you will be designing and finalising agendas and during the meeting, it's essential the meeting runs to time, for every meeting, all the time. If you are concerned about a busy agenda running over, pare it back and build in some Q&A or a short break. Should there be several speakers, ensure they know how much time they have, and that they (as well as you!) are using a timer of some sort. Bonus points for you if you identify a way to announce their time is up in a fun way (noise, toy, sounds, musical instrument etc. - again this can make them smile!). Ultimately, ensure each meeting is structured and kept to time throughout. 5. Share practice and approaches with other facilitators Communities of practice are keys to success, and while this may be your goal amongst your group members, don't ignore the community of practice amongst other facilitators. The EUA convened all the facilitators from the 5 working groups on several occasions to share experiences and further learn from each other. This can be both reassuring, inspiring and motivating. A professional development resource in itself! I was fortunate in that a National Forum colleague (Alison Farrell, a champion facilitator) was also a facilitator with another working group throughout the project, and sharing practices and ideas ahead of meetings was such a helpful approach to engage with. 6. Circulate meeting summaries regularly and punctually; agendas are needed well in advance Preparation and planning goes along way for group members; what did Roy Keane once say..."fail to prepare, prepare to fail"! Time has to be in your schedule to engage with other facilitators to compare plans and ideas, to design an inclusive agenda, and one that ensures the meeting has a way to achieve its goal. In industry terms, a high FTE goes in to these meetings. I circulated all my agendas one week in advance (we had pre-determined the best meeting date/time etc. via a doodle poll), and on this I included summary info. about the group as well as some pre-meeting work, with the latter being very light touch, present more to engage and frame the upcoming meeting. Following meetings, I converted my notes in to mindmaps, documents or powerpoint summaries and posted these to our group's 'shared space' online. I also highlighted key points, questions etc. in the summaries so group members could identify/relate back. I appreciate some meetings can be recorded to facilitate note-taking, but I wanted to ensure an open, trusted space was present in our meetings, hence I used the old fashioned hand written notes, capturing as much as I could while actively listening. Ensure to generate your summaries punctually after the meeting, so to maintain connection for group members - but also as this is when you are most familiar with the discussions, your shorthand notes taken and the key points which were made at the time. Hence a lot of your time facilitating a meeting goes in to the before & after the meeting work! Roy certainly does have a point with his advice :-) 7. Actively listen and synthesise, clarify as you go; build connections & reframe As meetings progress, and in the role of a facilitator, it can be helpful for everyone if you are consistently prepared to synthesise across the discussions, or to identify similarities and connections that arise across group members from points being made. You can identify the right person to bring in to the conversation at the right time, and this can really ensure everyone can have an opportunity to speak. Building in pauses, or reflection points, to summarise and reframe the discussion can also be of value. You can also use these to move things back to the main topic, in case people have ventured slightly off course via ongoing discussions. In addition, always allow some time in the agenda to 'close' the meeting, to bring the key topics discussed to the fore, determine any required actions, request any final thoughts or reflections from group members. It's vital not to be abruptly ending, you can lose a lot taking that approach. Consider employing a 'hold and share' approach at the end perhaps (if virtual; each person types their key take away point from the meeting in to their chat, but 'holds' off pressing 'send' until instructed). This allows thoughts to be captured without seeing others' opinions, which can sometimes away a conversation. Implementing this can provide a great discussion point to kick off your next meeting too! 8. Consider further opportunities for members to engage (e.g. breakout rooms) Like anyone who regularly attends online meetings, large groups can sometimes stifle engagement and openness. Therefore, considering what I outlined earlier about engaging with utilising technology, this can be further complemented by breakout rooms. Here, there is scope to allow members to meet in smaller numbers, to more openly discuss and share their viewpoints, before reporting back to the main group at a later stage. However, these sessions also require preparation. Questions must be clear to participants, and as facilitator, you must know how to notify the breakout room with the question/discussion point at hand and/or reminders informing how long left are also regularly required. Consider how you want them to document their discussion, via an elected rapporteur? perhaps via a shared google doc, with each group having a specific section?, or via a share google slide deck with each group work on a specific slide? The benefit of the latter, is at the end of the session, there is one complete slide deck with contributions from all breakout rooms. The rapporteur approach is useful if the meetings are tending to focus or rely more on a discussion/chat based approach back in the main room. 9. Remaining neutral So you are facilitating a working group, with several institutions being represented and you hear a specific point being made on topic that you have a wealth of experience on from your institution. What do you do? Firstly, it's important to realise that what works in one institution may not work in another. Also, each institution may be at a different stage on their path to T&L enhancement or may be limited by national policies or requirements - which you as facilitator may not be aware of. Hence, of course you may ask more questions to learn more, or mention a specific aspect of your expertise to support the group, but really you need to be mindful that you are not necessarily representing your institution when acting as a facilitator, so you need to allow time for each institution's context to come through, as opposed to believing you are doing something the one and only, best way (this could come across as boasting to some!). 10. Realising it can take time for groups to 'gel' and to see impact Sometimes over the course of a few meetings, you can begin to notice the group members begin to 'gel', connecting their points, challenges or priority areas more with those from other institutions as they speak. It occurs over time, and after a few meetings, I began to notice this more and more. For sure, we had reached this phase almost as the project was drawing to a close. Hence, embrace and appreciate once this phase is reached. When you consider that impact is often felt after the event/programme/project itself, you can see this connectivity between members as the platform for future impact. We created a shared contact sheet, with group members' details and areas of interest for future collaboration. Helpful relationships have been created, in a shared and open space, and I feel several future connections, partnerships and successes will further emanate from this group yet. Yes, we've seen some initial impact already, but there is a lot more to come yet as they return to their institutions following their involvement in the LOTUS project.
|
Ronan BreeEducation Developer,Science Lecturer, Archives
March 2023
Categories
All
Any opinions expressed here represent my own and not those of my employer.
|