Adjudicators, Bias and Criticism
Sat, 31 Jan 2004, 12:52 pmGreg Ross9 posts in thread
Adjudicators, Bias and Criticism
Sat, 31 Jan 2004, 12:52 pmAfter reading Ron Banks critique of Mike AnthonyÂ’s play in The West Australian (Saturday 31 January), I wondered how some of those who have expressed their anger about adjudication processes in recent weeks, would react to a similar methodical analysis. I suspect not well, for what appears to annoy the most, is the perception of personal bias. And yet, bias will always play a part in the judgment of any work in the arts.
I’m one of those bored to tears by the whole Irish thing, yet I consider Mike Anthony a good bloke and very talented. I also thoroughly enjoy time-out with the vampire books of Anne Rice, but literature critics don’t even bother to review them. I went to see “Japanese Story” in Kings Park last night. It was the second time I’d seen the film, as I like it so much. However, half a dozen people around me, loathed it and regretted coming.
WhoÂ’s right and who is wrong?
This leads to the arguments of those who would declare one set of theatre skills far more important than the other, which indirectly makes me smile, remembering an old joke about which was more important – the brain or the anus.
I am sure everyone is aware of the term “beautifully crafted.” To me, that phrase sums up the essence of an award winning play, musical, or film. There may be technical mistakes, there may be actors less skilled than others, however, the overall production is so good (how important does that make the director and technical crew?), that the production is clearly worthy of award. It will be interesting to watch how “Lord of the Rings” fares in the Oscars.
It is human nature to need acknowledgment and encouragement from families and peers, it’s vital to our psyche and our emotional growth. It’s also essential to learn from and understand our mistakes and perhaps just as important, to see things from another’s point of view – walk a mile in their shoes. Of course, criticism must always be given with the intention of helping someone to improve.
Where then for the process of adjudication? Some have asked for access to adjudication meetings and notes, but for what purpose? Although the adjudication process involves critiques of productions, the end-design of an award system is not the release and discussion of individual judgements of a production, rather the sum of judgements of a selection of productions.
It would appear, that with the very best of intentions, the Finley Award system has accidentally become unwieldy, in terms of sheer numbers of adjudicators and their ability (in the fullest sense of the word), to properly review every production. As more and more theatre groups seek sponsorship money and in consideration of the many people in community theatre who desperately wish to become professionals, itÂ’s obvious that the adjudication system must become more professional.
There is, for instance, no place for someone like myself seeking to adjudicate musicals, I’m biased, I don’t like them. Come to think of it, there’s no place for me adjudicating anything – I don’t have the expertise. However, there is equally, no place for an adjudicator who “only does musicals,” or falls asleep half way through a production. It would seem obvious that the adjudicator system needs revamping to be based on a group of people, with a well rounded, balanced and experienced understanding of theatre and the various crafts within.
Possibly this would entail a core group of four adjudicators, with two or three reserves. The adjudicators would be required to see all entered productions, (not necessarily together), with a reserve taking someoneÂ’s place in case of illness etc. Naturally an adjudicator would not be able to produce or direct a play entered for the FinleyÂ’s during their term of adjudication and would have to declare any interest in a particular club / production and step down from adjudicating that particular event. Equally, if a club kicks up a fuss about giving tickets to the adjudicators, they simply arenÂ’t adjudicated.
Of course, as with any organisation, there are always those frightened of change, or unwilling to lose control, not to mention the fear of that tap on the shoulder. So whilst the necessary changes may not be instantly achieved, they will happen. There are some very experienced, thoughtful and determined people currently serving as adjudicators, ITA committee members and interested theatre people, who have been and are putting enormous effort into bringing about the changes people are rightly clamouring for. IÂ’m prepared to bet that the accolades for next years Finley Awards will have the ITA Committee members in tears of joy.
Cheers
Greg Ross
NB: These are my private thoughts and are in no way meant to represent those of other ITA committee members.
Re: Adjudication: due process
Mon, 2 Feb 2004, 10:18 pmHi Greg
Thanks for the thoughts.
Greg Ross wrote:
> Where then for the process of adjudication? Some have asked
> for access to adjudication meetings and notes, but for what
> purpose?
I guess this request might have arisen elsewhere, or did I miss that post?
> Although the adjudication process involves critiques
> of productions, the end-design of an award system is not the
> release and discussion of individual judgements of a
> production, rather the sum of judgements of a selection of
> productions.
I think in a peer adjudicated, community-based awards scheme if the only outcome is a summary list of judgements, a great opportunity has been missed.
You mentioned earlier in this post:
> ItÂ’s also essential to learn from and
> understand our mistakes and perhaps just as important, to see
> things from another’s point of view – walk a mile in their
> shoes. Of course, criticism must always be given with the
> intention of helping someone to improve.
I think an adjudication system that is open to peer review should offer precisely these opportunities - amongst the people privileged to be adjudicating. Through the adjudicators discussing the marks awarded, hearing things from another's point of view - walking a mile in their shoes - and encouraging each other with helpful criticism its entirely possible for the adjudication process to yield rewards well beyond a paltry list of winners.
> It would appear, that with the very best of intentions, the
> Finley Award system has accidentally become unwieldy, in
> terms of sheer numbers of adjudicators and their ability (in
> the fullest sense of the word), to properly review every
> production.
How does the number of adjudicators adversely impact on the quality of the results obtained? I fail to see why a process that adequately involves a dozen people shouldn't return as satisfactory, if not more satisfactory, results than one with only four or so.
Maybe Morgans, Nielsen and a lot of other analysts could save a fortune by adopting this approach? In fact, let's have a word to the Australian Electoral Office about the next election!
;-)
There used to be an adjudication system in place that relied on a small team of three professional (i.e. "paid") adjudicators. My understanding is that the small number of adjudicators was considered by many to be an issue as there was a perception that this opened the process to bias when such a small number of points of view were considered.
> As more and more theatre groups seek sponsorship
> money and in consideration of the many people in community
> theatre who desperately wish to become professionals, itÂ’s
> obvious that the adjudication system must become more
> professional.
I'm not sure that I appreciate the distinction being made here. Professional is not a synonym for quality any more than amateur or community are its antonyms.
Accountability is a word that's been bandied around and with reference to an adjudicator being accountable to their fellow adjudicators for the marks awarded, I think it makes sense.
> However,
> there is equally, no place for an adjudicator who “only does
> musicals,”
Why?
Is there no place for the tone deaf, pigeon-toed adjudicator who only judges straight plays and not musicals?
I'm not sure of my own position on this one, but it does appear to me, returning to my refrain, that an adjudication process that opens all judgments to discussion and debate amongst the adjudicators might even out some of the wilder marks awarded and illuminate (if not eliminate) a few adjudicators along the way.
> It
> would seem obvious that the adjudicator system needs
> revamping to be based on a group of people, with a well
> rounded, balanced and experienced understanding of theatre
> and the various crafts within.
Two issues here. Firstly I don't believe the current selection process does much at all to ensure this happens. But neither do I subscribe to the view that a focus on attracting/retaining quality adjudicators need necessarily result in a reduction in the number of people appropriate for consideration as an adjudicator.
I wouldn't want to see an over-cautious approach to adjudicator quality reduce the number of adjudicators to the point where the perception is that a tiny clique are making all the decisions. This negates any accountability that might arise from marks being debated and discussed.
Keep the numbers of adjudicators up so that you have a flow of people gaining invaluable experience and being skilled up as adjudicators otherwise you might end up heavily reliant on a very, very small group of over-committed people preceived to be unrepresentative of the broader theatre community.
> Possibly this would entail a core group of four adjudicators,
> with two or three reserves. The adjudicators would be
> required to see all entered productions, (not necessarily
> together), with a reserve taking someoneÂ’s place in case of
> illness etc.
See my reservations above regarding reducing the number adjudicators to these sorts of numbers.
I'm not in the least hung up on this business of adjudicators having to see each production. Apart from being quite impractical, it places undue emphasis on a person's ability to undertake such a monumental task. Frankly I'd rather not have to be in the situation of choosing adjudicators based almost solely on the question of whether or not they're prepared to commit to seeing 40-50 shows entered in the awards each year. Inevitably we'll end up excluding people who might make an invaluable contribution through their adjudication of only 20-25.
> Naturally an adjudicator would not be able to
> produce or direct a play entered for the FinleyÂ’s during
> their term of adjudication and would have to declare any
> interest in a particular club / production and step down from
> adjudicating that particular event.
I understood this was already the case?
> Equally, if a club kicks
> up a fuss about giving tickets to the adjudicators, they
> simply arenÂ’t adjudicated.
Eh? Do tell!
A club refuses to be adjudicated by a particular adjudicator? This one is out of left field. Has this happened in the past? Is the perception of bias or incompetence in some adjudicators that strong?
> There are some very experienced, thoughtful and
> determined people currently serving as adjudicators, ITA
> committee members and interested theatre people, who have
> been and are putting enormous effort into bringing about the
> changes people are rightly clamouring for. IÂ’m prepared to
> bet that the accolades for next years Finley Awards will have
> the ITA Committee members in tears of joy.
Reduce the number of adjudicators to four? Even less accountability and openess? Less tangible rewards back to member companies through broader participation in the adjudication process? Refusal to adjudicate those who don't like the changes??
I appreciate this may be a first draft idea but I think it needs to go back to the drawing board.
Cheers
Grant
[%sig%]
Thanks for the thoughts.
Greg Ross wrote:
> Where then for the process of adjudication? Some have asked
> for access to adjudication meetings and notes, but for what
> purpose?
I guess this request might have arisen elsewhere, or did I miss that post?
> Although the adjudication process involves critiques
> of productions, the end-design of an award system is not the
> release and discussion of individual judgements of a
> production, rather the sum of judgements of a selection of
> productions.
I think in a peer adjudicated, community-based awards scheme if the only outcome is a summary list of judgements, a great opportunity has been missed.
You mentioned earlier in this post:
> ItÂ’s also essential to learn from and
> understand our mistakes and perhaps just as important, to see
> things from another’s point of view – walk a mile in their
> shoes. Of course, criticism must always be given with the
> intention of helping someone to improve.
I think an adjudication system that is open to peer review should offer precisely these opportunities - amongst the people privileged to be adjudicating. Through the adjudicators discussing the marks awarded, hearing things from another's point of view - walking a mile in their shoes - and encouraging each other with helpful criticism its entirely possible for the adjudication process to yield rewards well beyond a paltry list of winners.
> It would appear, that with the very best of intentions, the
> Finley Award system has accidentally become unwieldy, in
> terms of sheer numbers of adjudicators and their ability (in
> the fullest sense of the word), to properly review every
> production.
How does the number of adjudicators adversely impact on the quality of the results obtained? I fail to see why a process that adequately involves a dozen people shouldn't return as satisfactory, if not more satisfactory, results than one with only four or so.
Maybe Morgans, Nielsen and a lot of other analysts could save a fortune by adopting this approach? In fact, let's have a word to the Australian Electoral Office about the next election!
;-)
There used to be an adjudication system in place that relied on a small team of three professional (i.e. "paid") adjudicators. My understanding is that the small number of adjudicators was considered by many to be an issue as there was a perception that this opened the process to bias when such a small number of points of view were considered.
> As more and more theatre groups seek sponsorship
> money and in consideration of the many people in community
> theatre who desperately wish to become professionals, itÂ’s
> obvious that the adjudication system must become more
> professional.
I'm not sure that I appreciate the distinction being made here. Professional is not a synonym for quality any more than amateur or community are its antonyms.
Accountability is a word that's been bandied around and with reference to an adjudicator being accountable to their fellow adjudicators for the marks awarded, I think it makes sense.
> However,
> there is equally, no place for an adjudicator who “only does
> musicals,”
Why?
Is there no place for the tone deaf, pigeon-toed adjudicator who only judges straight plays and not musicals?
I'm not sure of my own position on this one, but it does appear to me, returning to my refrain, that an adjudication process that opens all judgments to discussion and debate amongst the adjudicators might even out some of the wilder marks awarded and illuminate (if not eliminate) a few adjudicators along the way.
> It
> would seem obvious that the adjudicator system needs
> revamping to be based on a group of people, with a well
> rounded, balanced and experienced understanding of theatre
> and the various crafts within.
Two issues here. Firstly I don't believe the current selection process does much at all to ensure this happens. But neither do I subscribe to the view that a focus on attracting/retaining quality adjudicators need necessarily result in a reduction in the number of people appropriate for consideration as an adjudicator.
I wouldn't want to see an over-cautious approach to adjudicator quality reduce the number of adjudicators to the point where the perception is that a tiny clique are making all the decisions. This negates any accountability that might arise from marks being debated and discussed.
Keep the numbers of adjudicators up so that you have a flow of people gaining invaluable experience and being skilled up as adjudicators otherwise you might end up heavily reliant on a very, very small group of over-committed people preceived to be unrepresentative of the broader theatre community.
> Possibly this would entail a core group of four adjudicators,
> with two or three reserves. The adjudicators would be
> required to see all entered productions, (not necessarily
> together), with a reserve taking someoneÂ’s place in case of
> illness etc.
See my reservations above regarding reducing the number adjudicators to these sorts of numbers.
I'm not in the least hung up on this business of adjudicators having to see each production. Apart from being quite impractical, it places undue emphasis on a person's ability to undertake such a monumental task. Frankly I'd rather not have to be in the situation of choosing adjudicators based almost solely on the question of whether or not they're prepared to commit to seeing 40-50 shows entered in the awards each year. Inevitably we'll end up excluding people who might make an invaluable contribution through their adjudication of only 20-25.
> Naturally an adjudicator would not be able to
> produce or direct a play entered for the FinleyÂ’s during
> their term of adjudication and would have to declare any
> interest in a particular club / production and step down from
> adjudicating that particular event.
I understood this was already the case?
> Equally, if a club kicks
> up a fuss about giving tickets to the adjudicators, they
> simply arenÂ’t adjudicated.
Eh? Do tell!
A club refuses to be adjudicated by a particular adjudicator? This one is out of left field. Has this happened in the past? Is the perception of bias or incompetence in some adjudicators that strong?
> There are some very experienced, thoughtful and
> determined people currently serving as adjudicators, ITA
> committee members and interested theatre people, who have
> been and are putting enormous effort into bringing about the
> changes people are rightly clamouring for. IÂ’m prepared to
> bet that the accolades for next years Finley Awards will have
> the ITA Committee members in tears of joy.
Reduce the number of adjudicators to four? Even less accountability and openess? Less tangible rewards back to member companies through broader participation in the adjudication process? Refusal to adjudicate those who don't like the changes??
I appreciate this may be a first draft idea but I think it needs to go back to the drawing board.
Cheers
Grant
[%sig%]
- ···