Theatre Australia

your portal for australian theatre

Adjudicators, Bias and Criticism

Sat, 31 Jan 2004, 12:52 pm
Greg Ross9 posts in thread

After reading Ron Banks critique of Mike AnthonyÂ’s play in The West Australian (Saturday 31 January), I wondered how some of those who have expressed their anger about adjudication processes in recent weeks, would react to a similar methodical analysis. I suspect not well, for what appears to annoy the most, is the perception of personal bias. And yet, bias will always play a part in the judgment of any work in the arts.

I’m one of those bored to tears by the whole Irish thing, yet I consider Mike Anthony a good bloke and very talented. I also thoroughly enjoy time-out with the vampire books of Anne Rice, but literature critics don’t even bother to review them. I went to see “Japanese Story” in Kings Park last night. It was the second time I’d seen the film, as I like it so much. However, half a dozen people around me, loathed it and regretted coming.

WhoÂ’s right and who is wrong?

This leads to the arguments of those who would declare one set of theatre skills far more important than the other, which indirectly makes me smile, remembering an old joke about which was more important – the brain or the anus.

I am sure everyone is aware of the term “beautifully crafted.” To me, that phrase sums up the essence of an award winning play, musical, or film. There may be technical mistakes, there may be actors less skilled than others, however, the overall production is so good (how important does that make the director and technical crew?), that the production is clearly worthy of award. It will be interesting to watch how “Lord of the Rings” fares in the Oscars.

It is human nature to need acknowledgment and encouragement from families and peers, it’s vital to our psyche and our emotional growth. It’s also essential to learn from and understand our mistakes and perhaps just as important, to see things from another’s point of view – walk a mile in their shoes. Of course, criticism must always be given with the intention of helping someone to improve.

Where then for the process of adjudication? Some have asked for access to adjudication meetings and notes, but for what purpose? Although the adjudication process involves critiques of productions, the end-design of an award system is not the release and discussion of individual judgements of a production, rather the sum of judgements of a selection of productions.

It would appear, that with the very best of intentions, the Finley Award system has accidentally become unwieldy, in terms of sheer numbers of adjudicators and their ability (in the fullest sense of the word), to properly review every production. As more and more theatre groups seek sponsorship money and in consideration of the many people in community theatre who desperately wish to become professionals, itÂ’s obvious that the adjudication system must become more professional.

There is, for instance, no place for someone like myself seeking to adjudicate musicals, I’m biased, I don’t like them. Come to think of it, there’s no place for me adjudicating anything – I don’t have the expertise. However, there is equally, no place for an adjudicator who “only does musicals,” or falls asleep half way through a production. It would seem obvious that the adjudicator system needs revamping to be based on a group of people, with a well rounded, balanced and experienced understanding of theatre and the various crafts within.

Possibly this would entail a core group of four adjudicators, with two or three reserves. The adjudicators would be required to see all entered productions, (not necessarily together), with a reserve taking someoneÂ’s place in case of illness etc. Naturally an adjudicator would not be able to produce or direct a play entered for the FinleyÂ’s during their term of adjudication and would have to declare any interest in a particular club / production and step down from adjudicating that particular event. Equally, if a club kicks up a fuss about giving tickets to the adjudicators, they simply arenÂ’t adjudicated.

Of course, as with any organisation, there are always those frightened of change, or unwilling to lose control, not to mention the fear of that tap on the shoulder. So whilst the necessary changes may not be instantly achieved, they will happen. There are some very experienced, thoughtful and determined people currently serving as adjudicators, ITA committee members and interested theatre people, who have been and are putting enormous effort into bringing about the changes people are rightly clamouring for. IÂ’m prepared to bet that the accolades for next years Finley Awards will have the ITA Committee members in tears of joy.
Cheers
Greg Ross

NB: These are my private thoughts and are in no way meant to represent those of other ITA committee members.

Thread (9 posts)

Greg RossSat, 31 Jan 2004, 12:52 pm

After reading Ron Banks critique of Mike AnthonyÂ’s play in The West Australian (Saturday 31 January), I wondered how some of those who have expressed their anger about adjudication processes in recent weeks, would react to a similar methodical analysis. I suspect not well, for what appears to annoy the most, is the perception of personal bias. And yet, bias will always play a part in the judgment of any work in the arts.

I’m one of those bored to tears by the whole Irish thing, yet I consider Mike Anthony a good bloke and very talented. I also thoroughly enjoy time-out with the vampire books of Anne Rice, but literature critics don’t even bother to review them. I went to see “Japanese Story” in Kings Park last night. It was the second time I’d seen the film, as I like it so much. However, half a dozen people around me, loathed it and regretted coming.

WhoÂ’s right and who is wrong?

This leads to the arguments of those who would declare one set of theatre skills far more important than the other, which indirectly makes me smile, remembering an old joke about which was more important – the brain or the anus.

I am sure everyone is aware of the term “beautifully crafted.” To me, that phrase sums up the essence of an award winning play, musical, or film. There may be technical mistakes, there may be actors less skilled than others, however, the overall production is so good (how important does that make the director and technical crew?), that the production is clearly worthy of award. It will be interesting to watch how “Lord of the Rings” fares in the Oscars.

It is human nature to need acknowledgment and encouragement from families and peers, it’s vital to our psyche and our emotional growth. It’s also essential to learn from and understand our mistakes and perhaps just as important, to see things from another’s point of view – walk a mile in their shoes. Of course, criticism must always be given with the intention of helping someone to improve.

Where then for the process of adjudication? Some have asked for access to adjudication meetings and notes, but for what purpose? Although the adjudication process involves critiques of productions, the end-design of an award system is not the release and discussion of individual judgements of a production, rather the sum of judgements of a selection of productions.

It would appear, that with the very best of intentions, the Finley Award system has accidentally become unwieldy, in terms of sheer numbers of adjudicators and their ability (in the fullest sense of the word), to properly review every production. As more and more theatre groups seek sponsorship money and in consideration of the many people in community theatre who desperately wish to become professionals, itÂ’s obvious that the adjudication system must become more professional.

There is, for instance, no place for someone like myself seeking to adjudicate musicals, I’m biased, I don’t like them. Come to think of it, there’s no place for me adjudicating anything – I don’t have the expertise. However, there is equally, no place for an adjudicator who “only does musicals,” or falls asleep half way through a production. It would seem obvious that the adjudicator system needs revamping to be based on a group of people, with a well rounded, balanced and experienced understanding of theatre and the various crafts within.

Possibly this would entail a core group of four adjudicators, with two or three reserves. The adjudicators would be required to see all entered productions, (not necessarily together), with a reserve taking someoneÂ’s place in case of illness etc. Naturally an adjudicator would not be able to produce or direct a play entered for the FinleyÂ’s during their term of adjudication and would have to declare any interest in a particular club / production and step down from adjudicating that particular event. Equally, if a club kicks up a fuss about giving tickets to the adjudicators, they simply arenÂ’t adjudicated.

Of course, as with any organisation, there are always those frightened of change, or unwilling to lose control, not to mention the fear of that tap on the shoulder. So whilst the necessary changes may not be instantly achieved, they will happen. There are some very experienced, thoughtful and determined people currently serving as adjudicators, ITA committee members and interested theatre people, who have been and are putting enormous effort into bringing about the changes people are rightly clamouring for. IÂ’m prepared to bet that the accolades for next years Finley Awards will have the ITA Committee members in tears of joy.
Cheers
Greg Ross

NB: These are my private thoughts and are in no way meant to represent those of other ITA committee members.
crgwllmsSat, 31 Jan 2004, 01:23 pm

Re: Definitions, Effectiveness, and Finleys

Greg Ross wrote:
>
> Of course, criticism must always be given with the
> intention of helping someone to improve.

A good point, and if we use this as a definition then a newspaper review is NOT criticism. A newspaper review of a theatre or film event is an (hopefully) informed OPINION, shared with the readership. There is no specific intent to contact the artists being reviewed and offer any suggestions for improvement...it doesn't pretend to be constructive criticism.

Neither, really, is an awards system. Winning or scoring points in an award category can only be taken in a particular context of comparison, and should not necessarily be held as a measure of worth, or as criticism.



> an old joke about which was more important – the brain or the anus.

So which IS?
Greg RossSat, 31 Jan 2004, 01:47 pm

Re: Definitions, Effectiveness, and Finleys

Greetings Craig

Oh dear Â… I can only answer truthfully!

People have far more often referred to me as the anus and rarely mentioned anything about a brain. And unfortunately, some these days, also refer to me as an old joke, but of course, thatÂ’s only their opinion!
Cheers
Greg
Grant MalcolmMon, 2 Feb 2004, 10:18 pm

Re: Adjudication: due process

Hi Greg

Thanks for the thoughts.

Greg Ross wrote:
> Where then for the process of adjudication? Some have asked
> for access to adjudication meetings and notes, but for what
> purpose?

I guess this request might have arisen elsewhere, or did I miss that post?

> Although the adjudication process involves critiques
> of productions, the end-design of an award system is not the
> release and discussion of individual judgements of a
> production, rather the sum of judgements of a selection of
> productions.

I think in a peer adjudicated, community-based awards scheme if the only outcome is a summary list of judgements, a great opportunity has been missed.

You mentioned earlier in this post:

> ItÂ’s also essential to learn from and
> understand our mistakes and perhaps just as important, to see
> things from another’s point of view – walk a mile in their
> shoes. Of course, criticism must always be given with the
> intention of helping someone to improve.

I think an adjudication system that is open to peer review should offer precisely these opportunities - amongst the people privileged to be adjudicating. Through the adjudicators discussing the marks awarded, hearing things from another's point of view - walking a mile in their shoes - and encouraging each other with helpful criticism its entirely possible for the adjudication process to yield rewards well beyond a paltry list of winners.

> It would appear, that with the very best of intentions, the
> Finley Award system has accidentally become unwieldy, in
> terms of sheer numbers of adjudicators and their ability (in
> the fullest sense of the word), to properly review every
> production.

How does the number of adjudicators adversely impact on the quality of the results obtained? I fail to see why a process that adequately involves a dozen people shouldn't return as satisfactory, if not more satisfactory, results than one with only four or so.

Maybe Morgans, Nielsen and a lot of other analysts could save a fortune by adopting this approach? In fact, let's have a word to the Australian Electoral Office about the next election!

;-)

There used to be an adjudication system in place that relied on a small team of three professional (i.e. "paid") adjudicators. My understanding is that the small number of adjudicators was considered by many to be an issue as there was a perception that this opened the process to bias when such a small number of points of view were considered.

> As more and more theatre groups seek sponsorship
> money and in consideration of the many people in community
> theatre who desperately wish to become professionals, itÂ’s
> obvious that the adjudication system must become more
> professional.

I'm not sure that I appreciate the distinction being made here. Professional is not a synonym for quality any more than amateur or community are its antonyms.

Accountability is a word that's been bandied around and with reference to an adjudicator being accountable to their fellow adjudicators for the marks awarded, I think it makes sense.

> However,
> there is equally, no place for an adjudicator who “only does
> musicals,”

Why?

Is there no place for the tone deaf, pigeon-toed adjudicator who only judges straight plays and not musicals?

I'm not sure of my own position on this one, but it does appear to me, returning to my refrain, that an adjudication process that opens all judgments to discussion and debate amongst the adjudicators might even out some of the wilder marks awarded and illuminate (if not eliminate) a few adjudicators along the way.

> It
> would seem obvious that the adjudicator system needs
> revamping to be based on a group of people, with a well
> rounded, balanced and experienced understanding of theatre
> and the various crafts within.

Two issues here. Firstly I don't believe the current selection process does much at all to ensure this happens. But neither do I subscribe to the view that a focus on attracting/retaining quality adjudicators need necessarily result in a reduction in the number of people appropriate for consideration as an adjudicator.

I wouldn't want to see an over-cautious approach to adjudicator quality reduce the number of adjudicators to the point where the perception is that a tiny clique are making all the decisions. This negates any accountability that might arise from marks being debated and discussed.

Keep the numbers of adjudicators up so that you have a flow of people gaining invaluable experience and being skilled up as adjudicators otherwise you might end up heavily reliant on a very, very small group of over-committed people preceived to be unrepresentative of the broader theatre community.

> Possibly this would entail a core group of four adjudicators,
> with two or three reserves. The adjudicators would be
> required to see all entered productions, (not necessarily
> together), with a reserve taking someoneÂ’s place in case of
> illness etc.

See my reservations above regarding reducing the number adjudicators to these sorts of numbers.

I'm not in the least hung up on this business of adjudicators having to see each production. Apart from being quite impractical, it places undue emphasis on a person's ability to undertake such a monumental task. Frankly I'd rather not have to be in the situation of choosing adjudicators based almost solely on the question of whether or not they're prepared to commit to seeing 40-50 shows entered in the awards each year. Inevitably we'll end up excluding people who might make an invaluable contribution through their adjudication of only 20-25.

> Naturally an adjudicator would not be able to
> produce or direct a play entered for the FinleyÂ’s during
> their term of adjudication and would have to declare any
> interest in a particular club / production and step down from
> adjudicating that particular event.

I understood this was already the case?

> Equally, if a club kicks
> up a fuss about giving tickets to the adjudicators, they
> simply arenÂ’t adjudicated.

Eh? Do tell!

A club refuses to be adjudicated by a particular adjudicator? This one is out of left field. Has this happened in the past? Is the perception of bias or incompetence in some adjudicators that strong?

> There are some very experienced, thoughtful and
> determined people currently serving as adjudicators, ITA
> committee members and interested theatre people, who have
> been and are putting enormous effort into bringing about the
> changes people are rightly clamouring for. IÂ’m prepared to
> bet that the accolades for next years Finley Awards will have
> the ITA Committee members in tears of joy.

Reduce the number of adjudicators to four? Even less accountability and openess? Less tangible rewards back to member companies through broader participation in the adjudication process? Refusal to adjudicate those who don't like the changes??

I appreciate this may be a first draft idea but I think it needs to go back to the drawing board.

Cheers
Grant

[%sig%]
crgwllmsMon, 2 Feb 2004, 11:49 pm

Re: Adjudication multiplication

Grant Malcolm wrote:
>
> How does the number of adjudicators adversely impact on the
> quality of the results obtained? I fail to see why a process
> that adequately involves a dozen people shouldn't return as
> satisfactory, if not more satisfactory, results than one with
> only four or so.


I think I tend to agree. For the purpose of example, let's exaggerate and allocate 100 people to be adjudicators for the year.
Assume that they all have some reasonable credentials and are probably keen theatre goers, possibly affiliated with a local club.

Allow for the rule that nobody can pass judgment on a show or a club they have personally been involved in.
(Do we really believe that if someone has been involved in a production, their bias is so great that they will automatically mark everybody else's production poorly? My tendency would be to allow a theatre practitioner to make observations and judgments concerning anyone else but their own shows...but I don't fully understand the politics and competitiveness behind these awards; perhaps some of the most suitable and experienced observers are going to be lost from the ranks of adjudicators, because of a perceived conflict of interest?)

We maybe now have 60 adjudicators who are eligible to assess every show.

Allow also for an attrition rate...it's just not possible for EVERYBODY to see EVERYTHING....
We maybe now have 20 who HAVE seen everything, but another 30 who have seen a great deal and are qualified to make comparative judgments.

Don't discount those who have fallen by the wayside because of only seeing two or three shows...those 50-odd can still submit reasonable criticism of the shows they DID see.

Out of the original 100 we have 20 who have attained 'seniority', meaning simply that they've seen everything. Their votes carry more weight. But on a sliding scale, we still have the other 80 who can contribute to the discussion in some way or another.

The marks are collated and averaged, and popular choices become apparent. Discrepancies that arise, from perceptions about the way marks were allocated, are made visible to the whole group, to discuss, justify, and modify if necessary, until a reasonable conclusion is formed.
This would include adjudicators with a 'specialist' interest, say Broadway musicals, or Elizabethan tragedies, being able to have a degree of influence over the marks allocated to a particular style of play, noting whether or not various aspects were executed well.

There would be the inevitable impossibility of getting everyone to agree, but with the influence of those with 'seniority', and the logical sense of 'majority rule', I reckon you'd get pretty close to having results that were generally agreed upon.



My figure of 100 people was pulled out of the air for convenience, and my smaller figures are not meant to represent any sort of accurate percentage breakdown, merely a plausible scenario.

But I think I'd have more confidence, and less inclination to complain about the results, with a larger number rather than if there was a gang of four responsible for all the decisions. This includes happily allowing for the fact that not all the judges saw my play, or have any particular interest in my style of hybrid-cross-platform-politically-based-group-devised-artaudian-tragi-comic-spoken-poem-underwater theatre....or the few that I KNOW don't like me because I criticised their poor spelling...



Cheers,
Craig
Walter PlingeTue, 3 Feb 2004, 07:54 am

Re: Adjudication multiplication

Hello all.

Posters interested in adjudication might like to read (free of charge) my one act play "Putting Back The Fest." It's set at the final night of a One Act Play Festival, the main character is an adjudicator, and the show debates some of the themes mentioned above.

"Fest.", which, as far as I know hasn't been produced in Perth, is published by Maverick Musicals -

www.mavmuse.com

Cheers,

Michael Jeffery
www.vicnet.net.au/~michaelj
KutchyeraTue, 3 Feb 2004, 11:28 am

Re: Adjudication multiplication

crgwllms wrote:
>
> For the purpose of example, let's
> exaggerate and allocate 100 people to be adjudicators for the
> year.
> Assume that they all have some reasonable credentials


Ah, but this is the issue, is it not? It doesn't matter how many people you include in your example, if they can't be held accountable then the system breaks down.

The reality is you're not going to find 100 people or even close. More like 10. And if it's a choice between 10 I don't have confidence in, or 4 I respect and trust, then the 'gang of four' will win.


Carolyn
Grant MalcolmSat, 7 Feb 2004, 06:48 pm

Re: Adjudication division

Kutchyera wrote:
> The reality is you're not going to find 100 people or even
> close. More like 10. And if it's a choice between 10 I don't
> have confidence in, or 4 I respect and trust, then the 'gang
> of four' will win.

Ah! But will I respect and trust the same four that you will? Chances are that even if I've heard of your four people, I'll have confidence in a very different group of four for very different reasons. Multiply this by the number of people and companies interested in the adjudication process and the gang of four look very unrepresentative.

Double or treble the gang of four and there's a far better chance that someone we all know and trust might be involved in the process.

Cheers
Grant

[%sig%]
Grant MalcolmSat, 7 Feb 2004, 11:25 pm

Re: Subtraction

d'oh!

Grant Malcolm wrote:
> Double or treble the gang of four and there's a far better
> chance that someone we all know and trust might be involved
> in the process.

Too much sun today setting up seating in the New Fortune Theatre*.

Last sentence should have read

"Double or treble the gang of four and there's a far better chance that we might all have someone we know and trust involved in the process."

Cheers
Grant

*Shameless plug for what will doubtless be a brilliant show in a unique venue:

http://theatre.asn.au/eventView.php3?event_id=3994

[%sig%]
← Back to Billboard Bulletins