Ask the Experts, Question 3:

Why do award programs vary their criteria?

by Míc Miller, Webmaster, The Beeline
21 November 2001, update 7 March 03

Print Version

Karen Pimtzner, Webmistress petalperfect Digital Photography Gallery (site closed)

Editor's Choice

The Emmys (TV), Tonys (Stage), Oscars (Film), Grammys (Music) ... these are awards that we are all familiar with. Would we ever want to standardize the criteria for winning these awards? Of course not. Why? Because each of these awards has a different purpose and yes, different criteria to be sure.

However, more importantly, the criteria for these awards were created to measure "specific" achievements within "specific" segments of the "general" or "broader" Entertainment field. Web site awards are no different.

If one were to believe that Web site awards should be standardized; they, too, must also believe that the above-mentioned Entertainment Awards should be standardized. This is a very broad (and boring) generalization at best.

But, as their peer awards above, all "quality" Web site awards are as "individual" or "specific", as the programs that offer them. Each Web site award measures "specific" achievements in "specific" segments of general Web site design. I can see some of you shaking your heads and saying ... but, all of the criteria for these awards is very similar. Isn't that standardization?

Although a particular award may have "similar" criteria to another, their purpose and scope may be entirely different. The criteria is merely a bare-bones outline for the evaluation. For the award-seeker, it clearly states specific areas of Web site creation that will be evaluated. For the award-giver, it provides a set of guidelines for conducting the review.

However, the actual "review" of the Web site will be done by "individual" evaluators. And, although all of your quality award programs have evaluators possessing a well-balanced blend of "visual" expertise, "technical" expertise and "literary" expertise ... they also all have their own tastes, styles, likes, dislikes and "unique" approach to a review.

It is this "individuality" that is used to add the depth or humanity to that skeleton outline (or criteria).

Standardizing award programs would be like standardizing all of the individual evaluators' personalities. And we all know what would happen if this horrifying phenomenon were to occur. Remember the "Stepford Wives"?

Although it would certainly make it easier on an award-seeker to make application once and earn a myriad of high-quality awards, what would this say about the awards earned? And, how would it measure the talents of the webmaster? Would they be counted among a small, specialized group of talented individuals who had excelled at their craft (the Oscar)? Or, would they be lumped together in a larger group of individuals who managed, by some stroke of dumb luck, to create a Web site?

Míc Miller, Webmaster
Beehive Awards

Since Web awards — at least those from credible award programs — reward sites for their "Web excellence," one has to define what is "Web excellence." This is impossible to do. It is impossible to come up with any universally acceptable, adoptable, or definitive understanding when there are so many design-development parameters that are variables.

Language barriers alone prevent this from happening in this situation. Also, there could only be one type of Web site in order to even theoretically discuss "standardized criteria." And as we know, there are many types of Web sites, each with appropriately different design-development parameters for fit, form, function, and the like.

In fact, even the most well-defined, simplest type of Web site has enough variable parameters to prevent its genre from having one "optimal" solution.

For example, the intended uses and audiences a Web site needs to successfully serve and support will determine and directly affect the many ways a Web site will be created, maintained, and expanded upon. A community Web site needs to address a broad range of audiences, issues, and topics, as well as many technical specifications, while a personal Web site may simply need to satisfy its author as an educational and/or creative outlet.

Web award programs are like Web sites. Despite their audiences, purposes, values, beliefs, weights, interpretations, preferences, and many other factors, award programs with original, detailed criteria cannot be the same simply due to the individual award givers. The best one can hope for is standardization within an award program.

The beauty of the idea behind Web awards lies in their diversity and variety. Any Web site that can provide a "fertile" environment for as many people, browsers, platforms, devices, configurations, and such, as possible will likely be appreciated in kind by them — whether they are, and belong to, casual, first-time visitors or world-ranked award givers conducting intensive, in-depth site evaluations.

Any Web site can "win" some kind of Web award. There is no challenge in this and, therefore, no great claim in calling oneself an "award-winning" webmaster or webmistress. The challenge lies in finding a form of glory that can only come from having a Web site that has passed muster from a great number of critically based Web award programs that incorporate and encompass a broad range of non-standardized criterion.

If one thinks of each award as a different type of flower, Web sites can harvest bouquets, gardens, and even a field of symbols representing recognitions of "Web excellence" for a variety of criterion, requirements, accomplishments, judgments, and such.

If every award program used standardized criteria — which isn’t even theoretically possible due to the subjective aspects of award giving — the best a Web site could hope for is a bunch of, say, red roses, whose symbolic meaning might as well then be reduced to a single red rose. Who would want this? Particularly when one can have a field of dreams realized — and confirmed!

Heidi Walsh, Webmistress
Heidi's Website Awards (program closed)

That's a very good question indeed. I have asked myself this same question numerous times when I applied for awards. What a waste of time to go through page after page of criteria when all you want to do is get to the application form quickly.

Some award givers want clean HTML, others don't care, some prefer sound content, others demand outstanding design and every time you apply for an award you have to find out if your site actually meets the rather arbitrary criteria.

Are there advantages of having standardised criteria for the award seeker?

Of course, there are. First of all, there is the time factor. Applicants could apply for far more awards in a short time without having to read different criteria. Actually you wouldn't have to read them at all and you would just have to click on "application form," fill in your data and that's it. That way you could apply for about ten awards daily and there would still be some time left that you could spend with your family.

Secondly, standardised criteria would transmit reliability and security (so much needed in our instable world). The applicant would know exactly where he stood, a bronze award from the first program he applied at would mean he would get bronze awards from all the other programs as well. That wouldn't only add to inner balance and consistency, but would save the applicant considerable time, too.

Just think of all the long hours you spend organising your awards won pages. You divide them up into gold, silver and bronze pages, which wouldn't be necessary any more.

And what about the advantages of standardised criteria for the award giver?

Award givers spend countless hours changing and tweaking their criteria for two reasons. The first one is they want to attract great sites that they really want to award and secondly to reach a higher level at Award Sites! or even to be recognised as a World's Top Award. Standardised criteria would make life so much easier for the award giver.

Different levels or WTAs would be useless because all award programs would be the same. Instead of classifying programs, websites could be classified into gold, silver and bronze sites. Evaluators or even teams of judges would be unnecessary. Applicants would just give their classification in the application form and the award giver would send them the appropriate award image and that's it.

Just think of the many hours that award givers waste by actually looking at sites at the moment! No problem any more!

I beg your pardon? You actually say giving and seeking awards wouldn't be fun any more? You can't be serious. And what about the sites that would never win a bronze, silver or gold award with standardised criteria? Well, too bad, isn't it? ;)

Dave Selig, Webmaster
Ultraweb Awards

Criteria for award programs are not standardized for a variety of reasons.

Web sites reflect the different personalities, likes, dislikes, and interests of people from many different backgrounds. Attempting to establish a single set of criteria for all reviewers and Webmasters would result in too many square pegs trying to fit into round holes.

Many skilled and creative webmasters choosing not to conform to a universal site mold would be excluded from the "awards community." Award reviewers would have diminished authority to express their learned opinions, and would become review franchisees for a MacDonalds-sized universal award program.

The awards community (including givers and seekers) would experience an air of reduced creativity and increased boredom. This could not be remedied by applying a universal criterion that a Web site must exhibit creativity because this would prohibit the most simple and straightforward, yet informative sites from being eligible for an award, and prevent reviewers who appreciate such sites from exercising their true judgment.

Further, adopting one standard set of criteria would dissuade potentially outstanding award reviewers from operating worthwhile award programs. Some superb award programs are owned and operated by experts in particular subject areas who focus on the quality of a particular type of content rather than someone else's idea of a good Web site.

Those award programs help include more types of websites into the "awards community" and help to encourage better work in the particular subject fields.

If there were only one set of criteria, it would seem redundant to ever apply for more than one award. Having more than one award graphic for identical reasons would be trite and trivial. Currently, unique award graphics are meaningful because the graphical differences are interesting and more importantly because they represent the unique opinions of different and independent award programs.

In fact, only one award graphic would be needed for all franchisees choosing to participate in the standardized award corporation. And who would get to be the "corporate leader" of all the "franchises?"

I could go on and on to further discuss reasons, but I'll spare you any further ado, and conclude by saying that I think it’s a rotten idea to standardize award criteria across the board.

Best of luck with your unique award programs.

Debbie Sharp, Webmistress
Mesweet's Awards

When I read the question, the word that ran through my head was dictatorship. I shudder at the thought.

Having standardized criteria would take away the freedom of the award givers reason to offer the award to begin with. Uniqueness of an individual's awards would be no more. Awards would become boring because all awards given would be for the same reason.

It is a webmaster's right to offer an award and state what his or her criteria is, because only the webmaster knows for what reason he or she wants to give out the award. I like this freedom and feel that is the way it should remain.

Barbara Tampieri, Webmistress
BTDesign Awards

The criteria page is probably the most important element in an award program. Having detailed criteria is surely a sign that the award program is well structured and the awardmaster is willing to offer the best help to the applicant in order to win the award he deserves.

Since some of the criteria can be found in almost every program, especially as far as disqualifiers are concerned, some unexperienced applicants might think all criteria pages are equal and standardized and that it is not necessary to read them if the applicant site does not contain any disqualifier. This is a big mistake which results as the most common cause of failure in gaining the award.

While every program will surely state there will be no awards for porn or hacking or hate sites, this does not mean all other sites will be awarded. Every award program has a peculiarity, looks for certain characteristics in the sites it awards and all this can be found in the criteria page.

If an applicant reads carefully the criteria he will discover not only if his site is eligible in general, but also if he is eligible for that particular award he is applying for and if he can hope to get the highest offered award.

We can say that a certain criteria standardization is typical of award programs in their early stages. The criteria are usually short, concentrating mostly on disqualifiers and on what the awardmaster does not want to see in general.

When the program grows the criteria are usually more and more detailed, a separate page is set for them, and the information contained is aimed to help the applicant finding out not only what he must not show but what he is expected to display in order to be eligible for the awards.

A detailed criteria page is usually the result of the growing awardmaster's experience in reviewing hundreds of sites and can be described as the collection of all the characteristics which made previous applicants winners.

Don Pressgrove, Webmaster
Riki's Perch Awards (program closed)

Standardized criteria by definition implies that once an award seeker gains the experience to earn any award, he or she would be eligible to win them all. It is the diversity each award site brings to the awards community that provides the inspiration and motivation to win the various awards offered.

Each of the better award sites establish criteria based on their individual definition of "web excellence". As award sites gain experience and sharpen their skills, this is often reflected by updated criteria with the result that their award is a little more difficult to achieve. Normally the popularity and desirability of an award is based upon the difficulty to acquire it.

This sets the stage for the award seeker to improve their skill levels to win the increasingly more difficult awards.

At the same time the award givers criteria is in a constant state of flux as new skills are learned and incorporated into their criteria, award program, and accompanying Web site. The end result is beneficial to the entire Internet global community.

As competition and the desire to win an award is directly reflected in the criteria process, this results in a higher level of "web excellence" not only to win an award, but will also promote improvements in the award seekers web site. The cycle is never ending and everyone benefits through this process.

If awards criteria were standardized the incentive to constantly improve your skill level would be diminished, if not totally eliminated.

Jim Docherty, Webmaster
Red Stag Awards

Award programs represent the same diversity in personal taste, design, content and functionality as the sites they evaluate. Standardized criteria would reduce their value to a question of award graphics.

Award-seekers apply to various programs for a number of different reasons. In that quest is an opportunity to qualify for multiple awards based on the unique criteria that exists for each. In short, many sites facing a single set of rules might be excluded from the process that would otherwise benefit from the currently diverse awards experience.

On a practical level, it is unlikely that any one group could achieve a consensus among the hundreds of serious award-givers to standardize program elements including criteria.

Ian McPherson, Webmaster
Best of the Web for MacOS Award (program closed)

To some degree, there is already an "informal" standardisation of award criteria. Certainly, few people write their own award criteria without considering someone else’s approach, or should. And many award programs use similar terminology, expressions and wording, as it makes great sense to agree on accepted terminology, simply to maximise communication.

"Content," "originality" and "excellence" are the meat and potatoes of the awards community, and I consider these terms in particular to be standards. Other criteria, such as "fast loading" and "no under construction signs," which hark from early studies of the web and the attitudes of early awardmasters, could also be considered criteria standards, as they are still featured in many leading programs years after their initial adoption.

In my case, I studied many different programs, and jotted down lots of notes and observations, which later were incorporated or dropped from my own criteria. I could have left out some of the "standardised criteria," but after mulling it over, I ended up incorporating the standards I agreed with. To be absolutely honest, I now find my own criteria to be around 60% derivative (and therefore standardised) and 40% theme-oriented and original. Check it out. See if you agree?

Criteria, however, whilst it may contain some of the previous informal standards, is vital in describing and defining the real purpose of your Award, and should always contain original thinking. A good example is Médaille d'Or, which contains some standard criteria, yet concentrates (and succeeds) in communicating the web philosophy of the reviewers. This unique criteria, and the quality of the site selections, makes this program one of the best in the world.

Purpose-driven award criteria is one of the reasons we enjoy an increasing variety of award programs. There are awards for art, design, Flash, sci-fi, humour, search engine success, speed, horror, pets and more — there is even an award for award graphics! Whatever people are looking for, no matter what their web experience level, they can build an award program around their chosen topic. With fully standardised award criteria, this variety may not be possible.

There is another important reason why there isn’t standardised criteria for awards, and that is because most awards are non-commercial. If the majority of award programs were run for money, there would be more associations, more committees, more rules, more compromises, more standards — and ultimately — less creativity and variety.

As long as awards criteria is not over-standardised, and people are free to choose the topic or passion that is reflected in their award program, variety is assured. My own program grew out of a passion for the Macintosh computer. When I realised there wasn't a Mac-specific award program listed at Award Sites!, I decided to start one. Since then, I have been continually entertained, amused, moved and amazed by the quality and variety of my "made on a Mac" award winners.

I wouldn't like to see that change too much :)