Montage

Montage is a tool for judging photos in a photo contest, and to select the winners. The Montage tool allows the selection to be completed in a simple manner with a number of jurors. The technical condition is that all jurors have a Wikipedia username. It is best to have at least 5 jury members.

Another tip: use a large flat screen (preferably not a laptop screen) to show the photos in their full glory.

This tool is open source, and has been used thousands of times to judge Wiki Loves contests worldwide and is very easy to use. It is very easy to use, and requires few documentation.

Roles in the jury team

• coordinator: create the campaign, register the organiser
• organiser: create the round, start the rounds, report the results
• juror: select the photo's

Actions for the organiser

(this section is only important for the organiser of the jury campaign; the coordinator can also be juror.)

• Request the creation of the campaign on the platform
• The administrator creates the campaign
• Create round 1
  • Decide the ready date
  • Identify the images
  • Add jury members (include additional blacklist accounts)
  • Create the round
  • Edit the round: remove the blacklist accounts
• Start and close the rounds
• Guides the jury
• Performs the follow-up, the round 2 and 3, and the reporting

Start the application

• Preferably use Google Chrome
• Go to https://nl.wikipedia.org
• Login with your Wikimedia account
• Go to https://montage.toolforge.org
• Click on "Login with your Wikimedia account" (top right)
• You give Assembly permission to use your username
• Check https://www.mediawiki.org/wiki/Special:OAuthManageMyGrants if this is the case.
• Check that your username is on the top right?
• Click on "MONTAGE" at the top left
• Scroll down to the right campaign
• Then click on "Vote" at the bottom right
• You can judge the photos in several sessions. So you don't have to cover the entire list in a single run....
• After you have cast your vote, you can still make changes by clicking on the option "Edit previous votes". Sort by score descending or ascending or by date of voting. Do not forget to save your changes. Click on the top right...

Selection criteria

• It must be about Belgian heritage.
• The licensing conditions must be OK.
• The photo must fall under the Freedom of Panorama or Public Domain (temporary exhibitions are therefore not OK).
• There must be no copyright or watermark.
• The subject must be in full, unless it is a detail.
• The photo must be sharp.
• The composition, colour and lighting must be good.
• The photo must be true to nature (edited as little as possible by e.g. Photoshop).
• The photo must be usable for Wikipedia.

Selection process

Images are distributed amongst the jury members. Every jury members votes on the images he has been assigned to. The next round can only be started when all jury members have voted for each of their assigned images.

• Round 1: Yes/No selection (arrow up/down)
• Round 2: Give points 1-5 (stars)
• Round 3: Rank the top-x (drag and drop)

The next round can only begin once all participants have judged their assigned photos and the coordinator has opened the next selection.Foto's met een minimumscore gaan door naar de volgende ronde.

Round 1

The 1st round is a simple yes/no selection. For each photo you have to click "yes" or "no". You can rate the photos with "yes" or "no" (thumb up/down or arrow up or down).

After all photos have been rated, you can check your choice again via "Edit previous votes", then click on "Order by score descending/ascending". Then you can change your score for certain photos. Then save your changes by clicking "Save changes" at the top right.

Each jury member jurors independently, and some are indeed stricter than others. From experience in previous sessions, the 1st selection is somewhere between 10-30% (based on individual judgement).

The scores of all jury members are added up. So every "yes" counts for 1 point. Suppose the tool is set up in such a way that all photos have to be judged by all jurors. Suppose there are 5 jurors, then there will be photos with 5-4-3-2-1-0 points.

In the 2nd round it can be decided to continue with photos that have got e.g. 4 and 5 scores...

If we only take score 5 into account, this means that only photos for which all jurors have said "yes" will be selected. If we also take into account the score 4, the photos where only one jury member said "no" will also be included. Etc.

If there are a lot of photos to be judged, it is worth having more jurors and allowing each photo to be judged by x of the y jurors. Preferably, each jury member jurors a maximum of 800 photos in the first round. If there are enough jurors, each judge can judge all photos in the first round.

It is worth repeating this process several times. You can then compare the photos better and adjust the selection if necessary. In this way, you can make a balanced and fair choice. Do not forget to save your adjustments.

Use the keyboard

• Another tip to quickly triage photos in the first round (using the keyboard - no need for a mouse -> goes faster).
• Accept the photo: Arrow Up
• Reject picture: Arrow Down
• Rate the photo later: Arrow Right

Round 2

After the first round, we keep about 10-20% of the photos. We typically have about 200-300 photos left to go in the second round. Depending on the number of voters per photo in the first round it is quite possible that you will find photos that you did not select in the first round, because sufficient other member of the jury did find those photos worthwhile.

In the second round, points are given from 1 to 5 (number of stars), covering artistic, composition, quality, originality, etc. Again, each photo is validated by at least 2 people and the points are added up.

After all photos have been rated, you can check your choice again via "Edit previous votes", then click on "Order by score descending/ascending". Then you can change your score for certain photos. Then save your changes by clicking on "Save changes" at the top right. There is a counter that gives the number of photos for which you have changed the score.

3 more important notices:

• To see the name of the photo "mouse over" at the bottom of the image
• Don't forget to click on "Save Changes" at the top right... otherwise your choice is not saved
• To check or change your vote, click "Edit previous votes".

Round 3

A global ranking top-x is created. Procedure: click on "Vote" or "Edit Previous Votes" and drag the photos (up or down) until the best photos are on top. Drag and drop the photos to rearrange their order... Hold the photos by the = sign at the top left of the photo. The best way is to compare two neighbouring photos each time and bring up the better one. You can go through the session several times (via "Edit score"). Always save your choice (click on "Saves Changes") at the top right.

If some photos are not visible, please press "Refresh" (F5) until all photos have been downloaded. Click on a photo to see it enlarged. Press "Esc" to go back to the selection.

It is worthwhile repeating this process several times, possibly every other day. This way you can better compare the photos and if necessary change the order. In this way, you can make a balanced and fair choice. Do not forget to save your adjustments each time.

Motivation

And finally, the jury briefly describes the qualities of the winning photos via a shared Google Docs document.

Award ceremony

The jury will be invited to the award ceremony.

Accessing the tool

• Voting
• Documentation
• Requesting a campaign
• GitHub source code
• GitHub issue tickets