Instructions

Click on the links on the top left to navigate to different random elements of the debates. The left half of the screen shows an original document of a debate (or a page thereof), and on the right you can see the processed version of the same document, or an utterance within it. If you refresh the browser, a new random document will be presented.

Full session and Single page are self-explanatory. Utterance, President and Orador present a single utterance, but in Presidnt the speaker is always the President of the Assembly and in Orador the speaker is someone who was given the floor, started speaking, was interrupted, and is identified only as "Orador" in subsequent utterances, until the President grants the floor to another speaker.

Note: There are three utterances in all utterance views, to provide context. The utterance you are evaluating is the one in the middle, between the two lines of dashes ('----------').

Note: If you see the text "Utterance does not exist in this session", this means that particular type of utterance (e.g. Orador) is not present in that document. It's not an error, just hit refresh and move to the next document.

Items

Header Detection

Evaluate the detection of page headers. Headers are the first line of the page, showing information like page number, date and session identification. They must be eliminated so they don't get included in speaker utterances.

• True Positive: A header is present in the original document and correctly removed from the processed version.
• True Negative: A header is not present in the original document and the text in the processed version is intact.
• False Positive: A header is not present in the original document and valid text is missing in the processed version.
• False Negative: A header is present in the original version and it is still present in the processed version.

Session End

Evaluate the detection of the end of session. The session ends on the last utterance, so this task consists of checking whether there are unidentified utterances in the end of a document (i.e. something like "O Sr. Presidente: - Blah blah blah", indicating that someone spoke)

• True Positive: The last utterance is correctly identified.
• False Positive: There are utterances after the last identified utterance.

Utterance Tagging

Evaluate whether utterances were correctly tagged.

• True Positive: An utterance is present in the original document and tagged as such in the processed version.
• False Positive: An utterance tag in the processed document that does not correspond to a real utterance in the original document.
• False Negative: An utterance is present in the original document and not tagged as such in the processed version.

President Identification

Evaluate whether utterances of the President of the Assembly are correctly attributed. Utterances from the president have the attribute role, with value "president".

• True Positive: An utterance from the President is present in the original document and tagged as such in the processed version.
• False Positive: An utterance is attributed to the president in the processed document but it was spoken by someone else.
• False Negative: An utterance from the president is not attributed to him in the processed version.

Orador Assignment

Evaluate whether Orador instances are correctly attributed. This involves going back to the utterance where the President granted the floor and verifying that the speaker identified as Orador (attribute putative_speaker in the processed document) is indeed the Orador.

• True Positive: An Orador utterance attributed to a speaker was indeed uttered by that speaker.
• False Positive: An Orador utterance is attributed to the wrong speaker.