The semantics of annotations

For the moment we'll think about text corpora. In roughly increasing order of cost you could decide to:

• Leave the text which you collected entirely unchanged.
• Correct obvious spelling errors. This is already a matter of judgement. It should be possible to recover the original text just in case you or anybody else later disagree with yourself about the alleged error.
• Mark parts of speech. You definitely need explicit guidelines about which parts of speech are available and how to tell the difference between them. Examples are essential but by no means sufficient.
• Mark the boundaries of constituents, but don't give the constituents names. It will become obvious that you need an agreed mechanism for being vague about things that the annotator is genuinely unsure about. (One which bit me was ``go out into the wilderness'' where it isn't easy to see how closely related the prepositions are to the verb).
• Mark full syntactic structure. In this case you need to convey to the annotators a common set of necessary and sufficient conditions for every possible construction. This is really hard.
• Mark word senses (but because you don't want to say that there is a sense of ``throw'' which means ``behave in an uncontrolled manner'' just because you can throw a wobbly or throw a fit, you may need to be explicit about exactly what sort of things can be annotated for word sense).

As you add more annotation you may feel the temptation to destructively edit the original input text. Don't ever do this, since it diminishes the value of your corpus. Always prefer to add an annotation meaning like:

On 24th October Chris Brew doesn't agree that the span from 321 to 324 is a complement, believes it to be an adjunct and marks the verb at 320 to 321 as having sense 5. If you really think its a complement I have no idea what sense the verb is.

If that seems verbose and pedantic, it is, but it is far more satisfactory than silently editing away the supposed mistake.