Probing the understanding of menu options

I recently (2006) asked several groups of students how they knew how to find particular menu items in the menu. Some students that I have asked have indicated that they have experienced some logical groupings of the commands, and that helps them. The majority, so far, have "just remembered". My sample size is pretty small, so I'm keen to ask more groups of kids. And one thing that I'll be interested in is whether they find what's in the File menu to be a "logical grouping" - to me, having a working understanding that a program exists within a broader system environment is pretty important (and that the commands to interact with that broader system environment tend to be clustered in the file menu).

I figure that knowing the location of every menu option by rote is ultimately inefficient, and that users should be making some "conceptual" sense of what they find. But what sense, or connections, are they making? And how would these make sense when transitioning into learning a new piece of software?

Here are three questions recently set on a multiple choice test given to approximately 150 Year 8 students:

1. In which menu are you most likely to be able to change the way your document looks?
(a) in the "File" menu
(b) in the "Edit" menu
(c) in the "Format" menu
(d) in the "Window" menu

2. In which menu are you most likely to find the "print" item?
(a) in the "File" menu
(b) in the "Format" menu
(c) in the "Tools" menu
(d) in the "Windows" menu

3. Sue has a set of folders as shown below. Sue wanted to make a new folder titled 'Emails'. Under which menu should she go to make a new folder?
(a) File
(b) Edit
(c) View
(d) Help

What is unusual about the results is that student opinion is in favour of the correct answer for questions 2 and 3 (answer "a" in both cases) but opinion strongly divided among (a), (b) and (c) for question 1. We can then take the view that students would have good reason for believing what they do, therefore, there can be something learnt from probing understanding of question 1.

What I am planning to do is as follows. Gather together 6 to 8 students who are representative of the diversity of opinion on question 1, and have prepared each of these questions on a separate sheet of paper (ie 3 pages for each student).

Stage 1: orientate students to their views by issuing question 2. As them to, privately and personally, commit themselves to a view about what they suggest is the correct answer, and write it down in such a way as no-one else can see it. Then go around the group: each person to state his/her opinion about the correct answer, and why they believe it to be the correct answer. Then ask the students if, by listening to others, they change their view about what might be correct, to go around the group a second time asking if they revised their opinion and why.

Stage 1a: repeat the above for question 3

Stage 2: probe for understanding. Repeat the schedule outlined above for question 1.

What is of interest is "why they believe it to be correct" in the first 'around the group', why they revise their opinion in the second 'around the group' for both stage 1 and stage 2.

wrt question 1 you change the way your document looks by both editing it (eg. copy and paste) and by formatting it. You can also edit your document after opening it through the file menu. So I think I can see why the answer is divided for question 1 between (a), (b) and (c). "... the way your document looks" is a fuzzy phrase.

Web design has now evolved to the point of clear separation of structure (headings, paragraphs, images etc.) and style (margins, colours, fonts, etc.) - even though over 90% of actual pages on the web do not conform to these W3C standards. These things were unclear in the early days of HTML, which initially had lots of formatting tags and attributes many of which are now deprecated.

It's not surprising that users confusing "structural content" (which is edited) and "stylistic appearance" (which is formatted) in Word. I think only those who appreciate the need and reasons for separation of structure, style and behaviour in web pages would be clear about this.

a heading

eg. an <h1> heading can be styled as small in size but still be the most important heading on the page, because it is at the top or has a special colour or background or is centred etc. (Bill June10)

I did a comparison of menu option between MSWord and Open Office Writer.

A couple of discrepancies that stand out:
OO Writer
Page Setup
File > Page Setup
Format > Page
Header & Footer
View > Header and Footer
Insert > Header Insert > Footer
The OO categorisations make more sense to me. Perhaps miscategorisations (or differences of opinion about where things belong) like this are a source of confusion for the user.

But I suppose the striking thing really is how similar the Word and Writer interfaces are. Open Office is mainly mimicking Word. See the interface elements page for links about interfaces that are much more different. (Bill June10)